Skip to content
MirrorFirm

How should I inspect the quality of generated schedule state?

Quality can be inspected through arithmetic invariants, cross-record consistency, policy-to-outcome agreement, variety across cases and a separate review of grounding and disclosure risk.

Start with checks that need no model at all. Schedule data is unusually easy to test because it must obey arithmetic and ordering rules.

**Arithmetic invariants.** Instalments should sum to the plan principal plus fees. Allocated payments should not exceed amounts received. Balances should equal the sum of what remains.

**Ordering and dates.** Sequence numbers should match due-date order. No instalment should fall after the plan end date, and no due date should fall before the plan start.

**Link integrity.** Every foreign key should resolve. Every event should reference records that exist in the before and after states.

**Policy agreement.** Take recorded actions and test whether they were permitted under the policy version in force. Approvals should appear where thresholds require them.

**Variety.** Look at the spread of plan lengths, amounts, statuses and exception types. A set where every plan is regular will not test anything.

**Grounding and disclosure.** Because the output is generated, ask how its structure was informed and review whether any record looks like a copy of a real source. Do not assume generation removes that risk.

Run the first four as scripts on the sample, then run a baseline agent to see whether tasks are solvable and whether the grader can tell good from poor behaviour.

Most quality problems can be found by automated checks on the records themselves, before any agent is run against them.

Audience:
Environment builders
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Repayment scheduling