Skip to content
MirrorFirm

How can a grader catch a quiet error in a premium instalment change?

A scenario where an insurer's agent changes a premium instalment plan correctly on the surface, and a state-diff grader finds an unreversed late fee that a text check would miss.

A home insurer's environment contains policy `POL-2204` on monthly premium instalments. The March debit failed, a late fee `FEE-871` was applied, and the customer asks through `CASE-1175` to move the due date to the 20th. Company policy says the fee must be waived once, on first failure, if the date change is accepted.

**The agent's run:** it reads the case, moves the due date by creating schedule version 3, updates the debit instruction and writes a polite note saying the change is complete. A reviewer reading the transcript would call it a success.

**What the grader checks against state**

- schedule version 3 active, version 2 superseded: pass - instruction date matches version 3: pass - `FEE-871` status should be `waived` with reason code `first_failure`: **fail**, it is still `charged` - case note cites the waiver: **fail**

The state diff shows the fee untouched. Left alone, it would appear on the next statement and prompt a complaint.

**What it teaches**

- Grade the resulting state, not the agent's description of it. - Include policy-driven side effects in the expected end state. - Report failures by check, so a team can see that the agent handles dates well but misses fee rules, and target training data at that gap.

State-diff grading finds side-effect errors that look fine in the agent's own summary, which is where schedule agents most often fail.

Audience:
Agent developers
Content type:
Scenario
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Repayment scheduling