Skip to content
MirrorFirm

How can evaluators recognise valid corrections without requiring one exact action sequence?

Outcome constraints and preserved invariants allow evaluators to recognise multiple valid correction paths while detecting collateral changes hidden by a correct closing balance.

Start with the properties that must hold after a valid correction. These can include the intended account allocation, preservation of the original source entry and a complete link between the correction and its supporting decision. Then specify what must remain unchanged, such as unrelated customer balances or supplier payment instructions.

This supports evaluation of more than one valid execution path. An agent might gather supporting records in a different order or prepare a draft before requesting approval. Those differences need not count as failures if the resulting state and required controls are correct.

Keep forbidden side effects separate from the main outcome. An agent that fixes the target balance while modifying an unrelated account should fail that constraint even if its target arithmetic is exact. A single aggregate score can otherwise hide a serious boundary error.

Generated cases can include evaluator-only invariants and a state-difference report. Reviewers can inspect which properties passed, which failed and what changed unexpectedly. This approach also makes disputed scores easier to diagnose because the judgement points to a business requirement rather than an arbitrary reference sequence.

Score the meaning and boundaries of the final state rather than matching a single reference trace.

Audience:
Evaluation teams
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Ledger adjustment