Skip to content
MirrorFirm

How can developers evaluate a correction beyond checking the final balance?

Evaluation should inspect business invariants and the permitted set of changes, because a correct final balance can conceal damage elsewhere in the records.

A final balance can be correct for the wrong reasons. An agent might remove an unrelated entry, alter a source amount or create offsetting errors that happen to cancel. Evaluation needs to inspect the allowed change set as well as the requested outcome.

For a financial posting task, the supplied accounting model can require balanced journal components and unchanged source documents. For an operational quantity correction, the invariant might require total stock across locations to remain constant. These are task-specific rules, not assumptions to apply indiscriminately across domains.

A generated evaluation pack could define protected records, permitted fields and expected relationships between the opening and closing state. It can compare the full state difference against those constraints. A successful run would resolve the target discrepancy without modifying unrelated customer accounts, allocation dimensions or historical evidence.

MirrorFirm could include evaluator explanations showing which invariant each check protects. Buyers can inspect whether the checks match the intended business process and whether legitimate alternative solutions are accepted. This approach is especially useful for agents with broad tool access: it measures the boundaries of their actions, not just whether they eventually produce the requested number.

Catch compensating errors and unintended mutations that an end balance check would miss.

Audience:
Agent developers
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Ledger adjustment