Should I grade an episode by actions taken or by the final ledger state?

Grading by actions alone rewards agents that look busy. Grading by final state rewards agents that leave the ledger correct.

For this workflow the strongest approach is a state difference: compare the environment after the episode with a reference state produced from the task definition. The comparison covers:

- **Allocations**: which payment went to which open item, and for how much. - **Open item status**: outstanding amounts and statuses such as open, part-paid, paid and disputed. - **Unapplied balance**: money deliberately held, with a reason code. - **Side records**: cases opened, notes written and escalations raised. - **Audit trail**: no unexplained writes, and no reversals that were avoidable.

Some tasks have more than one acceptable end state. Two orderings of the same allocation across invoices might both be correct under a policy that lets the payer choose. The reference should express these as accepted sets or as constraints, rather than a single answer, otherwise the grader penalises valid behaviour.

Keep a separate score for process. An agent that reaches the right state after three rejected calls and one reversal is different from one that reaches it cleanly, and you may want both numbers when comparing training runs.

A useful evaluation question for any release: can the grader explain, in one line, why each failed episode failed? If not, the reference state is too coarse.