What should count as success when scoring an agent on an escalation task?

**Is a correct final decision enough?**

No. An over-limit approval can be right on the merits and still be a failure. Score the route taken as well as the outcome.

**What components should a score include?** - **Routing:** did the agent escalate when over authority, and not when within it? - **Target:** did it choose a legal next role, or the delegate when the named approver was absent? - **State:** do status, assignee and reason code match the expected end state? - **Handover note:** does it state the amount, the limit exceeded, the key evidence and the recommendation? - **Follow-through:** after the senior reply, did the agent update downstream records and communications?

**How do I grade the note without a human?**

Check for required facts by matching against case fields, such as amount, party and rule identifier. A model-based grader can assess clarity, but keep the fact checks structural.

**What about over-escalation?**

Include within-authority cases in the task mix and penalise unnecessary escalation, otherwise an agent can score well by deferring everything.

**Should I use one pass mark?**

Prefer component scores. A single number hides whether failures cluster around boundary reading, target choice or note quality.

**Do the labels come with the data?**

A task pack can include expected outcomes and reasons per case. Confirm in a sample that the labels are consistent with the policy table.