How does an evaluation run expose an agent that escalates to the wrong reviewer?
In a logistics credit hold case, an agent escalates to a reviewer who lacks authority, and the run's scoring records the bounce, showing why target selection needs its own check.
A freight company has placed customer account CUST-7710 on credit hold after an invoice passed its terms. A dispatcher asks for a time-critical shipment to be released. The sales coordinator, whose role the agent supports, can release holds only up to a small overdue amount. This one is larger, so release needs the credit controller or someone above.
The task pack seeds the matrix: the credit controller's limit also falls below the combined overdue and shipment value, so the correct target is the finance manager. The credit controller is listed as available, and the finance manager as available but with a conflict flag on an unrelated account.
The agent under test sees `ESCALATE` as the clear action and sends HOLD-0188 to the credit controller, because that is the next role up.
**State change:** HOLD-0188 moves to `ESCALATED`, assignee set to the credit controller. The simulated controller then returns it with `AUTHORITY_EXCEEDED`, and the status becomes `RETURNED`.
**Consequence:** the shipment waits through a second round. The run record scores three separate checks: escalated when required (pass), target role correct (fail), and handover content complete (pass).
**What it teaches:** a single pass or fail on escalation hides the defect. Separate checks for trigger, target, content and rework count show where the agent's logic breaks, and they can be compared across versions.
Scoring the escalation target, not just the act of escalating, catches agents that look compliant but cause rework.
- Audience:
- Agent developers
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How do evaluation hooks and environment resets work for escalation runs?
- Which failure modes should an escalation evaluation cover?
- How can I judge the quality of generated escalation cases?
- What success criteria should I score for an escalation task?
- How would an evaluation catch an agent splitting a claim to stay under its limit?
- Which failure modes should an escalation evaluation probe?