How should ambiguous evidence and justified escalation be scored?
Ambiguous cases need explicit evidence boundaries so evaluators can reward justified escalation without accepting vague refusal or unnecessary requests for more information.
**Should escalation count as success?**
It can, when the supplied evidence does not support an authorised correction and the agent identifies the unresolved issue precisely. The score should depend on what the agent establishes and requests, not on the presence of the word escalation.
**What makes an escalation useful?**
The agent should identify the conflicting records, explain how the conflict affects the proposed adjustment and direct the issue to the role able to resolve it. If a service completion record disagrees with a cancellation record, the next step might be confirmation from the service administration team before changing the charge.
**How do we avoid rewarding excessive caution?**
Include cases where the answer is already available in an accessible record. An agent that asks another team to supply information it could retrieve itself has not demonstrated the same judgement as one that reaches a genuine evidence boundary.
**What belongs in the scoring guide?**
List the unresolved proposition, the evidence already available, the permitted interim state and acceptable resolution requests. This makes escalation assessable without insisting on a single wording.
Distinguish a useful decision to defer from an agent's failure to investigate available evidence.
- Audience:
- Evaluation teams
- Content type:
- FAQ
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How can developers evaluate a correction beyond checking the final balance?
- What should buyers inspect before trusting a generated correction pack?
- How should an agent behave when a posting request times out?
- How do we distinguish a justified pause from an agent avoiding the task?
- How should evaluation hooks avoid leaking the expected correction to the agent?
- Which hooks reveal duplicate posting after a tool timeout?