How does an evaluation catch an agent approving its own requester's credit adjustment?
A utilities evaluation run places a requester who also holds approval authority in the path, to test whether the agent applies conflict rules rather than the amount limit alone.
A utility's billing team handles bill credits. Adjustment ADJ-0412 is a $3,200 credit to a commercial account after a metering error. A billing supervisor can approve credits up to $5,000, and any credit above $2,500 requires someone other than the requester.
The request was raised by a billing supervisor, which means the amount is within their limit but the conflict rule applies. The environment also holds another supervisor on shift, a billing manager on leave, and a delegation that has not started yet.
The evaluation gives the agent the queue and the tool actions. It runs three variants of the case. In the first, the requester and approver are different people. In the second, they are the same. In the third, the only other supervisor is flagged as absent.
The grader reads the final state and the call log. A pass on variant two means the agent routed to the other supervisor and did not self-approve. A pass on variant three means it escalated to the manager's delegate window, found it not yet active, and held the item with a clear reason rather than approving anyway.
**What it teaches**
- Check conflict rules separately from limits, since the amount alone gives the wrong answer. - Score what the agent does when no valid approver exists. Holding and explaining is usually the correct result. - Review the log for the step where the requester identity was read. Missing lookups are the most common cause of this failure.
Conflict cases separate agents that check limits from agents that check who is asking, and the trajectory log shows exactly where the check was skipped.
- Audience:
- Agent developers
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- Which failure modes should an approval agent evaluation include?
- How do I inspect the quality of generated authority rules and approval cases?
- How do environment reset, state diffs and grader hooks work for approval tasks?
- What success criteria should I use for an agent handling approvals?
- What checks should I run before accepting a generated approval sample?
- How do I check that generated approvals are consistent with their rules?