How can a simulated interruption reveal duplicate-action failures?
A lost tool response tests whether an agent checks existing state before repeating a customer change that may already have created a service order.
A utility customer requests that a planned meter appointment move to a different access window. An agent calls `reschedule_visit` for visit VIS-2187. The environment commits the revised visit and creates work order WO-0412, but the response is interrupted before the agent receives confirmation.
The visible conversation now contains uncertainty, not proof of failure. Repeating the action without checking can create another work order or trigger a second notification. The correct recovery depends on querying the request status or using a stable operation identifier recognised by the environment.
A generated evaluation can inject the interruption after commitment and inspect what the agent does next. Success requires one effective reschedule, no duplicate work order and a customer response supported by recovered state. A comparison case can interrupt before commitment, where a retry is still necessary.
For licensed resellers, the interruption point is essential evaluation metadata. Keep it outside the agent-visible input while preserving it for scoring. This scenario teaches downstream buyers to assess recovery behaviour against actual side effects, rather than treating every tool error as evidence that nothing happened.
Resellers can offer recovery tasks that expose duplicate side effects hidden by ordinary successful tool responses.
- Audience:
- Licensed resellers
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How can a scenario test whether an agent handles another worker's concurrent change?
- How can buyers check whether an agent understands changes rather than memorising wording?
- What should buyers inspect before trusting a generated change evaluation set?
- Which evaluation hooks expose retry and duplicate-processing failures?
- How should evaluation distinguish complete handling from a correct-looking final record?
- How can buyers evaluate correctness without requiring one exact action sequence?