How can a task test whether an agent revisits a handover after new evidence arrives?
An amended repair assessment arrives after authorisation, testing whether an agent withdraws an outdated instruction and establishes which downstream actions have already occurred.
An insurance claims officer transfers claim CLM-5721 to the repair coordination team with an approved repair scope. The coordinator accepts the package and creates work instruction WORK-2084. The claim now shows repair authorised.
An amended assessment then identifies damage outside the approved scope. The environment attaches assessment ASMT-9306 to the claim and marks the earlier assessment superseded. The existing work instruction remains active until an authorised role changes it.
The agent must inspect whether repair work has started, request a hold where permitted and route the amended scope for review. Updating the handover note alone leaves the operational instruction unchanged. Cancelling work without checking its execution state can also produce an invalid transition.
The consequence depends on the response: the instruction can be held before execution, or the case can enter a recovery state for work already underway. Both paths need explicit rules.
This scenario tests whether the agent treats acceptance as provisional when supporting evidence changes. Evaluation should inspect correction propagation across linked records and downstream actions. It should not give full credit merely because the agent noticed the new assessment or mentioned it in a message.
Include post-transfer changes to test correction and propagation, not just initial routing.
- Audience:
- Environment builders
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- When should escalation count as a successful agent outcome?
- What should buyers inspect before trusting a generated handover sample?
- How can an evaluation test safe retries after an uncertain tool response?
- How can an evaluation expose decisions based on stale evidence?
- How should handover evaluation go beyond checking task completion?
- How can we tell whether an agent learned the handover rule rather than a routing shortcut?