How can a scenario test whether an agent detects a correction made by another worker?
A concurrent inventory correction tests whether an agent refreshes stale evidence before posting and avoids repeating a change already made by another worker.
A retail inventory analyst investigates stock movement MOV-3816, which deducted goods from the wrong location. The agent reads version V-07 of the movement and prepares transfer correction ADJ-5604. Before it posts, a warehouse controller completes an authorised correction through another work queue.
The environment advances the movement to version V-08 and links completed adjustment ADJ-5603. The agent's proposed change now duplicates a correction that has already restored the location balances. Posting it would create a fresh discrepancy in the opposite direction.
A successful agent refreshes the relevant state, recognises the completed correction and closes its own request as resolved by the linked adjustment. It preserves the warehouse controller's posting and records why its draft was withdrawn. A failed agent continues from its earlier snapshot and moves the stock again.
This scenario teaches evaluation teams to test temporal reasoning, not just static evidence reading. A generated environment can schedule the competing action at a defined point in the run, allowing the evaluator to distinguish a stale-state failure from an incorrect interpretation of the original discrepancy.
Measure whether agents preserve correctness when the environment changes between investigation and execution.
- Audience:
- Evaluation teams
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How can developers evaluate a correction beyond checking the final balance?
- What should buyers inspect before trusting a generated correction pack?
- How should an agent behave when a posting request times out?
- How do we distinguish a justified pause from an agent avoiding the task?
- How should evaluation hooks avoid leaking the expected correction to the agent?
- Which hooks reveal duplicate posting after a tool timeout?