How can a scenario test whether an agent handles another worker's concurrent change?
A concurrent scheduling edit tests whether an agent detects stale state, rereads the record and preserves a colleague's valid work during recovery.
A patient asks a scheduling agent to move appointment APT-0412 to an afternoon session. The agent reads version 7 of the booking. Before it writes the new time, a reception coordinator updates the accessibility requirement, producing version 8.
The environment rejects the agent's write because its expected version is stale. The agent must reread the booking, retain the coordinator's accessibility update and recheck whether the requested session still satisfies the booking constraints. Resubmitting the entire old record would erase valid work.
A generated evaluation can place the coordinator's action at a defined point between the read and write. The scoring criteria can inspect whether the agent recognised the conflict, preserved the intervening change and reached a valid scheduling outcome.
This teaches a specific recovery behaviour: a conflict is a reason to reassess current state, not merely repeat the previous command. Developers can compare an ordinary task with the interleaved version to isolate whether the agent's failure comes from scheduling reasoning or from its handling of concurrent work.
Use controlled interleaving to reveal lost updates that ordinary sequential tasks cannot expose.
- Audience:
- Agent developers
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How can buyers check whether an agent understands changes rather than memorising wording?
- What should buyers inspect before trusting a generated change evaluation set?
- Which evaluation hooks expose retry and duplicate-processing failures?
- How should evaluation distinguish complete handling from a correct-looking final record?
- How can buyers evaluate correctness without requiring one exact action sequence?
- What should buyers inspect before trusting a generated evaluation pack?