How can a scenario expose stale policy decisions during concurrent work?
A credit officer reads an application before a policy update, then submits a decision after the governing state has changed.
A credit officer opens application APP-2187 and receives case revision REV-0042 with policy version POL-0018. Before the officer submits the decision, a policy administrator activates POL-0019 and the environment marks the application for reassessment under that version.
The officer's agent submits its original decision with the revision it read. The decision tool rejects the write with a stale-context result and returns the current case revision. The application remains undecided, preserving the opportunity for a valid reassessment.
A successful agent reloads the application, retrieves the changed requirement and checks whether its earlier evidence still supports the decision. It then submits a decision tied to the current revision or requests the missing evidence. Repeating the original submission without inspection should not advance the case.
MirrorFirm could generate this interleaving as a controlled evaluation episode. The important evidence is the recovery sequence, not merely the eventual outcome. Builders can check whether the agent recognised the conflict, reconsidered the relevant rule and avoided overwriting state produced by another actor.
Test whether agents recover from stale reads by reassessing evidence instead of repeatedly submitting an obsolete decision.
- Audience:
- Environment builders
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How can a buyer inspect business grounding and disclosure risk in generated material?
- What evaluation hooks help test recovery from partial policy-update failures?
- How should evaluation distinguish a correct final state from a correct agent process?
- How can a policy-change scenario test whether an agent respects concurrent edits?
- Which checks reveal whether an agent understands applicability rather than memorising cues?
- How do we evaluate whether an agent changed the right work?