How can an environment test safe recovery from an uncertain posting response?
A posting timeout can leave the agent uncertain whether a change committed, making status checks and idempotent retry behaviour observable evaluation targets.
A simulated posting tool can accept `request_id`, `idempotency_key` and `expected_record_version`. Its response contract can distinguish rejected requests from responses lost after a successful commit. This creates a recoverable uncertainty that cannot be resolved from the timeout message alone.
Expose a `get_posting_status` action that searches by the original idempotency key. An agent can use that action before deciding whether another submission is needed. Reusing the same key for the same operation can return the existing result; using it with changed content can produce a defined conflict.
Evaluation hooks can inspect committed adjustment identifiers, submitted keys and the agent's recovery sequence. The success condition includes a single intended business effect and a correctly resolved case status, rather than merely a final successful tool response.
Reset must restore both ledger state and the idempotency registry. Otherwise, a later run may inherit a prior posting result. A buyer can ask for failure injection controls that reproduce the lost-response condition so different models face the same recovery problem.
Measure recovery behaviour when transport failure and business failure are different events.
- Audience:
- Model labs
- Content type:
- Integration
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How can developers evaluate a correction beyond checking the final balance?
- What should buyers inspect before trusting a generated correction pack?
- How should an agent behave when a posting request times out?
- How do we distinguish a justified pause from an agent avoiding the task?
- How should evaluation hooks avoid leaking the expected correction to the agent?
- Which hooks reveal duplicate posting after a tool timeout?