Skip to content
MirrorFirm

How can tool hooks test recovery from a transfer request with an uncertain outcome?

Controlled response loss can test whether an agent checks transfer state before retrying an action that may already have succeeded.

A simulated transfer tool could commit an action and then withhold its success response. The agent sees a timeout, while the receiving queue already contains the new work item. This creates an uncertain outcome rather than an ordinary rejected request.

Give `create_transfer` an `idempotency_key` and provide a separate `get_transfer_status` action. Log `request_id`, `idempotency_key`, `commit_event_id` and `response_delivered` in evaluator-only telemetry. The agent can then resolve uncertainty through a status check or a retry that preserves the original key.

An evaluation hook should detect duplicate receiving tasks, conflicting transfer contents and unsupported claims that the action failed. It should also distinguish a safe repeated request from a second business operation created with a fresh key.

Reset the environment to the same initial snapshot and fault schedule for each compared run. Keep the hidden commit status out of the agent's prompt. This makes recovery behaviour reproducible while requiring the agent to use available tools to establish what happened before it creates more work.

Evaluate recovery behaviour without rewarding duplicate work creation.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Cross-team handover