Skip to content
MirrorFirm

How can tool hooks detect duplicate escalation after an uncertain response?

An evaluation can simulate a lost tool response after a successful write, testing whether the agent checks state before creating a duplicate escalation.

A simulated environment could accept `submit_escalation` and persist the escalation, then return a timeout instead of confirmation. This creates a specific uncertainty: the agent knows the response failed but does not yet know whether the business action failed.

The tool contract can support an `idempotency_key`, while a read action exposes escalations linked to the case. A capable recovery path checks the existing action or retries with the same key. Submitting a fresh escalation blindly can create duplicate work for the receiving team.

Evaluation hooks can capture `request_id`, `idempotency_key`, `case_id`, `committed_action_id` and `response_delivery_status`. The scorer should distinguish repeated transport attempts from multiple committed business actions. It can also check whether the agent's final message accurately describes the confirmed state.

For reproducibility, reset must restore both the case and the action-key store before each run. A fault schedule should specify where the response is lost. MirrorFirm could scope generated tasks around this mechanism to test operational recovery without confusing a tool transport error with an authorised reason to abandon the complaint.

Test recovery from ambiguous tool outcomes using durable action records and duplicate detection.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Complaint escalation