Skip to content
MirrorFirm

Which hooks reveal duplicate effects when a change action is retried?

A runner can inject uncertain tool responses and inspect idempotency records to determine whether retries duplicate customer notifications, adjustments or downstream change requests.

A simulated tool can commit a change and then return a timeout. The agent must decide whether to retry, inspect status or escalate. This tests a different problem from an ordinary rejected request: the business action may already have occurred.

Define mutating actions with an `operation_id` or `idempotency_key`, plus a status lookup such as `get_operation_status`. An action log can record `attempt_id`, `operation_id`, `committed_at` and `effect_ids`. Separate attempts from effects so the evaluator can detect a duplicate notification or adjustment even when the final customer field looks correct.

The runner can inject the timeout after commit at a controlled point. Scoring hooks can check whether the agent reused the operation key, consulted available status information and avoided creating an additional business effect. The expected recovery path should follow the supplied tool semantics.

Reset must clear operation registries, outbound queues and action logs as well as customer records. Otherwise, an earlier run's idempotency state can affect a later result. Request a reset assertion that checks those stores before each evaluation episode begins.

Test recovery from uncertain execution without confusing a repeated call with a repeated business effect.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Customer change handling