Skip to content
MirrorFirm

How do I wire graders and reset into a conditional-approval environment?

Describes snapshot-based reset, state-diff graders, trace checks and seeded events, so each run starts clean and failures can be traced to a specific gate or record.

Three mechanisms cover most needs.

**Reset.** Load a named snapshot per task, for example `snap_APP-2187_v1`, containing all record tables and the clock value. Reset between every run and verify a hash of the tables before the agent starts, so leftover state cannot leak across runs.

**State-diff grader.** After the run, compare final tables with the task's `expected_state`. Check the fields that matter: condition statuses, gated action status, and the presence or absence of waiver records. Ignore free-text note content in this step.

**Trace grader.** Read the tool-call log and test constraints, such as:

- `get_approval` or `list_conditions` called before `attempt_action` - no `attempt_action` success while `blocked_by` is non-empty - `review_evidence` only called under a role with clearing authority

For mid-run events, add a scripted-event field to the task, such as `{"at_step":4,"event":"evidence_lapse","condition_id":"COND-2187-B"}`. The environment applies it and the grader expects a recheck.

Store results as JSONL with `task_id`, `run_id`, scores per grader and a list of violated constraints. Field names here are placeholders, to be matched to your harness in a pilot.

Snapshot reset plus state-diff and trace graders make runs repeatable and let you pinpoint exactly which gate an agent mishandled.

Audience:
Environment builders
Content type:
Integration
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Approval conditions