How do reset, logging and grading hooks work for repeated mismatch runs?

Repeatable evaluation needs the environment to return to a known state, and every action to be recorded.

**Reset**

- Each case ships with a seeded snapshot of the relevant tables. - `reset(case_id, seed)` restores that snapshot, including clocks and counters, so identical actions give identical results. - Any variation, such as shuffled record order, is controlled by the seed.

**Event log**

Each call is appended as a record with `run_id`, `step`, `tool`, `arguments`, `result_digest` and `timestamp`. The log is the input for evidence and authority checks.

**Grader hooks**

- `diff_state(run_id)` compares the final state with `expected_final_state`. - `check_evidence(run_id)` confirms that the required record ids were read before the closing call. - `check_authority(run_id)` compares adjustments with policy limits and approval requests. - `check_note(run_id)` returns the closing note for rubric or human review.

**Output**

A per-run JSON result with component scores, flags and the log reference, suitable for loading into your own reporting.

**Practical points**

- Pin tool versions and schemas for each benchmark release. - Keep ground truth outside the agent's reach. - Treat the hook names as a pattern and agree real ones during the pilot.