Which reset, logging and grading hooks does an evaluation harness need?

An evaluation harness for this workflow needs four hooks: reset, step logging, snapshot and grade.

**Reset.** Each episode starts from a named seed state loaded from the release files, for example `seed_id: S-0042`. Reset should be deterministic: loading the same seed twice produces identical entity ids and timestamps. Ask for a reset that also clears queued notifications and task timers.

**Step logging.** Record each tool call as a JSONL line with `episode_id`, `step`, `tool`, `args`, `result_status` and `changed_ids[]`. This supports trajectory analysis and lets you replay an episode.

**Snapshot.** A read-only call returns every entity after the episode. A differ then produces `created`, `updated` and `deleted` sets by id.

**Grade.** A grader compares the diff with `expected_state.jsonl` and the rule file, and emits a result such as:

- `required_ok`: all required records and links present - `forbidden_hit`: writes outside the allowed set - `rule_violations[]`: rule ids breached - `referral_correct`: whether stopping was the right call

Keep graders pure functions of the snapshot and the rule file so results are reproducible. If you also want a model-based judge for free-text outputs such as handover notes, run it as a separate score and keep it out of the state-based pass criterion. Confirm hook signatures and file layout against your own harness during a pilot.