How should evaluation hooks, state snapshots and environment reset be wired for scheduling runs?

A scheduling run is easiest to grade when the environment follows a fixed lifecycle.

1. **Reset.** Call `reset(task_id, seed)`. The environment loads the starting snapshot named in the task, sets the as-at clock and clears any prior run state. The same task and seed must always produce the same state. 2. **Run.** The agent calls tools. The environment logs each call with arguments, result, and a monotonic step number. Time only moves when the task says so. 3. **Snapshot.** At termination, or when the agent calls `finish`, the environment writes a post-run snapshot of schedules, lines, events, notes and approvals. 4. **Diff.** A diff step compares pre and post snapshots and emits a structured change list, such as `line L2: pending -> merged`, `schedule SCH-5521: v1 -> v2`. 5. **Grade.** Graders consume the diff, the task's expected end state and the policy references, and write a result record.

A result record might hold: `task_id`, `run_id`, `seed`, `outcome_action`, `state_match` (boolean plus tolerance used), `policy_violations` (list of policy IDs), `side_effects` (list), `message_figures_match` and `steps`.

Store results as JSONL so they can be aggregated across models and tasks. Version the grader code and record its hash in each result, so a score can be traced to the exact check that produced it.

Whether the harness runs in-process, as a container or behind an MCP server is a deployment choice to settle in the pilot.