What trace format and grader hooks make intake runs easy to score?

Score runs from two artefacts: the final case state and the event trace.

**Trace record (JSONL, one event per line):** `run_id`, `task_id`, `case_id`, `step`, `actor`, `action`, `args`, `result`, `at`. Include read actions such as `lookup_party` and `search_cases`, since missed lookups are a process signal.

**Graders as separate functions:**

- `grade_outcome(final_state, key)`: compares status, queue and outstanding items. Returns pass or fail, with a reason code such as `wrong_queue` or `false_complete`. - `grade_process(trace, key)`: checks that each entry in `expected_checks` appears before `route_case`, and flags forbidden actions. - `grade_links(final_state, key)`: checks duplicate and prior-case links the agent was expected to set. - `grade_note(note, record)`: checks handover claims against record fields. This can be rule-based first, with model-based grading only for open text, and a human audit of a subset.

**Reset.** Restore `cases` and `events` from a snapshot identified by a `snapshot_id` and a seed, then verify a hash of the starting tables before each task. Store the hash in the run record.

**Splits.** Request a development split with keys visible and a held-out split with keys withheld, so tuning does not contaminate the test.

Ask a pilot to supply reference grader code, so your harness and the provider's keys agree on reason codes from the first run.