How do I wire evaluation hooks to an intake task run?

Evaluation works best when the grader reads structured output rather than prose. A task run produces two artefacts: the final environment state and an ordered action trace.

**Final-state checks.** Compare `case.status`, `case.queue_id`, `case.links` and any created `information_request` records with the `reference` block. Use exact match for status and queue, and set comparison for requested items.

**Transition checks.** Validate each step against a transition table stored as rows of `from_status`, `to_status` and `required_checks`. Any move made without its required check results is flagged.

**Note checks.** Score the handover note for named missing items, cited `rule_id` values and the correct addressee role. Use rule-based matching first, and a model-based judge only for what rules cannot score, with the judge prompt kept alongside the results.

**Trace metrics.** Record step count, tool errors, unnecessary calls and whether the agent read the relevant policy before acting.

**Output format.** One JSONL line per run with `task_id`, `seed_id`, `model_ref`, per-criterion results and the trace. Aggregate in Parquet for slicing by criterion, queue or rule.

**Reproducibility.** Seeds, policy versions and tool schema versions are recorded in the run line, so a later run can be compared fairly.

A pilot can supply grader code and schemas, or only the reference blocks if you prefer your own harness.