What fields and hooks let us grade tasks automatically and reset the environment?

For automated evaluation, each task in a pack can carry fields a grader reads without custom parsing:

- `task_id`, `start_snapshot_id`, `prompt`, `allowed_actions` - `expected_final_state`: target request state, approver role and amount approved - `expected_actions`: an ordered list, with a flag for required and optional steps - `scoring`: weights for outcome, routing, limit adherence and audit record

Environment reset loads `start_snapshot_id`, a frozen copy of the tables with a seed and schema version, so every run starts from the same state. After the agent finishes, the harness exports the final tables and compares them with `expected_final_state`.

Useful hooks include:

- a limit check that recomputes whether each approval was within authority - a delegation check on validity windows - a trace log of every tool call with arguments and timestamps - a refusal check for tasks designed to have no valid approver

Results can be written as JSONL, one row per task per run, with `run_id`, `model_label`, `task_id`, `score_*` columns and `failure_tags` such as `over_limit_approval` or `expired_delegation_used`. A reseller can give customers the same result schema so scores compare across runs. Ask for a reference grader script in a pilot and read it before relying on it.