What task, grader and reset hooks support evaluating booking agents?

Evaluation of booking tasks works best when each task ships with a machine-readable definition, an expected outcome and a grader that returns structured results.

**Task file** (JSONL, one task per line): `task_id`, `scenario_seed`, `instruction`, `allowed_actions`, `expected_final_state`, `acceptable_action_paths`, `failure_mode_tags`, `split` (train or eval).

**Expected final state** is a diff against the seeded tables, such as `facility.FAC-6102.status = ACTIVE`, `facility.FAC-6102.rate_percent = 5.89` and a `booking_event` table containing one `raise_query` row.

**Grader output** (JSON): `task_id`, `passed`, `checks` (a list of named checks, each with `passed` and `detail`), `forbidden_actions_taken`, `steps_used` and `final_state_diff`. Named checks let you report which failure mode occurred, not only that the task failed.

**Hooks:**

- `reset(scenario_seed)` restores the starting snapshot - `apply(action, arguments)` runs one tool action and returns the result - `grade(task_id)` compares current state with the expected state and the action log

Run each task more than once if the agent is non-deterministic, and report pass rates per failure mode. Ask whether resets are deterministic and whether the grader is supplied as code or as rules in a data file, so your customers can run it in their own harness.