Which evaluation hooks should an intake environment expose?

An intake environment is easier to evaluate when it exposes a few hooks beyond the tool calls.

**Reset.** A reset call takes a snapshot reference and a seed and restores every table to the stored state. Two resets with the same inputs should give identical content hashes, which you can compare to confirm determinism.

**Step log.** Each tool call is written as a JSONL line with step, tool, arguments, result, and the ids of records read and written. This supports trace-based grading without re-running the episode.

**State diff.** A diff call returns changed records between the starting snapshot and the current state, field by field. Assertions on expected_end_state and protected_fields can be evaluated from this output alone.

**Grader interface.** A grader takes the task record, the diff and the step log, and returns per-criterion results, for example {criterion: 'duplicate_check', passed: false, evidence: ['PTY-7721']}.

**Error contract.** Rejected actions return a code such as INVALID_TRANSITION or MISSING_REQUIRED_FIELD, so an agent has something to learn from.

The interface style, whether function calls, a REST service or an MCP-style tool server, depends on your stack. Agree it in a pilot, and ask for a reference runner so you can test your harness against known outputs.