What reset, trace and scoring hooks does an intake evaluation pack need?

For evaluation, an intake pack needs hooks that let a buyer run an agent, score it and repeat the run. A scoped pack can provide:

- **Reset.** `reset(task_id, seed)` restores the environment to the task's `initial_state_ref`: queue contents, document store, policy version and any scripted customer replies. - **Tool actions.** A fixed set such as `get_case`, `open_item`, `run_check`, `request_information`, `route_case`, `escalate` and `add_note`. Each call is logged with arguments and the resulting state change. - **Trace output.** JSONL with `task_id`, `step`, `tool`, `args`, `state_before_hash` and `state_after_hash`. - **Scorer.** A function taking the trace and the final state and returning component scores such as `classification_correct`, `missing_items_f1`, `queue_correct`, `reason_code_match`, `unnecessary_requests` and `ordering_violations`. - **Success criteria.** Defined per task in the pack, for instance that the case reaches `routed` with the correct queue and no request is raised for an item already supplied.

Because state hashes are logged, a reseller can check that two runs from the same seed produce the same starting state. That tells you the environment is deterministic enough to compare agents. Ask for the scorer source or a documented specification so your customers can audit how scores are computed.

Keep the scorer version in your manifest. If the scorer changes, earlier results are not comparable.