What evaluation hooks and grader interface would a sufficiency task pack provide?

A task pack can be shaped around four hooks.

**1. Task input.** For each `case_id`, the harness passes the case, documents and policy as defined in the pack. A `mode` flag can supply raw documents or pre-extracted fields.

**2. Model output.** The model returns JSON with `decision`, `gaps[]`, `request` and `status_after`. A JSON Schema file validates the shape before grading, and malformed output is scored as its own failure type.

**3. Deterministic grader.** A script compares the output with gold and returns per-case values:

- `decision_correct` - `gap_precision` and `gap_recall` - `false_gap_ids` and `missed_gap_ids` - `status_correct` - `wrongly_held_clean`

**4. Reply simulator (optional).** A tool such as `send_request(case_id, items, recipient_role)` returns a scripted reply from the case's customer profile, which can be full, partial, late or off-topic. The case state is updated, and the harness continues until the case reaches a terminal status or a turn limit.

**Environment reset.** Each run starts from a seed that restores case state, message history and simulator behaviour, so a run can be repeated exactly.

Results write to JSONL with `run_id`, `model_id`, `case_id`, the measures above and the defect labels, which makes slicing by defect type a simple group-by. Confirm the language and interface of the harness in a pilot.