What should I check when inspecting a sample of mismatch cases?

Run these checks on a small sample before deciding whether to widen scope.

- Solve several cases by hand and compare your reasoning with the labelled root causes and expected outcomes. - Run automated checks that totals balance, links resolve and dates fall in a sensible order. - Test a trivial strategy, such as trusting the newest record, and see how often it scores well. - Confirm that decoy explanations fail on the evidence supplied, rather than on knowledge outside the case. - Check that escalate, wait and no-action cases are present and follow from the stated policy. - Read the notes, emails and procedure text for domain-appropriate wording, formats and level of detail. - Search for details that resemble a specific organisation or person, since generation does not rule this out. - Confirm the grader file is separate from the model-facing view and cannot be read by the model.