How do I judge the quality of generated intake material?

Quality in intake material shows up in four places: whether the cases are realistic, whether the rules are coherent, whether the answers are checkable, and whether the output carries disclosure risk.

**Realism of cases.** Read a spread of submissions. Look for the ordinary untidiness of intake: abbreviations, partial addresses, badly named attachments, details that contradict each other. Check that the mix also includes clean cases, otherwise a model learns to distrust everything.

**Coherence of rules.** Pick ten cases and trace each decision to a policy rule. Check that rules do not contradict each other unless the conflict is intended and has a documented tie-break. Check that effective dates and thresholds are consistent across records.

**Checkability.** Each task should have a reference outcome that two reviewers would agree on. Where several outcomes are acceptable, the reference should say so and list them.

**Disclosure and grounding.** Because the output is generated, ask how scenarios were informed and inspect free text for names, addresses or phrases that look lifted from a real source. Do not assume generated data is free of such traces.

A useful way to begin is to run your own baseline model on the sample and read the failures. Failures that trace to ambiguous tasks point to data problems; failures that trace to missed rules point to model gaps.