What quality checks should precede training on generated review cases?

Inspect the generated cases for shortcuts that could inflate apparent learning or evaluation performance.

- Check whether filenames reveal expected outcomes. - Look for decision labels embedded in evidence metadata. - Compare document length across outcome categories. - Inspect repeated wording tied to particular decisions. - Check whether identifier prefixes predict task difficulty. - Keep evaluator annotations outside agent-visible records. - Review whether missing fields unintentionally reveal answers.

These checks concern how the material is constructed, not whether the business rules are sensible. A supplier review may contain a valid evidence chain while also placing every adverse finding in a file with a revealing name. An agent could learn the naming pattern and ignore the evidence.

Inspect both individual cases and patterns across the release. A cue that looks harmless in one record can become a reliable shortcut when repeated throughout training material.

A useful sample inspection includes solving cases with the business text concealed while leaving packaging metadata visible. If outcomes remain easy to predict, investigate the packaging before treating subsequent agent results as evidence of review competence.