What quality checks should precede an agent evaluation?

Apply these checks before using a generated evidence environment to compare agent behaviour.

- Resolve every required foreign key against the supplied records. - Check that policy applicability matches the case's business context. - Verify that required conclusions are supported by accessible content. - Look for filenames or identifiers that reveal expected outcomes. - Confirm that distractor records remain plausible within the process. - Inspect grounding and disclosure risk in generated text and metadata. - Review ambiguous cases before assigning a single expected answer.

A coherent release can contain contradictions, but those contradictions should be intentional features of the task. A broken reference caused by packaging is different from a missing attachment that the agent is expected to investigate. The buyer needs a way to distinguish the two.

Unintended answer cues deserve particular attention. If every deficient case uses a recognisable identifier prefix, an agent may appear capable while avoiding the evidence entirely. Similar shortcuts can arise from document length, template wording or record ordering.

These checks establish whether the environment supports the intended assessment. They do not establish privacy compliance or regulatory acceptance. Generated output and its grounding should be inspected on their own terms before the buyer draws conclusions from agent results.