Skip to content
MirrorFirm

Can generated case files include conflicting, expired or irrelevant documents?

Yes, a release can mix valid documents with expired, mismatched, duplicated and irrelevant ones, each labelled with its defect so graders can check the model's reasoning.

**Question: Can you generate files where the documents are not simply present or absent?**

Yes. A buyer can ask for a defect taxonomy and for the mix in a release to be set against it. Typical defect types include:

- **Expired.** A certificate or statement outside its validity window. - **Wrong entity.** A document issued to a related company or a former name. - **Wrong version.** A form that predates the current wording. - **Partial.** Pages missing, or a period only partly covered. - **Conflicting.** Two documents that disagree on a name, address, amount or date. - **Irrelevant.** A document that looks plausible but does not meet any requirement. - **Duplicate.** The same item supplied twice, which can hide a different missing item.

Each document carries a defect label and each case carries a gold gap list, so a grader can compare the model's gaps with the intended ones.

A useful pilot step is to ask for a small set per defect type, plus clean cases where the right answer is to proceed. Clean cases matter, because a model that requests more documents on every file looks careful and is actually unhelpful.

All documents are generated, and the content is not drawn from real files. Review a sample for grounding and disclosure risk, and check that the defect types match those your business sees.

Controlled defects let a lab measure which kinds of document problem a model misses, rather than a single pass rate.

Audience:
Model labs
Content type:
FAQ
Buyer question:
What can be generated

Content release: r-a220eb6e099b243b

More on Document sufficiency