Can generated case files include conflicting, expired or irrelevant documents?
Yes, a release can mix valid documents with expired, mismatched, duplicated and irrelevant ones, each labelled with its defect so graders can check the model's reasoning.
**Question: Can you generate files where the documents are not simply present or absent?**
Yes. A buyer can ask for a defect taxonomy and for the mix in a release to be set against it. Typical defect types include:
- **Expired.** A certificate or statement outside its validity window. - **Wrong entity.** A document issued to a related company or a former name. - **Wrong version.** A form that predates the current wording. - **Partial.** Pages missing, or a period only partly covered. - **Conflicting.** Two documents that disagree on a name, address, amount or date. - **Irrelevant.** A document that looks plausible but does not meet any requirement. - **Duplicate.** The same item supplied twice, which can hide a different missing item.
Each document carries a defect label and each case carries a gold gap list, so a grader can compare the model's gaps with the intended ones.
A useful pilot step is to ask for a small set per defect type, plus clean cases where the right answer is to proceed. Clean cases matter, because a model that requests more documents on every file looks careful and is actually unhelpful.
All documents are generated, and the content is not drawn from real files. Review a sample for grounding and disclosure risk, and check that the defect types match those your business sees.
Controlled defects let a lab measure which kinds of document problem a model misses, rather than a single pass rate.
- Audience:
- Model labs
- Content type:
- FAQ
- Buyer question:
- What can be generated
Content release: r-a220eb6e099b243b
Related questions
- Which records and relationships can a sufficiency dataset or task pack contain?
- Will a release include the documents themselves, or only metadata about them?
- What might the JSONL or Parquet schema look like for sufficiency cases?
- What should I include in a release specification for a sufficiency task pack?
- What would a generated insurance case with conflicting documents contain?
- How can one freight import case be expanded into controlled variants?