Can a release include stale, conflicting or defective documents, and still allow 'proceed'?
Covers the defect axes a buyer can request, why to vary them one at a time first, and why clean files must remain in the mix.
**Can a release include stale, conflicting or poor-quality documents?** Yes, a buyer can ask for variation along defined axes. Typical ones:
- Expired or superseded versions of a document - Name, address or date mismatches between documents - Missing pages, unsigned forms or unreadable sections - Duplicates sent twice under different titles - Irrelevant attachments that look relevant - Late replies that only partly answer an earlier request
**Are these axes independent?** They can be combined, but a pilot should usually start with one axis per case so a failure can be traced to a cause. Combined cases suit later stages.
**Can the correct answer still be 'proceed'?** It should be, for a share of cases. A set where every file has a defect teaches an agent to always ask. Also ask for clean-but-unusual files, such as one that uses a permitted substitute document.
**How realistic are the defects?** That is for a buyer to inspect. The output is generated, so check a sample against what your own staff see, and look for patterns that are too tidy or too easy to detect from formatting alone.
**Can defect types be tied to a policy?** Yes. Each defect can reference the requirement it breaks, so the label is derived from the policy and not added afterwards by hand.
Ask for defects along defined axes, tied to the requirement they break, and keep clean and unusual-but-valid files so agents cannot score by always asking.
- Audience:
- Evaluation teams
- Content type:
- FAQ
- Buyer question:
- What can be generated
Content release: r-a220eb6e099b243b
Related questions
- Which records and relationships can a sufficiency dataset or task pack contain?
- Will a release include the documents themselves, or only metadata about them?
- What might the JSONL or Parquet schema look like for sufficiency cases?
- What should I include in a release specification for a sufficiency task pack?
- What would a generated insurance case with conflicting documents contain?
- How can one freight import case be expanded into controlled variants?