What should we specify when requesting generated sufficiency cases?
A checklist of the choices a buyer should settle before generation, covering case types, labels, defects, multi-turn behaviour and document form.
Before asking MirrorFirm for generated cases, settle these points so the release matches the decisions you want to test.
- Case types and domain, such as mortgage applications, warranty claims or supplier onboarding - The requirement sets, and how they change by amount, region or customer type - The decision labels you need: proceed, request, escalate, or your own vocabulary - Defect axes to include, and the share of clean cases to keep in the mix - Whether substitutes and waivers should appear, and under what conditions - Multi-turn behaviour: full replies, partial replies, late replies and silence - Document form: structured fields only, extracted text or rendered files - Output formats and the field names your harness already expects
If you have internal policy text, say whether it can be supplied as an input or whether generic rules should be used instead. Generic rules are quicker to inspect but may not match the thresholds your reviewers know. Either way, treat the first release as a draft to inspect, and expect to adjust the specification once you see how the cases read.
Deciding these points up front keeps the release matched to the decisions you want to test and avoids rework after the first sample.
- Audience:
- Evaluation teams
- Content type:
- Checklist
- Buyer question:
- What can be generated
Content release: r-a220eb6e099b243b
Related questions
- Which records and relationships can a sufficiency dataset or task pack contain?
- Will a release include the documents themselves, or only metadata about them?
- What might the JSONL or Parquet schema look like for sufficiency cases?
- What should I include in a release specification for a sufficiency task pack?
- What would a generated insurance case with conflicting documents contain?
- How can one freight import case be expanded into controlled variants?