How should a model lab scope a first engagement for this workflow?
Start with one or two domains, a defined defect taxonomy, a small held-out evaluation set and a clear question the data is meant to answer.
Begin with the question you want answered, then size the scope to answer it. Typical questions are whether your model over-requests, whether it misses expired or mismatched documents, or whether multi-turn follow-up fails after a partial reply.
A first scope can be set along five dimensions.
- **Domain.** One or two, for example lending and utility connections, rather than all seven. - **Policies.** A small set of versioned requirement lists with conditions that change what is needed. - **Defect taxonomy.** The types you care about, with the mix agreed in advance. - **Format.** Single-turn cases, multi-turn cases with a reply simulator, or both. - **Splits.** A held-out set kept apart from anything used for training or prompt tuning.
Licensing terms, volumes, delivery timing and price are agreed with MirrorFirm directly; none are assumed here. Raise early how you intend to use the data: training, fine-tuning, evaluation only, or internal research. Intended use affects how a licence is written.
The material is generated and informed by direct observation of how real businesses operate. It is not real business records. Include in the scope a review of grounding and disclosure risk, and decide who on your side signs off on it.
A narrow first scope produces a result you can judge quickly and a clear basis for deciding whether to expand.
- Audience:
- Model labs
- Content type:
- Explanation
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- How do environment reset, seeds and versioning work for a sufficiency environment?
- What licence terms and limits should I expect for a sufficiency release?
- How should I scope a sufficiency dataset, task pack or environment?
- What does a useful first pilot look like for a team building a utility onboarding agent?
- What should I ask for in a sufficiency sample or pilot?
- What should a first handoff package contain and how do we integrate it?