Skip to content
MirrorFirm

How should a model lab scope a first engagement for this workflow?

Start with one or two domains, a defined defect taxonomy, a small held-out evaluation set and a clear question the data is meant to answer.

Begin with the question you want answered, then size the scope to answer it. Typical questions are whether your model over-requests, whether it misses expired or mismatched documents, or whether multi-turn follow-up fails after a partial reply.

A first scope can be set along five dimensions.

- **Domain.** One or two, for example lending and utility connections, rather than all seven. - **Policies.** A small set of versioned requirement lists with conditions that change what is needed. - **Defect taxonomy.** The types you care about, with the mix agreed in advance. - **Format.** Single-turn cases, multi-turn cases with a reply simulator, or both. - **Splits.** A held-out set kept apart from anything used for training or prompt tuning.

Licensing terms, volumes, delivery timing and price are agreed with MirrorFirm directly; none are assumed here. Raise early how you intend to use the data: training, fine-tuning, evaluation only, or internal research. Intended use affects how a licence is written.

The material is generated and informed by direct observation of how real businesses operate. It is not real business records. Include in the scope a review of grounding and disclosure risk, and decide who on your side signs off on it.

A narrow first scope produces a result you can judge quickly and a clear basis for deciding whether to expand.

Audience:
Model labs
Content type:
Explanation
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Document sufficiency