Skip to content
MirrorFirm

What should I settle before requesting an escalation sample or pilot?

A checklist of decisions to make before requesting a sample or pilot, covering domain, form, use, case mix, difficulty, acceptance checks, review plan and format needs.

Before you request a sample or pilot, settle these points so the first delivery can be judged quickly.

- Domain and roles: which setting and how many approval levels you need - Form: dataset, task pack, simulated environment, or a mix - Use: training, evaluation, or both, and whether results will be published - Case mix: within-authority, over-limit, borderline, wrong-level and returned cases - Difficulty features: aggregated exposure, versioned limits, missing evidence - Acceptance checks: referential integrity, state transitions, gold outcomes, note quality - Review plan: who inspects grounding and disclosure risk, and what they will search for - Format needs: JSONL, Parquet, JSON Schema, tool definitions, reset method

Write the acceptance checks down before delivery, not after. It avoids arguing about quality once material has arrived. Ask for a data dictionary and a short account of how cases were constructed.

Keep the first scope narrow: one domain and one authority structure is easier to inspect than several, and what you learn tells you what to widen. Commercial terms, timing and pricing are agreed directly and are not described here.

Deciding these points first lets you judge the first delivery quickly and keep the initial scope narrow.

Audience:
Model labs
Content type:
Checklist
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Credit escalation