Skip to content
MirrorFirm

What should a buyer ask for in a first sample of facility material?

A checklist of eight written requests for a first sample, covering cases, tables, reference trajectories, graders, labels, reset, generation method and licence wording.

Put these requests in writing when you ask for a sample, so the first review tests what you actually need.

- **Case file.** Prompts, policy versions and expected outcomes for each case, including cases where booking is the right answer. - **Start-state tables.** Your preferred format, with a schema file listing types, keys and nullability. - **Reference trajectories.** Expert-style steps, with the role taking each action. - **The grader.** A description of what each check covers and what it ignores. - **Exception labels.** Tags so you can slice results by defect type. - **Reset procedure.** One you can run yourself, with a note on how seeds work. - **Generation statement.** How the material was generated and what disclosure review was applied. - **Licence wording.** The proposed text for your intended uses, such as training and published evaluation.

Putting these requests in writing means the first review tests what you actually need instead of what happens to be easiest to hand over.

Audience:
Model labs
Content type:
Checklist
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Loan booking