Skip to content
MirrorFirm

What should a buyer ask for in a first sample or pilot of schedule material?

A checklist of items to request in a sample or pilot, covering formats, schemas, policy documents, grader specifications, variation controls and a record of how the sample was produced.

Put these in your request so the sample can be tested against your own pipeline and not only read.

- **Format and schema:** the files in your preferred layout (JSONL, Parquet or both) with a schema per table and a list of link keys. - **Policy documents:** the versioned policy text each task relies on, including at least one superseded version. - **Task spread:** a mix of act, escalate, request-information and decline outcomes, across at least two domains. - **Grader specification:** expected final states, required step order, forbidden changes and how alternative valid answers are handled. - **Reset and seeds:** how to reproduce each start state, and which parameters can be varied. - **Known limits:** a plain statement of what the sample does not cover, such as time zones, currencies or specific product types. - **Generation notes:** how the material was produced and what grounding it draws on, so you can assess disclosure risk yourself. - **Acceptance checks:** the structural and semantic checks you intend to run, agreed in advance.

When the sample arrives, run your checks first and read a handful of episodes second. Send back specific findings, since those shape what a larger release should contain.

A precise sample request gets you material you can load, grade and review in days of your own effort, and surfaces gaps early.

Audience:
Model labs
Content type:
Checklist
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Repayment scheduling