What should we ask for in a first sample or pilot?

Use this list when asking for a first sample or pilot, so that the review focuses on decisions rather than volume.

- One case type with its full requirement set, readable by your policy owners - Cases for each decision: proceed, request and escalate - Paired cases that differ by a single threshold or one document defect - A gold file with derivation notes showing how each label follows from policy - At least one multi-turn case in which the reply only partly answers the request - Schema, manifest and tool definitions that match your harness - A written statement of how grounding and disclosure risk were handled in generation - Your agreed success criteria and baselines, written before the sample arrives

The last item is the one teams most often skip. If you decide what a good result looks like only after seeing the data, it is easy to read the sample favourably. Write down the baselines you will run, the label agreement you expect from your reviewer and the defects you expect to see, then compare. Quantity, delivery and licence terms are settled in discussion with MirrorFirm once you know what the sample showed.