What should I ask for in a sample or pilot?
A checklist of what to request in a first sample, including schema, task variety, expected states, rule files, delayed consequences, grounding notes, stated limits and a proposed next step.
A useful sample request is specific enough that you can judge the result in your own harness within a short review.
- A seed state in your preferred format, with a schema and a list of status values. - Tasks covering clean completion, a pending outcome and a referral, so each path is represented. - Expected end states per task, including acceptable variants and forbidden writes. - A rule file where each rule has an id, a plain-language statement and a check. - At least one task with a delayed consequence that exposes an earlier error. - A note on how structure and exceptions were informed, and what grounding or disclosure review you can carry out. - A statement of limits: what the sample does not cover, and what is not guaranteed. - A proposed next step after review, such as widening the domain or adding a second acting role.
A sample that covers every outcome path, with explicit limits, lets you judge fit in your own harness without a large commitment.
- Audience:
- Environment builders
- Content type:
- Checklist
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What does a first pilot look like for a client and matter opening agent?
- What licence and usage limits should we ask about before starting?
- How should we scope a first pilot for set-up agents?
- What should we ask for in a sample or pilot release for set-up?
- How do we wire a pilot release into our agent harness on day one?
- How should delivery formats, versioning and data splits be agreed for a pilot?