How should I scope a first pilot for conditional-approval material?
Start with one domain, one decision type and a few condition types, choose the deliverable depth and the costliest exceptions, and write success criteria before the sample arrives.
A useful first pilot is narrow. Pick one domain, one decision type and a short list of condition types, then grow from there.
**Choose the process.** Conditional loan approval through to settlement, or supplier onboarding through to first order, are typical choices. A single process lets you judge fit against something you know well.
**Choose the depth.** Decide whether you need a flat dataset for training, a task pack with ground truth for evaluation, or a simulated environment with tool calls and reset. The environment takes more scoping, so a task pack is often a sensible first step.
**Choose the exceptions.** Name the three or four exceptions that hurt most in your process, such as stale evidence, a waiver outside authority or a lapse between steps.
**Agree the limits.** Output is generated, so it will not reproduce your internal systems exactly. Treat it as a way to exercise behaviour, and expect to calibrate against your own environment. Licence terms, delivery format and volume are agreed for each engagement, and nothing here sets them.
**Define success before you start.** Write down what result would justify going further, such as clear separation between a naive and a careful baseline, or a useful list of failure modes.
A pilot should end with a decision: widen, adjust the scope, or stop.
A narrow pilot with written success criteria ends in a clear decision to widen, adjust or stop, rather than an open-ended trial.
- Audience:
- Agent developers
- Content type:
- Explanation
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What does a first pilot review look like for a referral-processing agent team?
- How is generated conditional-approval material licensed, and what limits apply?
- What should a pilot request include so the first sample can be judged quickly?
- How do I wire a pilot sample into my agent harness for a first run?
- What should I ask for in a first sample or pilot?
- What should I ask about licensing and permitted use before starting?