Scoping and evaluating a release

Bring a training objective. Scope the relevant operating patterns, deliverable and evaluation criteria together.

Before the conversation, decide what you want to train or evaluate and in what form you would prefer to start: a generated dataset, a task and evaluation pack, or a simulated environment.

Start with a small sample that exposes the schema, relationships and business logic before agreeing a larger release.

Checklist: does the sample show the schema, how records link to each other, and the rules behind the decisions? The fictional Aster Financial example is a demonstration of this shape, not a validated dataset.

Inspect a sample, discuss the business logic and define what a successful pilot needs to demonstrate.

Evaluation question: what result would make a scoped pilot worth continuing, and how will it be measured?

Pair a business objective with the policies, records and starting conditions an agent needs to work through it.

Evaluation question: what success criteria and checks accompany the objective, so a run can be judged consistently?