How should an evaluation team choose the first handover boundary to model?
A focused pilot starts with one consequential team boundary and a clear decision, then expands only after the evaluation contract works.
Start with a boundary where the receiving team's next decision is clear. A utility connection request moving from application review to field scheduling is a useful shape: the task has an identifiable sender, recipient, entry condition and next action. The pilot does not need to reproduce the whole organisation.
Specify what the evaluated agent controls. It might prepare the transfer, decide whether to accept it or resolve a returned case. Combining all three roles immediately can make a failure hard to attribute because the agent is effectively creating and judging its own work.
Agree the boundary of the simulation as well. If scheduling capacity is represented by fixed slots, say so. If customer replies are outside scope, define how unanswered questions affect completion. These limits are part of interpreting the evaluation, not incidental implementation notes.
A useful first request is a generated case specification with visible inputs, permitted actions, an expected end state and one contrasting failure case. Review whether those elements support a meaningful judgement before requesting broader variation or more connected workflows.
Keep the first pilot narrow enough to diagnose failures and judge whether the generated material is useful.
- Audience:
- Evaluation teams
- Content type:
- Explanation
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What licence scope should an agent developer clarify before starting?
- What integration artefacts should we request with a first sample?
- What does a narrowly scoped handover pilot look like?
- What should a pilot brief specify before generation begins?
- Can we scope a useful sample without supplying production records?
- What integration contract should be agreed before a generated environment pilot starts?