What does a useful first pilot look like for a team building a utility onboarding agent?
A product lead scopes a narrow sample of two connection case types, reviews several cases by hand, finds a missing policy-change scenario and adds it to the next round.
The product lead of a team building an agent for utility connection requests asks MirrorFirm for a sample. The agent handles new-connection applications for a retail energy provider, and the customer team says it asks for more documents than needed.
They scope a pilot around two case types: residential connection and tenant transfer. They ask for requirement sets for each, a spread of cases from clearly sufficient to clearly insufficient, and a smaller group of borderline cases where a substitute document is arguable.
The sample arrives as case records, document records, requirement sets and gap labels, with a held-out portion kept for evaluation. The team's reviewer opens several cases. In CONN-0731, the proof of occupancy is a lease extract rather than a signed lease; the requirement set lists extracts as acceptable, and the gap label is empty. The reviewer notes their own agent would have asked for the full lease.
They also find that no case includes a policy version change, though their process revises requirements each quarter. They add that to the scope as a second round, with mid-case version changes in the environment.
**What it teaches:** a first step is a narrow sample that exposes mismatches in scope, not a broad purchase. Check labels against your own policy, record defects, and decide what the next release should add.
Start with a narrow sample that exposes scope mismatches early; review a handful of cases by hand and let the findings shape the next release.
- Audience:
- Agent developers
- Content type:
- Scenario
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- How do environment reset, seeds and versioning work for a sufficiency environment?
- What licence terms and limits should I expect for a sufficiency release?
- How should I scope a sufficiency dataset, task pack or environment?
- What should I ask for in a sufficiency sample or pilot?
- What should a first handoff package contain and how do we integrate it?
- What licence questions and limits should we raise before starting?