How can a sample review uncover a missing requirement before a pilot starts?
A sample task exposes uncertainty about who can authorise a revised deadline, turning an unresolved operational assumption into an explicit pilot requirement.
A developer reviews sample task `TASK-0841`, where a warehouse supervisor asks for more time to complete an overdue stock audit. The generated policy describes reminders and escalation, but does not specify whether that role can approve a revised deadline.
The agent accepts the extension and updates the audit record. During review, the operations reviewer identifies the missing authority rule. The team cannot fairly classify the action as correct or incorrect using the supplied material.
The sample moves from ready for evaluation to requiring scope clarification. The buyer asks for an authority matrix linking deadline changes to approving roles and for cases that distinguish a request for more time from an approved extension. Once that evidence is included, the task has an assessable decision boundary.
A MirrorFirm pilot could use this review process to refine generated material before broader testing. The consequence is a change to the specification, rather than an unsupported correction to the agent. The scenario teaches buyers to treat ambiguity in a sample as evidence about missing requirements, not automatically as model failure.
Use sample disagreements to refine task boundaries before investing in agent tuning or broader integration.
- Audience:
- Agent developers
- Content type:
- Scenario
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What integration contract should be agreed before connecting a pilot to an agent harness?
- Which licence questions matter when generated material enters an agent development pipeline?
- What should developers include in a request for a generated sample?
- What makes a useful first pilot for an overdue-handling agent?
- What integration contract should we agree before a pilot is generated?
- What is a useful first pilot scope for an environment builder?