What should happen when a pilot exposes a missing dependency in the task scope?
A professional services review pilot reveals that approval records cannot establish authority without role history, prompting a precise scope decision before broader generation.
A model lab scopes a pilot around evidence that professional services engagements received authorised acceptance. The proposed inputs include engagement ENG-2187, acceptance form ACC-0412 and a policy describing which roles can approve the work.
During sample review, an evaluator asks whether the approving role held the required authority on the acceptance date. The form records a role identifier, but the pack contains no assignment history. The pilot issue changes from accepted for testing to blocked by a scope dependency: the intended conclusion cannot be judged from the proposed inputs.
The buyer has a concrete choice. Add generated authority history to support the original task, or narrow the task to checking that an acceptance form exists and contains the required fields. The latter remains testable but measures a different capability.
The consequence of ignoring the dependency would be a misleading evaluation that rewards assumed authority. This scenario teaches a useful scoping discipline: when a required inference depends on missing context, change the input contract or change the claim being tested. Additional document variety would not repair that gap.
Use sample review to expose indispensable dependencies and distinguish them from optional realism.
- Audience:
- Model labs
- Content type:
- Scenario
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What integration contract should we establish before loading a pilot release?
- What makes a useful first pilot scope for an agent team?
- What licence questions should we resolve before using generated evidence material?
- How can a pilot reveal that the agent needs a different action boundary?
- What should a sample request include before a pilot is agreed?
- What integration contract should be agreed before a pilot build?