What should buyers request with a sample before committing to broader scope?

Request these supporting items so your team can assess a generated sample against its intended evaluation use.

- A statement of the target agent capability. - An inventory of included and excluded records. - A rationale for each expected decision. - Notes on deliberately unresolved ambiguities. - A description of grounding and generation assumptions. - Known limitations and unsupported task variants. - A process for recording review disagreements.

The sample review should establish whether the task is understandable, assessable and relevant to the buyer's programme. Attractive correspondence alone cannot show whether the expected decision follows from the evidence or whether an essential business rule is missing.

Have an evaluator attempt the task from the agent-visible material, then compare that judgement with the supplied rationale. Record disagreements precisely: missing evidence, unclear authority and disputed scoring criteria require different corrections. A disagreement is useful when it reveals what the specification must make explicit.

MirrorFirm could scope the sample around these review needs. Before requesting broader coverage, agree which findings require changes to the records, which require changes to the rubric and which represent accepted limits of the proposed evaluation.