How should an evaluation team scope a first pilot for this workflow?

A useful scope for a first pilot is narrow: one case type, one requirement set family and a small number of defect axes. Breadth comes later.

Start with the decision you most want to test. A lender testing home-loan intake picks one product and the five or six documents that govern it. An insurer picks one claim type. A single domain slice lets your reviewers judge realism quickly, since they know the rules.

Then fix the output form. Decide whether the agent reads structured fields, extracted text or rendered documents, and whether the environment is static files or a stateful simulation with tools. Each choice changes the effort needed to build and to grade.

Agree the labels and their derivation next. Gold decisions should follow from the requirement set, and you should see the derivation rule before volume is produced.

Limits to expect: the output is generated, so it will not reproduce every quirk of a real file room; coverage depends on what is specified; and observation-informed structure is not a guarantee of fit with your internal policies. Your own policies may need to be supplied or adapted as inputs.

Finish by writing down how you will judge the pilot: the checks, the baselines and what result would justify a larger release. Quantities and delivery arrangements are scoped with MirrorFirm in the pilot discussion, as they depend on your intended use.