What should I ask for in a sample or pilot of this workflow?

Frame the pilot as a test of one question, and request the following.

- A stated question, such as whether the model misses expired or conflicting documents. - One or two domains with versioned policies and a short list of case types. - A defect taxonomy and an agreed mix, including clean cases that should proceed. - Gold decisions, gap lists and reason codes in a machine-readable form, with a schema description. - A held-out set separate from any set you may train or tune on. - A deterministic grader and, if multi-turn follow-up matters, a reply simulator with a reset seed. - A written note on how the data was grounded and what disclosure checks apply. - A proposed licence scope for your intended use, to be agreed directly with MirrorFirm.

Also decide what result would lead you to widen the scope, change it, or stop. Writing the threshold before seeing results prevents moving the goalposts.

Limits to keep in mind: the data is generated, so it can carry regularities that real files do not, and good pilot results do not establish that a model will behave the same way on your production documents. Treat the pilot as evidence about specific failure modes, not as a certificate of performance.