Skip to content
MirrorFirm

How should an evaluation team scope a pilot for approval-authority tasks?

Explains how to define domain, roles, limit types, conditions and format for a pilot, and what a sensible first step looks like before committing to a larger release.

Scope the pilot around the decision you want to test, not around volume.

**Choose one domain.** Pick a setting your agents will meet, such as procurement, claims payment, credit variation or refunds. Mixing domains early makes it hard to tell whether a failure comes from the model or the setting.

**Define the authority model.** State which limit types matter: single-item amount, aggregate exposure, risk band, category or role. Decide whether delegations, acting arrangements and revocations are in scope.

**List the conditions to test.** Choose from boundary amounts, linked requests, expired grants, self-approval and conflicts. Ask for each to be tagged so you can report per condition.

**Pick the form.** A task pack is the simplest start: prompts, records and gold outcomes. A simulated environment adds tools, state and reset, and takes more scoping.

**Set your format.** Say whether you need JSONL, Parquet or both, and share your harness conventions so field names fit.

**Agree how you will judge it.** Write down the checks and success criteria before you see the sample.

All content is generated, so ask how it was grounded and plan a disclosure review. Licence terms, quantities and timing are agreed with MirrorFirm during scoping, and this description does not set them.

A narrow pilot with explicit domain, limit types and failure conditions lets you test fit and quality before agreeing a larger scope.

Audience:
Evaluation teams
Content type:
Explanation
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Delegated approval