Skip to content
MirrorFirm

What does reviewing a pilot sample and revising the scope look like in practice?

An evaluation lead reviews a lending sample against written checks, finds senior responses too uniform, and revises the case mix before the scope is agreed.

An evaluation lead is building a benchmark for lending agents and asks MirrorFirm for a sample based on Aster Financial. The request is logged as **PILOT-014** with status `DRAFT_SCOPE`.

The lead's request names one product, unsecured business lending, two reviewer tiers above the credit officer, and a case mix that includes connected-exposure cases and within-authority controls. It also asks for the expected outcomes to be delivered as a separate file.

The sample arrives and the lead's team runs their mechanical checks. Exposure arithmetic and referential integrity pass. On reading the referral packages, the team notes that senior-reviewer responses are almost all approvals, so the return-for-information path is barely exercised. They record this against the sample and ask for a revised mix with more returns and at least one case where the second tier declines.

The lead updates `PILOT-014` to `SCOPE_AGREED`, with the changed mix, the evaluation-only use, and the acceptance checks written into the scope. Only then does the team ask for a larger set.

**What it teaches:** a sample is a way to find design gaps cheaply. Reviewing it against written checks, and recording what was wrong, turns the next request into a specific change instead of a general request for more data.

Reviewing a sample against written checks turns the next request into a specific change instead of a general call for more data.

Audience:
Evaluation teams
Content type:
Scenario
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Credit escalation