Skip to content
MirrorFirm

How should an evaluation team scope a scheduling pack, and what limits should it expect?

Explains the main scoping choices, such as domain, actions, policy fidelity and licensing use, and the limits that come with generated material.

Start by naming the decision the pack must inform: for example, whether an agent can be trusted to propose schedule adjustments within policy. Then fix the scope around that decision.

**Choices to settle**

- **One domain first.** Loan instalments, utility plans or delivery tranches. Mixing domains early makes results hard to read. - **A short list of actions.** Two or three adjustment types, with the exceptions that matter for each. - **Policy source.** Generated policies, or policies you supply that the pack should be built around. - **Purpose.** Held-out evaluation, training, or both. Keep evaluation tasks separate from any training material. - **Licence terms.** Permitted uses, redistribution, and whether derived results can be published. These are agreed contractually and are not assumed here.

**Limits to expect**

- The material is generated. It is informed by how businesses operate, but it is not a copy of any organisation's records or rulebook. - Realism has bounds. Your own systems may carry quirks, legacy codes or local rules that a pack will not reproduce unless you specify them. - Grounding and disclosure risk should be inspected on a sample, not assumed away. - A pack measures the situations it contains. Results do not transfer automatically to workflows outside that scope.

Write the scope as a one-page brief with the decision, the actions, the policy source and the acceptance checks. It gives both sides something concrete to test against.

Narrow, explicit scoping gives a pilot a clear pass or fail question and prevents a vague request from producing a pack that fits nobody.

Audience:
Evaluation teams
Content type:
Explanation
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Repayment scheduling