How narrow should the first routing evaluation pilot be?

Start by naming the decision the pilot must support. An evaluation team might need to determine whether an agent can assign incoming utilities billing requests using a supplied responsibility map. That is a clearer objective than evaluating general customer operations competence.

Define the operating envelope: intake channels, request categories, organisational structure, evidence access and permitted actions. Also state exclusions, such as emergency dispatch or decisions about entitlement to compensation. An excluded process should not appear unexpectedly as a required capability in the test set.

Ask for a sample that demonstrates the intended boundaries and exposes assumptions that could affect the evaluation. The sample review should establish whether the task represents the buyer's intended problem before extensive model testing begins.

Agree on what evidence would justify expanding the scope. This might be a sufficiently interpretable failure analysis or successful integration into the buyer's evaluation harness, rather than a predetermined score.

A pilot supports conclusions within its declared envelope. Performance on billing intake alone cannot establish suitability for other departments, policies or channels that require different evidence and authority.