Skip to content
MirrorFirm

How should evaluators score cases with several acceptable destinations?

Routing quality can require acceptable destination sets and explicit error costs, because different valid assignments and different mistakes do not have equal consequences.

Some requests have several legitimate destinations. A professional services organisation can allow either a regional intake team or a central specialist queue to accept the same enquiry. A benchmark that stores one preferred label as the only correct answer can penalise valid behaviour.

Define the acceptable destination set from the supplied policy and task state. Where alternatives have different obligations, make those obligations part of the rubric: a central queue might be valid only when the agent includes a required supporting document.

Errors also deserve separate treatment. Sending a request to an adjacent team that can redirect it differs from assigning it to a queue prohibited from viewing its contents. Evaluation teams can report these error categories separately and, where appropriate, apply buyer-defined costs.

Avoid hiding the distribution behind a single aggregate score. Report valid assignments, conditional assignments with missing obligations, recoverable misroutes and prohibited destinations. The rubric should state how each category is determined and how unresolved ambiguity is adjudicated. This makes the evaluation useful for choosing deployment controls rather than merely ranking agents.

Use a scoring rubric that distinguishes valid alternatives from materially different routing errors.

Audience:
Evaluation teams
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Service routing