How should evaluation teams score the quality of an assembled package?

A single pass label can hide very different behaviours. An agent may reach the right conclusion with the wrong evidence, or identify the right records but describe their significance incorrectly. Score the package on separate dimensions before deciding how to combine results.

Evidence coverage measures whether required claims receive relevant support or an explicit gap finding. Citation fidelity measures whether cited passages actually say what the agent attributes to them. Conclusion strength measures whether the wording stays within what those passages establish. Review usability measures whether the package is organised and free of unnecessary duplication.

Give partial credit deliberately. Correctly identifying an unresolved approval gap should score differently from silently omitting the approval claim. Retrieving a record without connecting it to a claim may earn retrieval credit without earning reasoning credit.

Define critical errors separately, such as asserting that a mandatory control passed despite direct contrary evidence. A benchmark can apply a failure condition for that error while retaining diagnostic scores. Evaluator guidance should explain acceptable alternative evidence paths so that a valid package is not penalised merely for differing from a reference answer’s presentation.