How can buyers test whether agents actually use the evidence?

A strong evaluation checks whether the agent's decision changes when the evidential basis changes. One way to do this is to construct paired cases that share most of their content but differ in a fact that should alter the outcome.

In a procurement review, both cases can contain the same purchase request and approval language. In one case, the approval covers the requested category. In the other, the approval explicitly excludes that category. The expected difference is the coverage assessment, not the number of records retrieved or the fluency of the final explanation.

Also include changes that should not affect the conclusion. Reordering attachments or changing an unrelated internal reference should not turn supported evidence into an exception. Together, consequential and inconsequential variations test whether the agent responds to the right features.

For a generated release, buyers can ask for the variation rule and the expected decision difference in evaluator-only metadata. Inspect whether the changed fact is actually discoverable through the agent's tools. If the distinction exists only in hidden labels, the pair measures guessing. Useful pairs connect accessible evidence to an explicit, reviewable difference in the permitted conclusion.