Why evaluate policy selection alongside evidence retrieval?
The same records can support different conclusions under different policy versions, making applicability decisions a necessary part of business evidence evaluation.
**Why is a relevant policy document not enough?**
A policy can match the subject of a review without governing the transaction. Applicability may depend on the business unit, service category, effective date or an approved transition arrangement. An agent that picks the newest document by default can reach the wrong conclusion using entirely accurate records.
**What does this look like in a task?**
A utilities review covers service changes initiated before a revised approval procedure took effect. The earlier procedure applies until each open request reaches its next assessment stage. Request REQ-0412 crossed that stage after the change; request REQ-0413 did not. Their evidence requirements differ despite similar completion dates.
**What should the agent return?**
The evidence package should identify the selected policy version, the applicability condition and the record that establishes that condition. Quoting an approval clause without explaining why it governs the request is incomplete.
**Why does this matter for evaluation design?**
It separates rule selection from rule satisfaction. A benchmark can reveal whether an agent understands the process boundary or simply matches policy language to document titles.
Test whether agents select the governing rule before judging whether the evidence satisfies it.
- Audience:
- Evaluation teams
- Content type:
- FAQ
- Buyer question:
- What it is and why it matters
Content release: r-a220eb6e099b243b
Related questions
- Why does evidence assembly require more than finding relevant documents?
- Why should an agent inspect handovers between teams?
- How can a late record change an evidence assembly task?
- What business context makes an evidence task useful for agent development?
- How should an environment distinguish process completion from evidence completeness?
- Which characteristics make a workflow useful for an interactive evidence environment?