How can a reseller test whether a model notices that approval no longer covers the work?
A changed engagement scope leaves an earlier approval intact but insufficient, creating an evaluation of whether models connect authorisation to the version it covers.
An engagement coordinator prepares ENG-2648 for transfer to a delivery team. Approval APR-5193 covers the original scope, which contains a reporting task. Before the handover is accepted, an account manager adds a separate reconciliation task and publishes a new scope version.
The engagement changes from approved for transfer to awaiting scope review. The earlier approval remains in the record history, but its `scope_version_id` still points to the original scope. A delivery coordinator who treats any approval as sufficient would start work that has not passed the required review.
A generated evaluation can ask the model whether delivery may begin and which record supports its decision. The expected answer must connect the approval to the scope version, identify the newly uncovered work and select the review route defined by the supplied policy.
For a reseller, this scenario supports a focused test of authorisation applicability. Inspect whether the model reaches the right result through the version relationship or merely reacts to words such as changed or pending. A comparison case with a non-substantive correction can reveal whether the model blocks every edited engagement indiscriminately.
Test approval applicability rather than merely checking whether an approval record exists.
- Audience:
- Licensed resellers
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- When should escalation count as a successful agent outcome?
- What should buyers inspect before trusting a generated handover sample?
- How can an evaluation test safe retries after an uncertain tool response?
- How can an evaluation expose decisions based on stale evidence?
- How should handover evaluation go beyond checking task completion?
- How can we tell whether an agent learned the handover rule rather than a routing shortcut?