What acceptance checks should a reseller run on a conditional decision sample?
A checklist of structural, behavioural and risk checks a reseller can run on a sample before accepting it for catalogue listing.
Run these checks on each sample before accepting it into your catalogue.
- **Referential integrity:** every condition has a decision, every evidence item has a condition. - **Time order:** no review before submission, no release before the decision. - **State vocabulary:** only the documented states appear, and transitions follow the policy. - **Policy adherence:** waivers and releases come from roles the generated policies permit, apart from labelled exceptions. - **Exception coverage:** rejected evidence, late evidence and reworded conditions are present in useful quantity for your intended claims. - **Text variety:** read a spread of free-text fields for repeated phrasing or odd artefacts. - **Risk review:** search for accidental matches to real people or organisations and note the results.
Record the outcome of each check with the sample version. Do not describe the data as anonymised or compliant on the basis of these checks; they reduce uncertainty but do not remove it.
Running a fixed set of acceptance checks makes sample review repeatable and gives resellers something concrete to cite in their own documentation.
- Audience:
- Licensed resellers
- Content type:
- Checklist
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- What trace format and grader hooks make conditional-approval evaluation automatic?
- How do I check that generated tasks are hard enough and do not leak the answer?
- How should I score an agent on conditional approvals beyond a single pass rate?
- Which checks should I run on a sample before committing to wider use?
- What does a failed evaluation run look like when an agent binds a policy too early?
- How do I wire graders and reset into a conditional-approval environment?