Why not treat sufficiency as a complete or incomplete classification task?
Answers why a binary label hides the work involved, and lists the extra fields a benchmark needs to separate correct decisions from correct-looking ones.
**Why not label each file complete or incomplete?** Because a binary label hides the work. Real decisions depend on which requirement set applies, which items satisfy which requirements and what to ask for.
**What else needs to be in the answer?** A useful case has an expected requirement list, a mapping from supplied documents to requirements and a set of acceptable next actions. Two agents can both say 'incomplete', yet one names the right missing item while the other asks for five unnecessary ones.
**Can a file be sufficient with items missing?** Yes. Policies often allow substitutes, such as a utility bill instead of a bank statement for address proof, or waivers when risk is low. A case can encode these so the right answer is 'proceed'.
**Can a complete-looking file be insufficient?** Yes. A document may be present but expired, unsigned, issued for the wrong entity or inconsistent with another document. Presence is not sufficiency.
**What about partial progress?** Some cases allow work to continue on one part while another waits. A supplier onboarding team might create the vendor record but hold payment setup until bank details are verified. Scoring should reflect whether the agent split the case correctly.
**What does this mean for benchmarks?** Score at least three things: the decision (proceed, request or escalate), the identified gaps and the quality of the request. A single accuracy figure over a binary label would blend these and hide where models fail.
Generated task packs can carry these fields so each is scored on its own.
Scoring the decision, the identified gaps and the request separately reveals failures that one accuracy number over a binary label would mix together.
- Audience:
- Evaluation teams
- Content type:
- FAQ
- Buyer question:
- What it is and why it matters
Content release: r-a220eb6e099b243b
Related questions
- What does it mean for an agent to decide whether the supplied documents are enough?
- What does a sufficiency decision look like on a real loan application?
- How do I tell whether my process suits sufficiency-focused training or evaluation?
- Where does the sufficiency decision sit in an agent loop, and which tool actions does it need?
- Why not handle this with a fixed checklist or rules engine instead of an agent?
- How does a sufficiency decision map onto environment state and tool actions?