Which checks should I run on a sample before it goes into a catalogue?
A checklist of intake checks covering key resolution, schedule reconciliation, legal status histories, date order, case diversity, practitioner review, similarity searches and train/eval separation.
Run these checks on any sample before it goes into a catalogue.
- **Keys resolve:** every foreign key inside the release resolves, including links from facilities to approvals, conditions, security and schedules. - **Schedules reconcile:** repayment schedules match the facility's limit, rate, term and frequency, and each facility has one current schedule. - **Status histories are legal:** no facility reaches ACTIVE with an open condition unless an override is recorded. - **Dates run in order:** application, approval, booking, first drawdown and first repayment follow a sensible sequence. - **Cases differ:** product, amount, exception and outcome vary, not only the identifiers. - **Practitioner read:** a lending operations practitioner reads a handful of cases and flags any step a real team would not take. - **Similarity search:** text and identifiers are searched for passages that look copied from a specific institution or repeat across records. - **Splits are separate:** training and evaluation cases sit in separate files with no shared cases.
Running the same checks on every sample gives you a consistent standard and a record of what you inspected before listing.
- Audience:
- Licensed resellers
- Content type:
- Checklist
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- What checks should run on every booking task in an evaluation?
- Which failure modes should we expect to see and how should we score them?
- How can a buyer inspect the quality of generated booking material?
- What reset, snapshot and grader hooks would an evaluation harness need?
- How would an evaluation expose an agent that creates duplicate accounts on retry?
- How do I know whether my success criteria for booking tasks are strict enough?