How could generated records and policies be packaged for an evaluation harness?
A structured release can separate cases, interactions, referrals and policy clauses while supplying explicit joins, schema versions and validation rules for reliable harness ingestion.
A generated release could supply Parquet tables for `accounts`, `cases`, `referrals` and `policy_clauses`, with JSONL for ordered messages. A case row might contain `case_id`, `account_id`, `opened_at`, `policy_version` and `initial_status`. Referral rows can reference both the case and the destination queue.
The schema should declare keys, allowed nulls and enumerated values. A missing `referral_id` can mean that no referral exists; it should not be confused with a broken export. Timestamps need a documented timezone convention so message ordering survives ingestion.
Policy links should resolve to specific clauses within the supplied version. Validation can reject orphaned references, duplicate primary keys and cases linked to absent policies before the evaluation begins.
An integration sample should include both the files and a small manifest describing their relationships. Keep evaluator-only expected outcomes in a separate namespace from agent-visible records. That separation lets the harness load scoring material without exposing answers through an ordinary case lookup. Buyers can then inspect the same linked structure through tabular analysis and environment tools.
Make relationship errors visible before they become misleading evaluation failures.
- Audience:
- Evaluation teams
- Content type:
- Integration
- Buyer question:
- What can be generated
Content release: r-a220eb6e099b243b
Related questions
- Which variations should a generated intake pack contain?
- What generated context can surround a customer's request for assistance?
- How can generated account state be packaged for repeatable agent runs?
- Can a generated pack include conflicting policy documents and outdated routing instructions?
- How can generated tasks connect premium difficulty with a separate insurance claim?
- Which generated records help simulate a credible referral without exposing every detail?