How do we connect pilot material to our agent harness?
A pilot folder layout, release and task identifiers, dev and test splits, a tool mapping table and reset-and-export calls let delivery match an existing harness.
A pilot is easier to adopt when delivery matches how your harness already loads work.
- **Delivery layout:** a folder per release with `cases.jsonl`, `parties.parquet`, `documents.parquet`, `policies.json`, `tasks.jsonl` and a `schema` file describing every field and key. - **Versioning:** each release carries a `release_id` and each task a `task_id`, so results can be tied to the exact material used. - **Splits:** a `split` field marks tasks as `dev` or `test`, and test tasks can be held out from training runs. - **Tool mapping:** a short table maps environment actions to your agent's tool definitions, for instance `set_status` to your case system's update call. - **Reset and export:** a command or API call loads a named snapshot and returns final state as JSONL.
Wire it up in three steps. Load the schema and run the key checks. Run a baseline agent against dev tasks and inspect the action logs. Then run the test split once per agent version and store reports with `release_id` and `agent_version`.
Licence terms govern where outputs can be stored and who can access them, so settle that before copying files into shared repositories. Keep pilot material apart from any real customer data in your own systems, so generated and real records are never mixed in one store.
Matching delivery layout and identifiers to your harness lets you run baselines and tie every result to an exact release.
- Audience:
- Agent developers
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What does a scoped pilot for a claim lodgement agent look like?
- What should a pilot brief for intake include?
- What should we ask for in an intake sample or pilot?
- How should a team scope a first intake dataset or pilot?
- What delivery layout should I ask for in a pilot so my harness can load it?
- What does a first pilot look like for a professional services engagement request environment?