How could a scheduling pack be delivered, versioned and connected to an evaluation harness?
Covers file formats, manifest and versioning, a containerised or MCP-served environment, and the compatibility questions to settle before a larger release.
A pack can be delivered in more than one form, and the choice should follow how your harness runs.
**Static files**
- JSONL per entity for readability and diffing, Parquet for large tables. - A `manifest.json` with schema version, generation parameters, seed, file hashes and a change log. - A `tasks` file linking each task to its starting snapshot, as-at date, policy references and expected end state.
**Interactive environment**
Where tasks need tool use, the environment can be offered as a container or as tools served over MCP. Typical tool set: `get_agreement`, `get_schedule`, `propose_adjustment`, `commit_adjustment`, `request_approval`, `add_note`, `list_postings`, plus `reset` and `snapshot` for the harness only. Keep harness-only tools hidden from the agent.
**Versioning**
- Use semantic versions for the schema and a separate release identifier for the data. - Never change a released task's expected state in place. Issue a new task version and mark the old one deprecated. - Record the release identifier in every result so scores can be compared like for like.
**Questions to settle in the pilot**
- Id formats and money representation (integer cents is a safe default). - Time zone and calendar handling. - Whether your tool names should be mirrored. - Where the environment runs, and how reset is triggered between runs.
These details are agreed during scoping, not fixed in advance.
Settling format, versioning and access up front keeps results reproducible and avoids rework when a pilot grows into a larger pack.
- Audience:
- Evaluation teams
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What might a first pilot look like for a delivery scheduling agent?
- How do I wire a first pilot into my agent stack?
- What limits and licence questions should I raise before starting?
- How should I scope and license a schedule dataset or environment?
- How should a pilot release be delivered and versioned for my pipeline?
- What licensing terms and limits apply to generated schedule data?