Skip to content
MirrorFirm

How could a scheduling pack be delivered, versioned and connected to an evaluation harness?

Covers file formats, manifest and versioning, a containerised or MCP-served environment, and the compatibility questions to settle before a larger release.

A pack can be delivered in more than one form, and the choice should follow how your harness runs.

**Static files**

- JSONL per entity for readability and diffing, Parquet for large tables. - A `manifest.json` with schema version, generation parameters, seed, file hashes and a change log. - A `tasks` file linking each task to its starting snapshot, as-at date, policy references and expected end state.

**Interactive environment**

Where tasks need tool use, the environment can be offered as a container or as tools served over MCP. Typical tool set: `get_agreement`, `get_schedule`, `propose_adjustment`, `commit_adjustment`, `request_approval`, `add_note`, `list_postings`, plus `reset` and `snapshot` for the harness only. Keep harness-only tools hidden from the agent.

**Versioning**

- Use semantic versions for the schema and a separate release identifier for the data. - Never change a released task's expected state in place. Issue a new task version and mark the old one deprecated. - Record the release identifier in every result so scores can be compared like for like.

**Questions to settle in the pilot**

- Id formats and money representation (integer cents is a safe default). - Time zone and calendar handling. - Whether your tool names should be mirrored. - Where the environment runs, and how reset is triggered between runs.

These details are agreed during scoping, not fixed in advance.

Settling format, versioning and access up front keeps results reproducible and avoids rework when a pilot grows into a larger pack.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Repayment scheduling