Skip to content
MirrorFirm

How could schedule records be laid out in JSONL and Parquet files?

One file per record type, keyed by stable identifiers, lets JSONL serve task and event streams while Parquet serves larger state tables, with a manifest describing the schema and links.

A layout that works for most environment pipelines uses one file per entity and stable string keys.

- **Parquet for state tables:** `plans`, `instalments`, `payments`, `policies` and `accounts`. Columnar storage suits large snapshots and typed columns for dates and decimals. - **JSONL for sequences:** `events` and `tasks`, one object per line. Events are ordered by `at`; tasks carry nested expectations. - **A manifest:** lists each file, its schema version, its row count and the key relationships between files.

A task line might carry `task_id`, `snapshot_id`, `goal`, `allowed_tools`, `expected_end_state` and `invariants`. The `snapshot_id` points at the set of state rows to load before the run, so the same task can be replayed from the same starting point.

Use decimal types for money, or integer minor units, never floating point, and store dates as ISO 8601 with an explicit time zone where cut-offs matter.

Identifiers follow a readable style such as `PLAN-0412` or `INST-0412-03`, and every foreign key should resolve inside the release. A loader check that fails on dangling keys is worth running first.

The data is generated, so schema fidelity is a design choice. Agree the field list in a pilot before building tooling around it.

A file-per-entity layout with stable keys loads into most simulators and warehouses without custom parsing.

Audience:
Environment builders
Content type:
Integration
Buyer question:
What can be generated

Content release: r-a220eb6e099b243b

More on Repayment scheduling