Artifact contract

A completed run contains manifest.json, report.json, distance.npy, labels.json, tree.json, tree.nwk, membership.csv, clusters.json, and checkpoints. Diagnostics and normalized objects are optional according to the run configuration.

JSON is UTF-8, two-space indented, sorted, finite-only, and LF terminated. Artifacts are versioned individually rather than together: one that carries a schema version declares it in its own payload as schema_version, and manifest.json is the one load_result refuses to read a mismatch of. Internal artifact paths are relative and contained in the run directory. Manifests and small results use same-directory temporary files, fsync, and atomic replacement. load_result verifies every declared size and SHA-256 before returning a read-only matrix view.

manifest.json carries a method block naming the rule in force at each stage, so a run directory says which variant of the method produced it without being asked to the release that wrote it. One entry per stage — objects, distance, tree, clusters — each a versioned name and the citation keys that are its basis, resolvable in docs/refs.json:

"method": {
  "distance": {"name": "ncd-max/v1", "refs": ["cilibrasi2005"]},
  "tree": {"name": "neighbor-joining-q/v1", "refs": ["saitou1987", "studier1988", "adr-0017"]}
}

A rule is not a parameter. Which compressor and how many clusters stay in config, which config_hash covers; the rule names the method those choices were made inside. The block is optional and the schema version did not change for it, so a manifest written before it reads back with method absent — which states that the run does not say which rules produced it, rather than guessing. See ADR 0018.

Only completed is a successful terminal state. failed and interrupted reports remain diagnostic and may be resumed when their input, configuration, schema, runtime fingerprint, receipts, and checkpoints remain compatible. config_hash covers every config field except the execution settings – workers, csv_chunk_rows, compression_chunk_bytes, pairs_per_shard, pandas_materialization_limit_bytes and limits – which config still records as the invocation that created the directory resolved them. See ADR 0022.

report.json carries compressor_window_bytes and pairs_over_window beside the NCD range: what the compressor declared it can match across, and how many of this run’s pairs were longer than that. Both are None and zero for a compressor that declares no window, which is ppmd and is a measured fact rather than an omission; the factor the run was admitted under is config.limits.max_pair_window_factor, None on a run written before the gate existed. pairs_over_window is itself None when a resumed run reuses a distancing receipt written before the count existed: the stage is not recomputed, so the run does not know, and it says so rather than reporting a zero. See ADR 0023.

partition.json, written by damicore_normalizer.partition_dataset beside the subset files it describes, is a user-facing artifact rather than an inter-stage one: no stage reads it. It carries its own schema_version and the partition_rule its files depend on, the input’s path, size and SHA-256, the column and the resolved decimal separator, the row count, and one entry per emitted file with its method, parameter, end, rank interval, size and SHA-256. It is written last, atomically, and its presence is what marks a partition complete. The link from an emitted file to a later run is that file’s SHA-256, which the run manifest records as its input digest.