ADR 0005: Process pool for distance workers

Distance shards run on ProcessPoolExecutor with the spawn context. Threads were measured as the alternative, because zlib releases the GIL and they would remove the if __name__ == "__main__": guard that spawning imposes on callers.

They were rejected on the measurement. NCD needs one fresh compressobj per pair, and at level 6 each allocates roughly 256 KiB of deflate state, so worker threads contend on the shared heap instead of scaling. On a 400-object CSV of 1.5 KiB objects the distance stage took 15.5 s serial, 5.0 s on four processes, and 26.9 s on four threads: threads are slower than running serially. Threads only match processes once objects are large enough for compression to dominate allocation, around 200 KiB.

forkserver was tested and re-imports the caller’s __main__ exactly as spawn does, so no process context removes the guard. The guard requirement is documented instead, and a dead pool is reported as a DistanceError naming both of its real causes rather than as BrokenProcessPool.

Revisit if the compressor changes to one that reuses its state across pairs, or if per-object sizes in practice move above the crossover.