The evidence behind the roadmap¶
What this page is¶
The roadmap says which capabilities the published DAMICORE literature attests and DAMICORE 0.3 does not have. This page is where those claims come from: an audit of 82 documents of DAMICORE work, read against the 0.3 source tree. It states the method, the limitations, and the counts, and it ships the per-document data the counts are computed from, so a reader can disagree with a number by recomputing it rather than by argument.
The corpus is the literature that descends from [sanches2011], the work this implementation takes its pipeline from.
The corpus¶
82 documents, classified by what each one does with the method:
Role |
Documents |
What it means |
|---|---|---|
|
45 |
applies DAMICORE to a domain problem |
|
16 |
cites or surveys it without applying or extending it |
|
15 |
changes the method itself: a stage, a criterion, an encoding |
|
4 |
a clustering method used as a comparison or a component |
|
2 |
a review covering the method among others |
How a document was read¶
Each document was read for what it states about the pipeline, and twelve fields were extracted from it: the novelty it claims, the library feature it would need, the compressors it uses, the distance, the tree stage, the partition stage, the criterion for the number of groups, the input encoding, the labeling, the validation, the limitations it states, and the scale it reports.
A capability is attested by a document when the lexical pattern for that capability matches the concatenation of those fields, case-insensitively. The count for a capability is the number of documents it matches. Nothing is weighted: a paper whose whole contribution is cluster labeling counts once, and so does one that mentions labeling as future work.
The data¶
evidence-corpus.csv, beside this page, is one row per document: year, first author, short
title, role, the object count it reports if it reports one, and one 0/1 column per
capability. The title is the document’s title cut to its first fifty characters, so it
identifies a document rather than citing it; where two documents by the same author and year
cut to the same string, the second carries a (2) suffix and they are two distinct works.
The table below is that file’s column sums, which is what makes the counts checkable: load the file, sum the column, compare.
What the corpus attests, and what 0.3 has¶
Capability attested in the literature |
Documents |
Years |
State in 0.3 |
Column |
|---|---|---|---|---|
Automatic cluster labeling |
19 |
2012-2024 |
absent |
|
Cluster validation and stability |
18 |
2012-2025 |
partial - modularity only |
|
Compressors beyond DEFLATE (PPMd, bzip2, LZMA) |
15 |
2015-2024 |
partial - zlib and gzip only |
|
Tree and dendrogram visualization |
15 |
2012-2023 |
absent - Newick export only |
|
Discretization and symbolic encoding |
13 |
2013-2023 |
partial - JSON lines, Fisher-Jenks |
|
Consensus and resampling of trees |
10 |
2016-2024 |
absent - one deterministic tree |
|
Number-of-clusters criterion |
9 |
2013-2025 |
partial - modularity or manual k |
|
Objects larger than the compressor window |
7 |
2013-2022 |
absent |
|
Scale beyond 1,000 objects |
7 |
2016-2024 |
partial - capped at 1,000 objects |
|
Incremental NCD reusing compressions |
7 |
2016-2024 |
absent |
|
Pluggable distance metric |
5 |
2016-2024 |
absent - NCD fixed |
|
Alternative community detection |
5 |
2014-2019 |
absent - FastGreedy only |
|
Alternative tree builders (UPGMA, linkage) |
3 |
2017-2024 |
absent |
|
Multiple representations per object |
2 |
2013-2022 |
absent |
|
Binary front-end (strings, CFG, lifting) |
2 |
2024-2024 |
absent |
|
Out-of-sample classification |
1 |
2017-2017 |
absent |
|
The two most attested capabilities in the whole corpus – automatic cluster labeling and cluster validation – are exactly the two the library does not have. A 0.3 run returns a partition and a modularity value: groups with no meaning attached and no measure of how stable they are.
The scale profile¶
27 of the 82 documents report how many objects they ran on. The median is 133, the third quartile 846, the maximum 8,755, and six report more than 1,000. The preflight cap of 1,000 objects is therefore above the median published study and below the six largest, which is the measurement behind the roadmap treating scale as a real limit rather than a theoretical one.
What has changed since the audit¶
The table is the audit’s reading of the 0.3 source tree and does not move: it is the baseline the roadmap was planned against, and it is the column sums of the data file. What the library has done since belongs here instead.
0.4.0 closed “compressors beyond DEFLATE” — 15 documents, behind only automatic labeling (19) and validation (18), and tied at 15 with tree visualization. The compressors the distance stage now measures with, and what each one sees, are ADR 0020 and ADR 0021; the work was issues #60 and #61. As of 0.4.0, nothing else in the table has moved.
“Objects larger than the compressor window” is detected, not closed. 0.5.0 refuses a run
whose largest pair exceeds the window its compressor declares, which turns the condition those
7 documents report from a silent distortion into a named preflight violation
(ADR 0023, issue #59).
Measuring such a corpus still needs a representation that fits – chunking, or a distance that
does not read the pair as one stream – so the row stays absent.
Limitations¶
Pages were sampled, not read whole. Each document was read at its method-dense pages rather than end to end. A capability stated only in a passage the sampling missed is not counted, so every count is a lower bound.
Six documents carry no text layer and were read as page images. Their extraction depends on what was legible in the rendering.
The match is lexical. A work that implements a capability without naming it in the vocabulary the pattern expects is missed, and a work that only mentions one is counted. This is why the table is evidence of attention in the literature, not a census of implementations.
What this page is not¶
It is not a bibliography. The works counted here are not cited by key and are not in
docs/refs.json, which records the works this implementation rests on – a different set,
held to a different rule: every entry there is fetched from a resolver and cited by a unit
that implements it (ADR 0015). The audit corpus is data about the literature, and it ships as
data.