A patch, and the number is the honest one. A minor closes an arc, and the arc in progress is 1.57 — the optimiser earns its name again — whose thesis is what belongs on the model's side of the line. Nothing here answers that. What is here is the groundwork the arc could not proceed without: the deterministic side measured, exercised and pinned, so a decision about what to add is made against figures instead of against a headline percentage nobody can reproduce.
Added#
trazum rules --measure <dir> — and what does each rule actually recover?
The first chapter of arc 1.57, and it starts by measuring the number the whole
arc is about. The README says plainly that the deterministic rules recover about
one per cent, which is the fair complaint about this tool — and it is an
aggregate, consistent with every rule pulling its weight and equally
consistent with two rules doing all of it beside five that have never changed a
byte of anybody's prompt. Nobody here had measured which, and deciding what
belongs on the model's side of the line is a question about what the dictionary
side already covers.
The rule order decides what the reader is told, and now something says so. The measurement showed three deletion rules each recovering the same tokens, which looked like a defect worth chasing. It is not: a repeated stanza is a repeated block and also a set of repeated lines, all three can find it, and whichever runs first takes it. The applied run credits exactly one rule — the leave-one-out measurement credits all three because each would have caught it alone, and those are different questions.
What had no guard was the consequence. Coarsest-first means the same saving is reported as one repeated paragraph instead of three repeated lines, and the only reason written down for that order was that block deletions leave less text for the rest to walk. A reorder would have changed what users read while every number stayed identical and no test noticed. The order is pinned now, adjacency included, and the comment says what it actually decides.
A corpus that exercises every rule. rules --measure could only ever
answer "inert here", because nothing in this repository contained what most
rules look for. Twelve fixtures now do — one per rule, each a short realistic
prompt carrying exactly what that rule is written to find — and the guard is
derived from the rule catalogue, so a rule added without a fixture fails the
build rather than joining a list nobody reads.
And building it split one field into two. inertHere was "saved nothing",
which quietly merged never fired with fired and recovered nothing. Those
look identical in a saving column and mean opposite things: the first is a fact
about the corpus, the second is a finding about the rule. emphasis is the
case — it lowercases shouted words, so the prompt changes, the instruction
changes, and the token count does not move. It now lands in
firedWithoutSavingHere instead of being filed beside rules nothing exercised.
Two figures per rule, and they are never added together. alone is what a
rule saves as the only rule running; marginal is what the whole set loses when
it is removed. They diverge exactly where rules overlap — on a prompt with a
repeated stanza, duplicate-blocks, near-duplicate-blocks and
duplicate-lines each recover forty tokens alone and nothing at the margin,
because the other two still find it. Reporting alone alone makes an
overlapping rule look load-bearing; reporting marginal alone makes it look
inert. sumOfAlone sits beside tokensSaved and the gap between them is stated
as the overlap rather than resolved into a total, because a single figure there
is the one number that cannot be true.
The floor is separated out, and finding that was the point. The optimiser
normalises whitespace whether or not any rule is enabled. The first version of
this measurement credited that to the rules — and run over this repository's
tokenizer corpus it reported that the optimiser saved twenty-one tokens and that
every rule was redundant, a sentence assembled entirely out of somebody else's
arithmetic. floor is now its own field and tokensSaved is the rules' work
alone.
"Inert" is always said about the corpus. A rule that finds nothing in these files has not been shown to find nothing anywhere, and the difference is the whole distance between "delete this rule" and "measure it on something else".
And a path defect found by running it: --measure <dir> read the walk's
relative names without putting the root back on, so it worked only from a run
whose working directory happened to be the root — which is the one case a first
probe uses.
Recorded rather than asserted: on the two sample prompts this repository
ships, the deterministic rules recover nothing at all — every rule lands in
inertHere. That is not a defect in the rules or in the samples. It is the
reason the measurement exists, because an aggregate quoted from elsewhere is not
a measurement of what is here.