Changed#
A receipt's rates did not reconstruct its money, and now the money is on the document.
receiptFrompublished the catalogue'sinputPerMTokandoutputPerMTokbeside a totalprofileUsagehad computed, and those are not always the same rates. A model inside a promotional window is billed at the promotion. One whose long-context tier applied is billed at the tier. And a cached read is billed at a fraction of input that no published field named at all — the catalogue holds it as a multiplier, which never reached the receipt.So the obvious check a stranger would run — tokens times the stated rate — disagreed with the stated total, with nothing on the document saying which of the two to believe. In a file whose entire purpose is that a figure still answers when it is read somewhere else, that is the defect rather than a rounding difference.
A line now carries
money: input, cache reads, cache writes and output, as they were actually apportioned, withusdtheir sum. The rates stay as provenance and are labelled as the catalogue's published figures rather than as the arithmetic. Dividing a bucket by its own token count recovers the rate that was really charged, including the cached-read rate, which is the one figure a consumer most wants and the one nothing carried.Two more fields come with it, for the same reason
UsageBreakdownkeeps them:cacheWrite5mTokensandcacheWrite1hTokens, because the ratio between the TTLs is not a constant across providers and a write total that has lost the split can be repriced only by guessing. And a new gap,assumed-write-ttl, because a log that records a write without stating its TTL gets the cheaper rate — the right assumption and the flattering one, which makes the total a floor on those calls rather than a measurement.receipt-arithmetic.test.jsholds all of it: the buckets add to the line and the lines to the document, no bucket holds money for tokens that are not there, the charged rate is recoverable, and a cached read costs less than fresh input. Five plants fire — a bucket dropped, every bucket's money moved intoinputUsd(which still balances, and is what the second check exists for), the money recomputed from the published rates, the TTL split broken and the assumed-TTL gap left unsaid.One plant did not fire on its first run and the plant was right; the fixture was wrong. Zeroing
cacheWrite1hTokenschanged nothing, because every write in the fixture came from an unstated TTL and had landed in the 5-minute bucket. A second log line with a stated split was added, along with an assertion that some line carries a 1-hour write at all — a guard that cannot notice a field being dropped is not guarding it.The receipt has never been published to npm, so no consumer can be holding the older shape and
schemaVersionstays at 1. Said here rather than assumed, because the same change after 1.83.0 ships would have needed a 2.The claim that the four surfaces cannot disagree is now checked. "One core does the measuring; four surfaces carry it. They cannot disagree, because they are the same functions" is on the landing in five languages, and it is the reason a reader is expected to believe the figure in their terminal, the one their agent quotes and the one on the page are the same figure. Nothing checked it.
"The same functions" is an argument, not a proof: two surfaces can call one core and still disagree if either rounds, filters, defaults or windows differently on the way in or out, and each of the four does its own argument handling, formatting and locale.
surface-parity.test.mjssends one log and one prompt through all four and compares what a reader is shown: the money and the call count from the core, the CLI's--json, the MCP server and the browser's own run. The MCP server is spoken to over stdio with real JSON-RPC framing rather than called as a function, which is the standardpackages/mcp/test/server.test.jsalready sets — a tool registered under a name no client can reach is a tool that does not exist.It lives in the web package because that is the only one that can reach all four, and it reads the claim off the landing rather than quoting it, so if the sentence ever goes the guard for it is reconsidered rather than left running against a promise nobody makes.
Proven by planting a disagreement in each surface in turn: the MCP rounding differently, the browser counting one call more, and the CLI repacking a token count on its way out.
security.test.js's subprocess rule was scoped by directory layout rather than by intent. It forbidsnode:child_processoutsidegit.tsacrosspackages/core/src,packages/cli/srcandapps/web— two package source roots, and the web app whole, because the web app has nosrc. Sopackages/cli/testhas spawned the CLI in dozens of files since the guard was written and was never an offender, while the first test underapps/webto speak to a subprocess was. Test directories are excluded by name now, with the reason on the record; achild_processimport in shipped web source is still caught, which is planted and proven.
Changed#
The offline promise is now proven for every command, not three. "Your prompts never leave your machine" is the sentence the README opens with and the reason a reader who cannot use a hosted tool is reading at all. It was proven for
optimize,checkandrules: the right three to pick first, and seven percent of the product.offline.test.jsnow builds a workspace with a real log, a real prompt, a plan and four months of reports, and runs all forty commands that do work against it withfetchremoved.--helpwould prove nothing, so each command gets the arguments that make it read something.Twice each, and compared. Checking only for the stub's marker would pass a command that failed before it reached anything — an early error touches no network either, and forty commands quietly erroring would look exactly like forty commands working. Every command is run with the network and without, and the two have to answer the same bytes.
The invocation table is required to be complete: a command with no entry and no documented reason fails the first check, so a forty-sixth cannot be added without proving it offline. Five are excused by name —
connectandgatewayare declared outbound surfaces thatoutbound-surfaces.test.jsalready holds the list of, andserve,writeandbenchnever return on their own.It runs in eight seconds. The first version took 108, because it rebuilt the workspace for each of eighty runs and building it runs the product six times; the workspace is built once and copied now.
Fixed#
A spend gate passed on a log nobody could read.
trazum profile broken.jsonl --max-usd 50exited 0. So did an empty log, and so did one whose every model is unpriced, on the human path and under--json. The doctrine has a rule for exactly this, learned twice: a period, or a service, nobody measured is not one under budget. Name the gap; never report the absence as a pass.The product had already applied that rule once. A
--sincematching no record throws, and the comment above it says why in the same words: under--max-usdit would pass a budget gate over a period the log does not cover. Three other ways of measuring nothing reached the same gate and were never given the same answer.All four now exit 1 and name which gap they found.
--allow-emptyis how a nightly job says a period with no calls is the expected answer, because that is a real thing to want and it should be said rather than inferred from silence.gate-nothing-measured.test.jsholds the whole matrix — three emptinesses × four gate flags × two output paths — and is proven from both sides: 25 of its checks fail when the refusal is removed, and one fails when it fires without a gate armed.Seven commands answered a mistyped path with a syscall.
optimize,check,profile,position,diff,semanticandconformprintedENOENT: no such file or directory, open '/nope/x.txt', while the five converters answered/nope/x.json: not found. The CLI disagreed with itself about how to refuse, and the majority spelling named a syscall — against the doctrine's a refusal never arrives bare, a rule three other tests already cite.Translated at the top-level handler rather than at each of the seven, because the seven is the part that changes: a command added next year reads a path too.
EISDIRandEACCESare covered with it, including the pathlessEISDIRNode raises fromreadrather thanopen.bare-refusals.test.jsruns every command out ofCOMMAND_FLAGSagainst a missing path and a directory, so a command added later is covered by existing rather than by anyone remembering. Three that neither read a path nor terminate on their own —serve,write,bench— are named with their reason, and the guard fails if the coverage drops below forty commands. It runs eight at a time: in series it took 100 seconds, which is a guard that gets deleted rather than kept.
Changed#
The
profilereport was 1,200 lines of sequential printing inside one function. The four alternative outputs left it in 1.82.x; what remained was the report itself, and it was one concern printed straight down with every local it had ever needed still in scope.It is twelve section printers now, one per heading a reader sees: the cache verdict, the labels and budgets, the levers,
--what-if, conversation history, output shape, input shape, context pressure, mix drift, repeated calls, the comparison against a previous log, and outcomes with coverage.commandProfileis 570 lines, from 2,359 before the arc began.Most of them need two things.
reportCacheVerdictis 134 lines and takes the report and the messages;reportMixDrifttakes the same two. The one with a real parameter list isreportAgainstPrevious, which needs eight — two of them drivers computed three hundred lines earlier and used nowhere else, which was a thing you could only learn by reading both places.Verified byte for byte at every step, the same way the first half was: one fixed log, fifteen flag combinations, both locales, every byte of stdout and stderr compared against a baseline captured before the first extraction.
Changed#
commandProfilewas a single 2,359-line function. The largest thing in this repository by a factor of five, and four complete outputs shared its scope with the report they are alternatives to:--by-source,--dry-run, the money gates and the side-file writer could each only be understood by reading the body they sat in.They are four named functions now, and naming them is most of the point:
--dry-runneeds three of the thirty-one values that were in scope around it, and--by-sourceneeds seven.commandProfileis 1,673 lines.The gates stopped writing to their enclosing scope.
gateFailedandgateVerdictswere declared three hundred lines above where the gates set them and read three hundred lines below, an output channel nothing declared.runProfileGatesreturns them.failedis still read offprocess.exitCodeexactly as it was, because the gates set the exit code and the function reports what they set.One reach forward was removed rather than reproduced: the gate block's failure explanation used
levers, declared several hundred lines later in the same function, which was legal only because the closure that read it ran after.billLeversis pure, so the extracted function asks for it.Verified byte-for-byte:
trazum profilewas run over one fixed log across fifteen flag combinations in both locales, thirty captures of every byte on stdout and stderr, before and after each extraction. Identical every time, and the 1,167 CLI tests unchanged.
Added#
trazum from-langsmithreads a LangSmith run export as a usage log. The fifth converter, and the only one where the unit is wrong before anything else can be right: LangSmith records a run, and a trace is a tree of them. The chain that wrapped a model call carries the same tokens as the call, and the agent above it carries them again, so summing an export bills the same tokens once per level.Only
run_type: "llm"is a call, and every run that is not one is counted out loud. Most of a LangSmith export is chains, tools and retrievers; a converter that turned a thousand runs into three hundred records without saying why would look exactly like one that failed to read the file.The model is refused rather than inferred. There is no model column — the name lives in
extra.metadata.ls_model_nameor in the invocation parameters beside it — and the obvious substitute is right there and wrong: LangChain sets a run'snameto the client class, so pricing a call byChatAnthropicwould attribute a figure to something that is not a model. Those runs are counted inunnamedModeland dropped. A test plants exactly that run and asserts the class never reaches a record.The trace is the conversation.
trace_idspans the calls one request made, which is the identity Trazum's conversation findings need;idis a single call. This is the one converter of the five that can answer those questions rather than declining them.What does not cross:
inputsandoutputsare the prompt and the completion on every run, and neither is read.extra.metadatais a free-form bag the operator fills, so it is read for named keys and never copied — the next thing somebody puts in there is not a leak by default. One marker is planted in each and the whole conversion is grepped for it.What it refuses to invent:
total_costandprompt_cost_detailsare LangSmith's arithmetic over LangSmith's price table. They never enter a record and the figure is printed beside Trazum's rather than inside it.prompt_cost_detailslooks like a cache split and is dollars rather than tokens, so the cache verdicts readcannot-tell.looksLikeLangsmithis planted against a Helicone export, a LiteLLM spend log and an OpenAI usage response: all three carryprompt_tokens, and a detector loose enough to claim any of them would take somebody's export away from the converter that understands it.Forty-five commands.
Fixed#
Two shipped defects the suite had no shape for. It verifies prose and arithmetic exhaustively and verified nothing about what a reader could see or press, so both of these passed CI and were visible on first paint. The landing rendered five of its six sections at zero opacity for two days — a
Revealwrapper whose observer never fired below the fold — and the Write panel opened with two buttons both reading "Skip this" for four days, because a ternary label's falsy branch was its neighbour's label and the falsy branch is the state the panel opens in.Neither needed a browser to catch.
apps/web/test/reachable.test.mjsrefuses content hidden behind client state that starts falsy, a reveal on a timeline that may not advance (the second attempt at fixing the blank landing, also blank), and two buttons in one component that can carry one label at once. It claims a collision between two conditional labels only when the conditions are written identically, because proving mutual exclusion means evaluating them.Every page counted the commands by hand, and three of them were wrong.
README.md,docs/licensing.mdandplugin/README.mdall said 42 commands against a CLI dispatching 45; the README disagreed with itself, saying 42 in its architecture diagram and forty-five in the table of contents four lines below. The landing said 39, in five languages, and spelled its contract count as a word.plugin/README.mdships with the Claude Code plugin, so that one was wrong in an artefact a stranger installs.every-page.test.jsreads every documentedtrazum <command>invocation out ofdocs/and checks it againstCOMMAND_FLAGS, so a renamed command cannot survive in the prose. A wrong total can, and its walk stops atdocs/.derived-counts.test.jsreads every page that describes the product as it is, plus the landing's copy in five languages, and holds five counts to what the product has: commands fromCOMMAND_FLAGS, the playground's subset fromPLAYGROUND_COMMANDS, contracts fromCONTRACT_NAMES, providers from the pricing catalogue, and rules from eitherRULESor the doctrine's headings depending on which the paragraph is about. Counts are written in digits: a figure spelled as a word is outside what any derivation can reach.Fenced blocks, lines naming the release they describe, and the dated files end to end are excluded, because each would otherwise fail on a true sentence or invite rewriting history. The README's architecture diagram is the one claim living in a fence, so it is checked by name.
Sonnet 5 was priced at a number nobody charges, on a timer. The catalogue carried it as $3/$15 with an introductory promotion of $2/$10 running to 2026-08-31 on top. Anthropic cancelled that increase and made $2/$10 the standard price; the table did not know, so on 2026-09-01 every Sonnet 5 figure this tool printed would have risen 50% with no code change, no release and nothing for a reader to notice. Four days out when it was found.
A promotion is the one price change this table can see coming, so
pricing-review.test.jsnow refuses one that expires insideSTALE_PRICING_DAYS— before the next review this product tells readers it performs. That check would have asked the question on 2026-07-17. It cannot catch a provider quietly changing a price, and says so; that is the release checklist's job, and step 7 of docs/releasing.md is now it.The lever table's headline moved with the price, and it turned out to be the product's headline argument. Opus 5 → Sonnet 5 is 60% off, not 40%, and the span "which model a call goes to moves 40% to 80%" was wrong in 15 places: both CLI locales, the type documenting them, two comments in the CLI, the README, the Claude Code skill and the plugin copy generated from it, and the landing in five languages plus its proof row. Its floor had been computed from the $3/$15 list price while every test, transcript and the product's own output used the $2/$10 actually charged, so
docs/commands.mddisagreed with the sample printed four lines below it and the first sentence a visitor reads was 20 points low. Floor and ceiling are now derived fromMODELS, and the guard reads the ceiling and checks the number in front of it, so it holds in a language it cannot parse.The landing's headline figure was not one the product printed. The page says, in a comment above its own copy, that every number on it is the product's own output, and the largest — −37.4%, in the proof row, drawn in the hero ledger and printed on the share card every link to the page renders — described itself as
optimize --level aggressiveover the demo prompt. No prompt in this repository produces it. The demo prompt is the one the Playground tab loads and the Optimiser fills itself with when a visitor clicks through, so it is the prompt whose figure the page is entitled to quote, and the rules take 20.1% off it.marketing.test.mjsused to pin the figure by listing it —['−37.4%', '40–80%', '−50%'], asserted to appear on the page. Both halves of that were the same hand-typed number, so it could catch a figure being deleted and never one being wrong. It now runs the rules over the demo prompt and compares every reduction on the landing and on the share card against what comes out, because the hero draws the figure and the proof row states it and either could drift alone.The picture at the top of the README was arithmetic nothing checked. It is a terminal transcript: a prompt of 238 tokens shortened to 142, priced on Claude Opus 5 at 50,000 calls a month, with the model switch and the Batch API underneath. The prompt is illustrative and allowed to be — a drawing may invent its subject. The money may not: every dollar in it follows from two token counts and the prices of two models, and those prices live in a table that changes. An Opus 5 re-price would have left the first thing every visitor sees showing figures that were right once.
demo-image.test.jsreads the scenario out of the drawing — the token counts, the calls, the output length — and recomputes each figure fromMODELS: the bill before and after, what the rules save, the output and input halves, and what the switch to Haiku 4.5 is worth. Its band comes frombandForrather than the number that happened to be published when it was drawn.Seven copies of one staleness threshold.
profile, the MCP report and the browser's bill each typed45into their own comparison, and four locale strings stated it again in prose. It isSTALE_PRICING_DAYSin@trazum/corenow, and a guard refuses both a surface that compares an age to a typed number and a locale sentence that tells the reader a different one.One review date for seven providers. The date was written on 2026-08-04 already reading 2026-06-24 and never moved, because moving it required reviewing all seven at once: a partial review could only overstate the rest or throw itself away.
PROVIDER_REVIEWEDcarries a date per provider, the catalogue's headline date is derived as the oldest of them, andtrazum modelsprints the breakdown when they disagree — so a reader pricing Claude calls is no longer told their prices are two months old when that half of the table was checked this morning.The README said there was deliberately no staleness threshold. There has been one for as long as three surfaces have been printing a warning past 45 days. Corrected, along with the illustrative day count beside it, which was a number that could only rot.
The Playground's terminal was a paper-coloured card. The one place in the app where a reader types commands was the same white surface as the forms around it, with monospace text in it — so it read as another text box. A terminal is dark; that is its use scene, not a preference, and this repository already draws one at the top of its own README. It is now the same window as
docs/assets/demo.svg: the same dark tokens in both schemes, the same mark in the title bar, and a terracotta prompt. What the README promises and what the page delivers are recognisably the same object.Its lede was the fourth panel heading printed twice; the sentence survives as the empty terminal's own first line, which is where a reader looking at a blank prompt actually wants it.
The Write panel opened with two buttons, both reading "Skip this", and the emphasised one skipped. The primary control's label was
typed ? t.write.tab : t.write.decline— the rail's own label ("Write") once something had been typed, and the decline label while the box was empty, which is the state every reader arrives in. There was no key for the action the button performs, so it borrowed two that were not it.write.answerexists now in both locales, the control names one action, and it is disabled until there is an answer to record rather than quietly turning into the button beside it.Three panels printed their own heading twice. Write, Your bill and Compare each rendered a lede directly under the page header that already states the panel's purpose — read off the rail's label so it cannot disagree with the nav. On Your bill the two sentences were near-identical ("says where the money actually went: which workload, which model, whether caching paid for itself") and the repetition pushed the drop zone, the one thing a reader came for, further down. Each panel now says the half the header does not: the file's shape on Your bill, the privacy claim on Write, and on Compare an empty state naming what the report will hold instead of a card containing the title again.
A coloured 3px left border was the notice style in ten places. The class string was typed at each call site across three panels, each picking its own tone — the callout every framework ships and no design chose, and one that vanishes entirely under
forced-colors, for the reader who most needs the distinction. OneNotecomponent with four tones replaces it, carrying its meaning in the surface and the text colour, both of which survive a high-contrast substitution.A disabled primary button looked like a loading one. shadcn's
disabled:opacity-50on the terracotta fill produced a full-width slab of half-strength brand colour; on Your bill that slab sat under an empty paste box reading "Read the bill", and the only way to tell waiting for you from working was to notice nothing moved. Disabled now drops out of the brand entirely — the muted surface and the faint ink tier — so it cannot be confused with the loading state, which keeps the fill and animates its label.The results column scrolled away from the reader editing what it priced. On Optimise and Compare the answer sat at the top of a column beside a form three times its height, so going back to change the prompt took the report off screen. Both are sticky from
lgnow.The landing page was blank below the fold, and had been since it shipped.
Revealrendered its children atopacity-0and waited for an IntersectionObserver to say otherwise. A full-page capture of the built site showed the hero and two paragraphs on seven thousand pixels of blank paper: the four doors, the measured figures, the licence and the closing call were all in the DOM, laid out, and invisible, because nothing had scrolled and so nothing had intersected. A crawler, a print, a link preview and a screenshot all got that version.The app's own
AnimatedContenthad written the rule down two releases earlier — "content is present before it animates", "failing open is the only acceptable direction when the alternative is invisible content" — and the one page that sells the product had never been held to it.Two fixes were tried and the second was also wrong. Replacing the observer with
animation-timeline: view()removed the script and kept the fault: a scroll timeline has no progress when there is nothing to scroll, so on a document shorter than the viewportfill-mode: bothpins the element at thefromkeyframe, which is zero opacity. Tightening the range moved the boundary without removing it, and a capture caught it again.Revealis deleted. Fourteen identical fade-ups were buying a real risk of invisible copy for the effect the craft floor names as the failure mode — one identical entrance on every section. The page's motion is now one authored moment: the hero's ledger figure filling once on load, time-based so it cannot wait on a scroll position that may not exist, and written so the finished state is the element's own style and the animation is subtracted from it. Text is text, and it is there on the first paint.bg-layerwas a class that painted nothing. The landing asked for it on the story and proof bands; no--layertoken existed, Tailwind emitted no rule, and three sections meant to alternate against the hero were flat. The token exists now, in both themes, a hair off the paper rather than the full sunken step — on a long page a band is a change of ground, not a panel.Two stale figures on the landing, in five languages each. It advertised 39 commands (45) and nineteen contracts (24). Both are the kind of number this repository treats as load-bearing, and both were wrong in every locale.
The prompt editor grew to 900 pixels and pushed its own button off the screen. shadcn's
Textareacarriesfield-sizing-content, so the box grows to whatever is pasted in it. With the bundled demo prompt the input column measured 1307px against a results column of 270 — a thousand pixels of empty paper beside the form — and the Optimise button sat below the fold on a 1000px display, so the first thing a first-time reader had to do was scroll past a wall of somebody else's prompt to find the one control on the page. It is capped at 26rem and scrolls past that; short prompts still get short boxes, which is the point of content sizing.The roadmap promised four things it had already delivered.
ROADMAP.md'sNextsection said the verdict bridge "stays named and waits to be asked for, as do thefrom-langsmith,from-heliconeandfrom-litellmconverters, which need real exports before their formats can be read rather than guessed". All four were inmain. A reader deciding whether Trazum could read their LiteLLM spend log was told, by the file whose whole job is saying what exists, that it could not.Every guard here pointed one way, and this is the other way.
every-page.test.jsfails a page that shows a command the CLI does not dispatch; nothing failed a page claiming less than is built. That direction is not harmless: this project's rule is that a blocked arc stays named and is never faked, and the mirror of that rule is that a delivered arc stops being called pending.roadmap-forecast.test.jsreads theNextandUnder considerationsections, sentence by sentence, and fails when one of this repository's named-not-built phrasings lands in the same sentence as a commandCOMMAND_FLAGSdispatches. It is sentence-bounded on purpose, so a paragraph that mentions a shipped command near an unbuilt one is not rewritten to please a test.The released record is excluded, and a test says so. "The vendor-specific converters stay named and unbuilt until a real export of each is seen" is a true statement about 1.71.0, and sweeping it to match the present would falsify a record — the same objection
token-band.test.jsskips the changelog for. A third assertion pins that sentence where it is, so the exclusion is a decision something checks rather than one that quietly becomes a bug.Proven by planting the sentence back in and watching it fail with "from-litellm is dispatched, and this calls it pending".
The new guard missed the sentence next to the one it was written for. It matched a command inside backticks, and four lines below the paragraph it fixed sat "Vendor-specific converters (LangSmith, Helicone, LiteLLM) are named as next but stay unbuilt" — three shipped converters, named as the products they read rather than as the commands that read them. A guard that only knows one spelling of its subject is a guard with a hole in it.
A
from-xcommand is now also looked for by its subject:from-langsmithanswers to LangSmith,from-claude-codeto Claude Code. Adding it fired on all three immediately, which is how the miss was found.Under considerationstill said nobody had measured the per-family tokenizer error. That entry set its own threshold — within 5% across families the dependency is not worth taking; 40% out and it is — and said deciding without the number would be deciding blind. 1.82.0 measured two families at 94.5% and 103.1%, both more than twice that threshold. The entry now says the question is answered for those two, that four remain unmeasured and what each needs, and that the item stays unscheduled for a distribution reason rather than an evidentiary one.This is the same defect as the one above, in the section the guard does read, and the guard did not catch it: it keys on command names, and this entry names no command. Written down rather than papered over.
An existing guard then caught the new one twice, which is the ratchet working.
publish.test.jsfails any suite that bounds a section of prose by naming the heading after it, "the pattern this repository has had to fix nine times" — and the first draft did exactly that, then wrote the offending call into a comment explaining why it had stopped, which fails too because that guard reads the raw file. The record is now computed as everything the forecast is not, from the same boundary the check itself uses.