Skip to content
Trazum

1.8.0

No date in the file

The first version published to npm, and it collapses everything since the 1.0.0 milestone into one release. Those milestones are numbered 1.1.0 to 1.8.0 in ROADMAP.md and none of them was ever tagged or uploaded — the scope did not exist. Publishing as 1.8.0 rather than 1.1.0 makes the version on npm agree with the record in this repository, which is the only thing the number has to do. 1.1.0 through 1.7.0 will never appear on the registry, because they never existed anywhere a consumer could reach.

Added#

--cache-suggestions: --suggest answers from disk when the question has not changed. A content-addressed cache under $XDG_CACHE_HOME/trazum/suggestions, keyed on provider, model, system prompt and the author's prompt, kept for seven days. Re-running --suggest over a directory after editing two files out of forty makes thirty-eight fewer requests. trazum --clear-suggestion-cache empties it, with no command and no config read.

Opt-in, and printed on stderr every time it answers: a hit is a week-old response from something that is not a pure function, and the other three model-touching flags all make you ask twice.

The raw response is stored, not the checked suggestions. All five checks in suggestRewrites — before appears byte for byte, nothing touches protected content, after introduces none, it actually saves tokens, overlaps are dropped — re-run on a hit, so an answer from last week is judged by this week's rules instead of replaying an older version's verdict. Same reasoning as recomputing token counts on read rather than storing them.

Files are 0600 in a 0700 directory that this code creates. The cache holds prompt text, which is the most sensitive thing the tool touches.

This is not the API's prompt caching, and the reason is a number. The roadmap item asked for cache_control on the stable prefix. The minimum cacheable prefix is 512 tokens on the most generous model and 4,096 on others; Trazum's suggest system prompt is 291 tokens; and a prefix below the minimum is silently not cached — no error, cache_creation_input_tokens: 0. Everything after the system prompt is the author's text, which differs on every call, so no placement of cache_control helps. Marking it would have looked like an optimisation, cost one line, changed nothing, and been undetectable. suggest-cache.test.js measures the prompt against the published minima per model, so if a model ever lowers its floor below 291 the claim fails loudly rather than staying in a comment that has quietly stopped being true.

A README badge at /badge/<token>.svg. The share link, as a picture.

It rides on the share token rather than inventing one. A badge is strictly less information than the page at /c/<token>, and a second capability for a smaller disclosure would have been two things to revoke instead of one.

It is recomputed on every load, like the page. A badge is the single most likely artefact to be looked at a year from now, and a stored number is the most likely thing to have quietly stopped being true — which is the failure mode of every hand-written "saves 30%" line in every README.

It always answers 200. Unknown, expired and malformed tokens all render the same neutral badge. A non-2xx makes GitHub's image proxy show a broken image, telling every reader of the page that something is wrong without saying what; the three cases have to be indistinguishable anyway; and a revoked link should stop reporting rather than announce that it used to exist.

The document is inert — no script, no foreignObject, no external font or stylesheet — and served with nosniff and default-src 'none'; sandbox, because an SVG from your own origin is a page when navigated to rather than embedded in an <img>. Everything interpolated is XML-escaped even though the only inputs are numbers this route computed: "no untrusted text reaches here" is a property one commit can break.

Cached for five minutes, unlike everything else behind a share token. Safe because the token is in the URL, and necessary because a README badge is fetched by every reader of the page through an image proxy.

Nineteen mutants, nineteen killed, after three survived a first pass and each was a test that had only asked the easy half:

  • Only the label was tested for escaping, never the message — so deleting the message's escape changed nothing any assertion could see. Both are constants or numbers today, which is exactly why one of them went untested.
  • textWidth('WWWW') > textWidth('iiii') holds whether or not capitals get their own width, because i is narrow either way. Compared against wwww now.
  • The UUID-guard problem again, one route over: a malformed token produces the same neutral badge whether or not it is refused before the lookup, so the test watches whether the store was asked rather than what came back.

Also two assertions that were wrong rather than the code: one required the SVG to contain no http at all, which its own XML namespace fails, and one required no "/> anywhere, which every <rect> ends with.

A deployment overview at /admin. The last of the team features, and the one whose hardest part was deciding what it is allowed to claim.

The request was "aggregate spending across the org", and neither half of that phrase survives contact with what Trazum knows. It has never seen a bill, an API call or a token counter — it reads prompt text and measures it. A dashboard headed "spend" would print a figure nobody can reconcile against an invoice, and the rule here is that a number a reader cannot reproduce by hand does not get printed. So the headline is input tokens, which is a property of a prompt alone, and the second figure is what running the rules would remove — measured by running them, the standard trazum rank is already held to. No score. The disclaimer sits above the first number rather than in a footnote, because a footnote is read second.

And there is no organisation model, which is also a decision rather than an omission: a self-hosted instance is the team. The alternative was reading GitHub organisation membership, which would mean asking for read:org on every sign-in so that some deployments could skip an environment variable. Sign-in asks for read:user and nothing else, and keeping that true is worth more than the convenience.

TRAZUM_ADMINS is unset by default and unset means the page does not exist — 404, identical to the 404 a signed-in non-admin gets, because a 403 would confirm a dashboard is here and that they are outside it.

It reports counts, prompt names and logins, and never a line of anybody's prompt. An admin is an operator, not an auditor of what their colleagues wrote, and "which prompt is expensive" is answerable from a name. One overview reads at most 500 prompts; past that the page states both numbers instead of reporting a total that quietly covers part of the deployment.

A guard caught its own author. census was first written as a method on PromptStore, with a comment calling itself "the documented hole" in the rule that every lookup binds an owner — and the guard written to enforce that rule failed on it immediately, which was the guard being right. A rule with an exception written inside it is a rule somebody adds a second exception beside. It moved to its own AdminStore interface, the way ShareStore.findShare already had, so "every PromptStore lookup binds an owner" stays a true sentence rather than a mostly-true one.

Fixing that surfaced one more: the guard read PromptStore by slicing from its declaration to the end of the file, which was fine while it was the last interface there and reported AdminStore.census as a hole the moment it was not. Third unbounded slice in this repository to read past the thing it meant — the previous one read a returning list into an on conflict clause.

Sixteen mutants, sixteen killed — but only after three of them survived the first pass, and all three were tests that had encoded the easy case:

  • A GitHub username may be entirely digits. A version of adminSource that checked the numeric-id list against the login as well passed everything, because no fixture had an account whose username collided with somebody's id. It does now: listing 1001 means boss's account and must not admit whoever registered the username 1001.
  • Ranking by size and ranking by waste give the same answer whenever the biggest prompt is also the most wasteful, which was the only case the fixture had. A long prompt of unique prose against a short one full of known filler tells them apart — and telling them apart is the whole value of the ranking, because the long one is what an admin would have guessed.
  • "Some notFound() precedes the census call" stayed true when the admin guard was deleted, because the signed-out guard above it kept the ordering. Each of the four guards is now required by name.

One more fixture lesson, the same one as always: the prompt this suite used to represent "wordy" was "You should always make sure to carefully read the entire text below", which reads bloated and which no rule touches. Every savings assertion was comparing zero to zero.

Share links for comparisons. POST /api/shares publishes a comparison at /c/<token>, readable by anyone holding the URL with no account at all.

This is the first thing in Trazum with a bearer-capability security model, and almost every decision follows from naming that honestly rather than treating it as "the prompt library but public".

The token is the secret, 32 bytes from the same CSPRNG that mints session cookies — not a slug, not a short id, not derived from the content. Stored in the clear, and the asymmetry with sessions is deliberate: hashing a session token means a leaked table is hashes rather than live logins; hashing this one would protect nothing, because the row it points at is the secret.

Reading writes nothing. No view counter, no last-seen column, and the schema says why it does not have one. An unauthenticated request that can cause a write is a lever, and "how many people opened this" is not worth being one.

Expiry is a default, not an option. Thirty days unless you choose 7, 90 or never. A link that never expires is a permanent publication made by somebody thinking about the next ten minutes, so never exists and has to be asked for. Expired is indistinguishable from never-existed — "this link has expired" tells a stranger the token was real, which is one bit more than they had.

Kept out of search twice, by two defences that fail differently: noindex in the page metadata stops indexing, robots.txt stops the fetch. Plus no-referrer, because the token is in the path and one outbound navigation would otherwise put the whole capability in someone else's access log.

The settings are canonicalised from a whitelist. They are replayed into the core on every future view, by a reader who did not choose them and cannot see them, so the parser builds a fully-populated object from known keys rather than merging over what arrived. Numbers are clamped — refusing a whole publication over a call volume of −1 helps nobody — but the model id and rule ids are rejected outright, because a silent fallback would price the comparison against something the sharer did not pick.

The warning lives above the button and is always visible. A confirm dialog is a thing people dismiss; a sentence above the control is a thing they read while deciding.

Eighteen mutants, eighteen killed — but only after the pass found the one that mattered most:

The share URL could have been built from the Host header and no test would have noticed. Every request in the suite was constructed on the same origin as TRAZUM_PUBLIC_URL, so a header-derived URL came out byte-identical and the assertion passed either way. The failure it was hiding is not cosmetic: a link built from a client-supplied host points wherever the client said, and is then handed to a colleague by somebody who trusted it. There is now a request whose Host deliberately disagrees with the configuration.

Also: a test asserting the shared page never reaches for dangerouslySetInnerHTML failed on the page's own comment saying exactly that — the second time this repository has made that mistake, after a schema comment about force row level security. Both tests now strip comments before reading the source.

A prompt library with version history. The first thing accounts were for.

Everything about it follows from one question: what did last month's edit do to this prompt? A store that answers that has to be append-only, has to price every version the same way, and has to be unable to show one person another person's work. Each of those is a decision that was cheaper to make now than to migrate to.

Append-only. Saving over a prompt writes a new row; nothing updates one. A history you can edit is not a history.

A save that changed nothing writes nothing and answers 200 with saved: false — not an error, because pressing Save on unedited text is a reasonable thing to do, and not a duplicate row, because the history's only job is showing what moved. The UI turns that flag into "no changes to save"; a screen that says "Saved" when nothing was written is training its reader to distrust it.

Token counts are recomputed on read, never stored. This is the decision that looks wrong from a caching point of view and is not. The history is a chart. Two versions saved a year apart, each priced by the estimator of its day, produce a line that moves when the estimator changed and the prompts did not. Recomputing every version with today's costs a little and is the only way any two of them are comparable to each other.

Somebody else's prompt is 404, not 403. A 403 confirms the id is real, which turns the route into an oracle for enumerating other people's libraries, and a legitimate caller can do nothing with the distinction because they were never getting in. Enforced in the query rather than after it: every store method takes an owner id and puts it in the where clause. PromptStore has no lookup that takes an id without an owner at all, so a handler cannot tell "not yours" from "not there" — the mistake is unrepresentable rather than untested, and a guard pins that shape because the way it comes back is somebody adding a convenience method.

Ceilings — 200 prompts, 500 versions, 100k characters — are refused loudly rather than trimmed silently. Evicting the oldest version to make room deletes the record the prompt was kept for.

Seventeen of eighteen mutants killed, including every ownership predicate in both drivers. Three findings from the pass, all of which had passed the suite first:

  • The Postgres prompt driver had no tests at all. The route suite runs entirely on the memory driver, so every ownership assertion in it passed against SQL that selected by id alone. prompts-postgres.test.mjs now drives all six methods and sweeps the recorded statements mechanically: anything that names a prompt must also bind an owner, and anything that binds one must compare it.
  • The UUID check on the path segment looked decorative and is not. Every test around it asserted the outcome — 404 — which holds with the check removed, because the memory driver answers null for any key it does not hold. Bind '../../etc/passwd' to a uuid column and Postgres raises, which is a 500 where the caller should have had a 404. What distinguishes the two is not the answer but whether the store was asked, so that is what the test now watches.
  • One mutant survives and is documented on the line it survives. The memory driver's version sweep on delete changes nothing observable: the prompt is gone, ids are UUIDs, nothing will ask again. What it costs is memory — which is exactly the kind of line somebody deletes during a cleanup because no test complained.

Changed#

The prompt lives at page level now, next to the scenario. The Library tab has to save the prompt that is on screen and put a restored version back on it, and neither is possible for a sibling holding its own copy — two components with two copies of "the prompt" is how a library quietly stores something else. Same reasoning that moved the usage scenario, same shape: a hook the page owns.

A tab test counted tabs when it meant to check a property. "Keeps both tabs mounted" asserted there were exactly two TabsContent and that both were forceMount, and it failed the moment a third arrived — on a tab that deliberately is not one, because the library holds nothing the server does not already have and is better re-read on return. Rewritten to assert the actual invariant: a tab holding state the server does not have must stay mounted, and anything opting out has to be named. Mutation-tested both ways.

Sign in with GitHub. Optional, off by default, and the first thing in this repository that stores anything about a person.

The reason it exists is that everything a team wants — a prompt library with history, a budget somebody else set, an organisation's spend in one place — needs an answer to "who is asking", and there wasn't one. So this change is the foundation and deliberately not the features: identity, a session, and a store those can be built on.

Off by default is load-bearing rather than polite. Most people running Trazum run it for themselves, and a tool that suddenly requires a database to start is a tool they stop running. With no TRAZUM_GITHUB_CLIENT_ID, authConfig reports disabled with a reason, /api/auth/* answers 503 naming the variable to set, and the header renders nothing at all. Not a disabled button: a disabled button is a promise, and the endpoint behind it is a 503.

No framework. The OAuth flow is about two hundred lines — authorize, exchange, read the profile — and hand-writing it keeps apps/web at one new dependency (postgres, which has none of its own) instead of a tree. The three things a framework would have gotten right are gotten right here, and each is worth naming because each looks like a detail:

  • The redirect URI is built from TRAZUM_PUBLIC_URL and never from a request header. Host is client-supplied; a callback built from it lets an attacker send a victim to /api/auth/github with Host: evil.example and collect the authorisation code. GitHub's own callback allowlist would catch that one case, which is not a reason to depend on one checkbox in someone else's console.
  • state is verified before the code is exchanged. Verifying it afterwards passes every functional test and defends against nothing, so the order is asserted directly: the test counts calls to a recording fetch and requires zero.
  • The __Host- cookie is cleared as Secure even when the deployment is not. A __Host- cookie without Secure is rejected outright — including the one meant to delete it — so getting this wrong makes sign-out appear to work and do nothing.

Sessions are opaque 256-bit tokens rather than signed claims, stored as SHA-256. Revoking one is a DELETE, which is the property a JWT does not have, and a leaked table is hashes rather than cookies. Sign-out deletes the row before clearing the cookie; the other order leaves a live session the browser has forgotten — invisible, unrevokable, valid for a month.

Two store drivers behind one interface. Memory is the default and reports ephemeral: true, which /api/auth/session passes to the browser and the header draws as "temporary session" — because on a platform that runs several instances the alternative is being signed out at random with no explanation. Postgres is the other, and its schema enables row level security with no policies: that blocks the REST layer platforms like Supabase put in front of public, while Trazum, connecting as the table owner, is exempt. ENABLE and not FORCE — FORCE applies the policies to the owner too and locks the app out of its own tables, which is the stricter-looking word and the one that takes the site down.

Twelve mutants, twelve killed: state checked after the exchange, the __Host- delete without Secure, the redirect filter without //, the expiry boundary at < instead of <=, the sweep that could delete a live session, an upsert that resets created_at, a token exchange that trusts the HTTP status (GitHub answers a bad code with 200 and an error field), storing the raw token, accepting HTTP off localhost, a cacheable session response, and a cross-origin sign-out.

Honest about the gap: the Postgres driver has never run against Postgres. Its SQL is checked against a recording tagged template, which catches a mistyped column, a value bound in the wrong position and a DELETE whose predicate is too wide, and cannot catch SQL that Postgres would reject. docs/accounts.md says so, and lists the rest of what is not covered.

Added#

Guards that the published packages are actually public. A scoped package is restricted by default, and this project is open source — so the two manifests carry publishConfig.access: "public" and both release steps pass --access public. Both were already correct; nothing checked either.

The failure that would have gone unnoticed is the quiet one. On a free account a missing --access public fails the publish, which is fine. On a paid one it succeeds and uploads a package nobody outside the org can install — a release that looks entirely normal, and unpublishable after 72 hours.

Three assertions: every publishable manifest declares public access; the release workflow publishes exactly the publishable set, each with --access public and --provenance; and nothing is publishable by accident, with apps/web staying private: true so a Next application never reaches a registry as though it were a library.

The set of workspaces is now derived from the root workspaces globs rather than listed. It had been a hardcoded pair, which made the whole file blind to any workspace added after it was written — a new publishable workspace now fails the suite until somebody decides what it is. Eight mutants, all killed.

Route invariants, checked against every route rather than the ones with tests. Five API subsystems landed in five consecutive merges — auth, share links, the library, the admin overview, the badge — each getting its rules right because the author remembered them. Nothing was checking, and "remembered five times" is not a property.

Two invariants, read from source: every state-changing handler in a route that reads credentials reaches a same-origin check, and any response a route builds by hand carries no-store. Routes are walked from app/api; the exemption for /api/optimize and /api/compare is derived from their reading no cookie and no session, not from a list, so either one stops being exempt the moment it reads one.

The gap this closes is measured, not asserted. Deleting the same-origin check from DELETE /api/prompts/[id] passes the entire pre-existing web suite: every write funnels through one requireCaller, so the behavioural tests prove that function refuses a hostile Origin and prove nothing about whether a given handler asked it to. Five such mutants survive everything except the new file. Going the other way, changing requireCaller's condition to if (false) survives the new file — sameOrigin( is still there to match — and is killed by the behavioural tests. Neither layer is redundant; neither covers the other's mutant. Ten mutants, all killed once both layers run.

Both guards were wrong before they were right. The cache-control one originally asked whether a route used the response helpers, which is a proxy for the property rather than the property, and failed against /api/auth/session — a route that sets no-store correctly on both branches without them. Its stated premise was false too: jsonError and redirect set no cache-control at all, and the comment claimed they did. They carry no session data, so that is fine; asserting it without checking was not.

Fixed#

A rate limiter that could be turned into a way to take the deployment down. The expired-entry sweep ran on every miss once the map passed sweepAbove, and a miss is any key not seen before. clientKey reads x-forwarded-for, which this module already documented as freely spoofable — so an attacker rotating that header made every request a miss, and every request an O(n) walk of a map their earlier requests had grown. Almost nothing was reclaimed while they did it, because entries in the current window have not expired yet.

Measured rather than reasoned about:

                 before                            after
N= 20000    1,410ms   149,985,000 compares      14ms   10,001
N= 40000    7,576ms   749,975,000 compares      19ms   10,001
N= 80000   46,759ms 3,149,955,000 compares      79ms   10,001

Doubling the requests multiplied the work by 4.4, then 7.6. Eighty thousand requests — not an interesting number of requests — was 46 seconds of a single-threaded event loop during which the deployment serves nobody. The limiter answered every one of them correctly; it just answered quadratically.

Sweeping at most once per window makes the total linear. Memory is unchanged and worth stating: a window's worth of distinct keys is held until the next sweep, which is inherent to counting per key rather than a consequence of the fix, and it does not accumulate across windows.

The limiter now exposes a sweeps count, so the test asserts the sweep's frequency instead of trusting a comment about it — the bug was invisible from outside, since every verdict it gave was right. Eight mutants, all killed. Asserted on the count and not on elapsed time: a timing assertion on shared CI hardware is a flake generator, and the count is the thing that changed.

The roadmap said the web app's features would deliberately never be built. ROADMAP.md listed Prompt library under "Under consideration" with the reasoning "storing prompts is a different product, and one that would mean sending them to a server. Trazum's privacy story is that it never does" — while the app shipped a prompt library with version history, share links, an admin overview and a badge. Seven user-visible surfaces existed and the document mentioned none of them, one of them by explicitly explaining why it never would.

That reasoning was also a conflation worth naming rather than deleting. Rule 1 binds the optimiser: the CLI sends nothing, needs no account and works with the network unplugged, and none of that changed. It never said nobody may run a service that stores what they chose to save to it. The entry is struck through and kept, because a roadmap that silently removes what it went back on has no record of having gone back on anything.

Added: milestone sections for the account-and-sharing work and for --cache-suggestions, and a guard deriving every route from apps/web/app and requiring the roadmap to name it. The guard is deliberately narrow — it proves the document knows a surface exists, not that what it says about it is true — and it caught two of its own defects first: a substring match let /api/admin satisfy /admin, and /c counted as documented by accident because two characters occur inside almost any path.

Japanese is now stated as a deliberate absence rather than a gap. There is no Japanese trimming dictionary and one is not planned — deciding a phrase says something in more words than it needs is a judgement about the language. But --reorder's backward-reference list does cover Japanese and Chinese, matched without word boundaries. Those are different claims: refusing to rearrange needs only enough of a language to spot a phrase pointing backwards, while offering to shorten means asserting the shorter version still asks for the same thing.

The help-text guard could not see a flag that belongs to no command. It derives its list from the unknown-flag rejection message, which is printed per-command — so --clear-suggestion-cache, an errand that runs with no command named, was documented nowhere in either locale and the suite stayed green. A second guard now reads main() for flags handled before dispatch and requires each in the help. Both are derived rather than listed, which is the reason the first one exists: a hardcoded list is how --reorder shipped fully implemented and absent from --help.

That flag also did not work. It was handled below the branch that prints usage when no command is given, so trazum --clear-suggestion-cache printed the help and cleared nothing — silently, which is the failure mode the flag list is there to prevent. Found by an end-to-end test counting requests at a socket rather than by reading the code.

A control-character filter written with control characters. The first draft of the ?next= guard spelled its character class literally, which put a NUL and a run of C0 bytes into github.ts and made the source file binary — the same defect this repository shipped once before in reorder-properties.test.js and had to be told about by a guard. Replaced with a code-point comparison, which cannot carry the bytes it is checking for. An intermediate repair was worse: a class written as [ -] matched space through hyphen, so every path with a hyphen in it — which is most of them — was silently redirected to /. Both are pinned by tests.

RELEASES.md said "Nine commands" for two merges after there were ten. The guard written to catch exactly this drift read README.md only, so correcting the README was mistaken for correcting the count. Widened to both files — and then found to be blind anyway: the pattern was lowercase-only and the sentence starts with a capital, so Nine commands had never been visible to it. Both fixed, and the guard mutation-tested in both files and both cases rather than trusted.

Widening it also made it cry wolf on "the two commands that answer which prompt is worth an afternoon and who made this one expensive", which is correct prose counting a subset. A restrictive that or which now marks a subset the way other already did — a guard that fails on true sentences gets deleted.

Changed#

Rate limiting is one function with private buckets. Both API routes carried a copy of the same sliding window, with a comment on one saying the duplication was deliberate because a shared Map would let comparisons spend the optimise budget. That is right about the state and wrong about the code: createRateLimiter hands each caller its own Map. The sign-in routes were going to be the third and fourth copy.

Its limit is thirty a minute per address rather than the ten a sign-in route appears to need, because the limiter keys on an address and not on a person: ten is generous for an individual and refuses the eleventh person behind a corporate NAT.

The price list says how old it is, not just when it was checked. doctor and models print Prices reviewed 2026-06-24 (46 days ago).

Every dollar figure Trazum prints descends from that list. The date alone makes the reader subtract against today to learn the one thing they wanted — whether to trust the figures — and a reader who is not already suspicious will not bother, which is exactly the reader the line is for.

No threshold and no warning: "stale" would be a number nobody could check, and the age is the fact. reviewAgeDays takes now as a parameter, for the reason computeSavings takes a Date. Compared at UTC midnight on both sides, so the answer does not shift by one depending on what time of day the command runs, and so a daylight-saving gap cannot turn two days into one.

A future date reports unknown rather than negative days — that is a typo or a wrong clock, and "reviewed in −12 days" reads as a bug either way.

A guard that no test could distinguish, until one could. Deleting the YYYY-MM-DD format check failed nothing: every malformed value it had been checked against is NaN to Date.parse regardless. The input that separates them is "2026-06" — Date.parse("2026-06T00:00:00Z") is 2026-06-01, a day nobody wrote, and without the guard an overlay carrying that string gets an age computed from an invented day of the month and printed as confidently as a real one. Found by mutation, and now pinned.

Added#

suggest-fixes: true on the Action — the optimised prompt as a GitHub suggested change, applied with one button.

A suggestion, not a commit. The obvious build commits the fix to the pull request branch, which needs contents: write on the workflow of everybody who installs this action, against a SECURITY.md that documents contents: read, no pull_request_target, and has a test asserting it. Widening that is a decision for the people running the workflow. A suggestion block needs only the pull-requests: write the comment mode already requires, lands in the same place with the same one click, and leaves the maintainer as the one who commits.

Safe level only — the aggressive level is defensible when a human reads the diff it produced, and a one-click apply is not that moment; asserted by a test that also refuses to let the level come from the environment. Never fails the build: no pull request, a read-only fork token, an oversized prompt or a partly-changed file are all notices.

Two defects found by running it, both invisible to reading:

  • It would have suggested deleting the trailing newline of every prompt in the pull request. optimize returns text with no trailing newline even when no rule fires — "Classify {{t}}.\n" comes back as "Classify {{t}}." with rules: [] — so a plain optimized !== original is true for virtually every file on disk. The comparison now ignores the terminator, and the suggestion omits it, because inside a fence the lines are the replacement lines.
  • Every anchor was one line past the end of the file. "abc\n".split("\n") has length 2, so line pointed at a phantom last line and GitHub would have answered 422 — on essentially every file, quietly, as a declined API call.

One test fixture was wrong too: 600 identical padded paragraphs collapse to 42 characters, because the duplicate-blocks rule removes them all, so the size guard never fired. Distinct paragraphs now.

23 tests, five mutants — both defects above, a hunk parser that requires the optional comma, suggesting on a partial diff, and the aggressive level.

Added#

trazum doctor --otlp-out <file> — the survey as OpenTelemetry metrics. Five gauges: tokens per prompt, over-budget per prompt, the unbudgeted count, and each advisory's monthly figure and prompt count. The model and call volume are resource attributes, because a dollar figure whose scenario is not recorded beside it is a number nobody can check later.

Trazum writes the payload; it does not send it. Pushing to a collector means holding an endpoint and a credential, and this project has twice shipped an SSRF where a URL reached fetch without being the URL that was checked. A file has no such failure mode, and the pipeline that already holds the credential can post it.

No @opentelemetry/* dependency: the JSON encoding is a documented wire format and this package has none. Two of its rules fail silently — a collector does not reject a malformed payload, it charts it wrong — so both are pinned:

  • 64-bit integers are JSON strings. timeUnixNano and asInt are int64; a collector reading 1786000000000000000 as a double loses the last digits of every timestamp it stores.
  • Money is asDouble. asInt would report $4,912.40 as 4912, and nothing downstream would look wrong.

toOtlpMetrics takes the timestamp as a parameter rather than calling Date.now(), for the reason computeSavings takes a Date: a function that reads the clock can only be asserted for shape, and here the timestamps are half the payload. Eleven core tests, four CLI tests, five mutants — each of the two encoding rules, the millisecond conversion, empty series, and unbudgeted prompts reported as within budget.

Fixed#

A test file was binary, and the guard against exactly that did not look in test directories.

packages/core/test/reorder-properties.test.js joined a token bag on a raw NUL:

const bag = (text) => text.split(/\s+/).filter(Boolean).sort().join('<NUL>');

One byte, typed as a literal instead of \0. git calls such a file binary, so every change to those 400-prompt property tests rendered as Bin 8385 -> 8386 bytes and no reviewer could read a line of it.

This is the same defect as scripts/measure-token-band.mjs, which spent three commits unreviewable — one of them fixing a security finding in that file. The guard written afterwards, every source file is reviewable as a diff, walked packages/core/src, packages/cli/src and scripts. The relapse landed in packages/core/test, one directory outside its reach, and the guard sat green beside it for as long as it existed.

A test directory is where it matters most. Tests are the argument that the code is right; a test nobody can read in a diff is an assertion taken on trust, which is the thing this repository spends its effort refusing to do.

The walk now covers both packages' src and test, apps/web, action, scripts and .github, and the staleness floor rises from 40 files to 100 so a future re-narrowing fails loudly instead of passing over less. Verified by putting the byte back: the widened guard names the file, the old roots do not.

Repo-wide scan afterwards: 341 text files, one NUL, now zero.

Added#

trazum diff --all <before> <after> — a whole prompt library at once. diff answered the question for one prompt; a team refactoring forty of them wants it answered forty times and totalled, and running the command by hand loses the total, which is the figure the decision turns on.

A prompt on only one side is named, never counted. A refactor that deletes a prompt and one that renames it look identical from a token count, so folding the deletion into the total would report a library getting cheaper when a file went missing. They are listed under only before / only after and excluded from the totals, with a line saying why.

--max-growth applies per prompt, not to the total — the rule check already states about budgets. In the worked example the total is +3 and --max-growth 10 still fails, because one prompt grew 14 while another shrank 11. A gate on the total would pass that, and the prompt that doubled is the one somebody has to look at.

Sorted worst first, the sign convention stated above the first figure, and the totals asserted to equal the sum of the per-prompt figures they claim to total. Four mutants, each killed: deletions folded into the totals, the gate moved to the total, the convention moved below the figures, and the list sorted best-first.

The command count in the README drifted again while this was being written. Adding doctor made "nine commands" wrong in two places, in a file corrected two commits earlier for exactly this. publish.test.js now checks it against COMMAND_FLAGS, and checks that every command is mentioned in the README at all — a command nobody documented is a command nobody runs. The guard distinguishes "ten commands" from "the other nine commands", because the second is correct prose and a guard that cries wolf gets deleted.

Added#

A pre-commit hook, at scripts/pre-commit. ln -s ../../scripts/pre-commit .git/hooks/pre-commit and a commit whose own prompts are over budget is refused before it reaches CI.

It asks trazum doctor --json rather than running trazum check prompts/, and that is the design rather than an implementation detail. check exits 1 when anything under a directory is over budget — right for CI, wrong for a hook, because it would refuse a commit that touches one file over a different prompt somebody else committed last month. A hook that fails for reasons outside the commit is a hook people learn to pass --no-verify to, and then it is worse than no hook, because it taught them the habit too. So the hook intersects doctor's over-budget list with the files actually staged.

It gets out of the way rather than guessing: nothing staged, no Trazum installed, no prompts found, an unreadable config — none of those are a budget failure, so none of them block a commit. Each says so once and exits 0.

Two defects found by running it, both of the same kind — a message that promises something it does not deliver:

  • It announced a list of over-budget prompts and printed nothing under it. The detail line matched path, tokens and budget on one line, and doctor --json pretty-prints. It blocked the right commit and said nothing useful about why. It names the paths now and points at the command for the figures, rather than parsing multi-line JSON in sh.
  • The scope guard latched open on an empty array. "overBudget": [] closes on the line that opens it, so inside = 1 stayed set for the rest of the document and a path key in any later section blocked the commit — the exact bug the scoping was added to prevent. Ten tests, four mutants, including one that puts each defect back.

Stated in the README because it is real: Trazum reads the working tree, not the staged blobs, so a prompt staged and then edited further is judged on the newer text.

Added#

trazum doctor [dir] — the survey before the gate. Which prompts nothing is watching, which are already over budget, and what every advisory adds up to across the whole workspace.

There is no score, and that is the design. A health check invites one, and a number assembled from weights nobody can reproduce gets quietly tuned until the output looks right — rank refused it for the same reason. So doctor invents no judgement at all: every finding is an advisory optimize raises on that prompt on its own, summed, so any figure can be checked against a single file. A test adds up the individual runs and requires the total to match, which it does to the last float. "16 prompts only need a cheaper model" is sixteen copies of one advisory, each with a file name beside it.

Two things it reports that nothing else did: prompts no budget pattern matches, because an unwatched prompt is how the cost got there, and prompts already over budget — before a red build says so, which is too late to think about it.

It exits 0 even when it finds things. trazum check is the gate. The model recommendation is a keyword heuristic, and gating a build on a keyword heuristic teaches people to re-run until green, which costs more than the tool ever saves.

Offline and free, like the rules. It deliberately does not check prompts against their own --suggest recommendations: that is an LLM call per prompt, and this is the command you run over forty files before deciding to spend anything. Four mutants, each killed — averaging instead of summing, dropping the advisories that carry no figure, printing the whole unbudgeted list instead of capping and counting it, and exiting 1 on a finding.

Fixed#

The README described a tool that only shortens prompts, and two counts in the docs had drifted.

Nine commands have landed since that front page was written, and it had been updated by inserting a paragraph into whichever section each one belonged to — which is exactly how a summary goes stale while every detail below it stays correct. "What it actually does" listed four things, all of them optimize. A reader who stopped there never learned that rank says which of forty prompts is worth an afternoon, that blame names the commit, or that four commands write markdown for a pull request comment. There is a table now, and the architecture diagram includes the Action, which it had never mentioned.

Layout was missing a whole workspace (action/), plus scripts/ and the eleven core modules added since it was written.

Two numbers were simply wrong. The README advertised 580 tests where the real figure had reached 798. RELEASES.md claimed thirteen deterministic rules where there are twelve, and listed "restated output formats" among what they cut — an advisory that is deliberately never cut, so that sentence was wrong twice.

Both are now checked rather than corrected. publish.test.js compares the rule count in prose against RULES.length, refuses to let an advisory be described as something the rules cut, and fails if the README ever advertises a test total again — because a total across four suites cannot be verified from one of them, and a number nobody maintains is worse than no number. Three mutants, each killed by reintroducing the exact error it replaces.

A blame test threw away the evidence when it failed. shows the most recent N was the only test in blame.test.js that checked neither the exit code nor passed the command's output into an assertion, so when it failed once on CI the entire report was 0 !== 3 — which is what "the table has no rows" looks like whether the history was short, the path was rejected, or git never answered. The output was right there in the variable, unused.

It carries the output now, and asserts the exit code first. Verified by forcing the failure: the report goes from 0 !== 3 to Error: git has no commits touching p.txt.

This is not a fix for that failure. It has never reproduced here — 26 runs including under 8× CPU load — and the identical commit passed in the same CI minute on the other event, so it is intermittent and its cause is still unknown. What changed is that the next occurrence will say something.

Added#

comparePrompts reaches the web app, as a Compare tab and POST /api/compare. Two versions of a prompt, the token delta, what it costs, and which advisories and rules the edit introduced or resolved.

The whole hazard of this surface is a sign. Everywhere else in Trazum a positive number is money you get back; here every figure is after - before, so positive means the edit made things worse. Getting that backwards throws nothing, fails no typecheck, and tells somebody their prompt got cheaper on the commit that doubled its cost. So the convention is stated above the figures rather than beside them — a reader arriving from the Optimise tab has the opposite expectation already loaded, and a caveat under the number is a caveat read after the conclusion — and most of the new tests assert a direction rather than a value, including the swapped-pair case that a Math.abs mutant walks straight through otherwise.

optimizeBoth is off by default and the default is the interesting half: the edit changed the text as written, so the text as written is what the reader is being asked about. Trimming both sides first hides a prompt that doubled in length and happened to double in courtesy. Honoured only on a literal true, like every other boolean these routes take. A missing before and a missing after are told apart, because one message for two fields leaves the caller guessing.

The usage scenario is now owned once, by the page. It sits beside the locale in App, for the same reason and with the same shape: both tabs price their answers through it, and setting 50,000 calls on one while reading 10,000 on the other would make the two answers incomparable while looking like they were about one workload. Optimizer reads it from a prop and writes back through it; the history panel restores all five fields in one update rather than five, because five setters on shared state is five renders and, in between, four scenarios that are nobody's.

And formatUsd stops being defined twice. The web app had a copy that was byte-identical to the one @trazum/core exports, for as long as it existed. The core also had the formatSignedUsd that Compare needed.

Nine mutants. Two are worth naming. One test asserted the absence of local scenario state in Optimizer, and a mutant renaming the local to callsPerMonth2 satisfied every pattern looking for const [callsPerMonth, while the two tabs went back to disagreeing — a test that enumerates ways to be wrong is always one rename behind, so it asserts the positive property now: every field read from scenario.usage, every setter delegating to scenario.set. The other is that the sharing behaviour cannot be seen from source at all, and was verified by driving the built page in a headless browser: set the calls on Compare, read them back on Optimise, and the reverse.

rank and blame take --markdown-out. The flag existed on check and diff, which meant the two commands that answer which of these forty prompts is worth an afternoon and who made this one expensive could not put their answers where those decisions get made. Both now render a table for a job summary or a pull request comment, written before any exit code is set and independently of --json — the file's job is to survive the run.

Every string but the heading comes from the same t.rank / t.blame objects the terminal report reads. Not tidiness: a second copy of "there is no score" is a second thing to keep true, and whoever eventually softens one of those sentences will soften the copy they happened to be looking at. blame's priced movement moved into a shared netCostOf for the same reason — two copies of that arithmetic is precisely how a comment and a job log start disagreeing about one history.

A third escaper, mdTextCell, for untrusted prose in a table cell. mdCell is safe and announces "this is code" by wrapping in <code>; the first draft of the blame report used it for the author and the subject, so the table typeset a person's name as a code span and an English sentence with it. Correct, and plainly wrong the moment it was rendered. The new one keeps mdCell's entity encoding — no | in the output at all, so the row cannot split — and adds the inline-markdown set, which mdCell never needed because backticks make its content literal.

This is the least trusted input in the repository: on a pull request from a fork, the commit subject is written by whoever opened it, and it lands in a table maintainers read. Verified by building a repository with grow | </table><script> as a commit message and asserting every row still has five cells.

Five mutants, each killed. One of them was killed only by a fixture coincidence — reverting to mdCell kept the table intact and failed just the hostile test, because &#124; happened to differ — so a test that asks the actual typographic question went in beside it.

--suggest is on the web too, as two switches: one asks the model for phrase-level rewrites, the second takes them. Turning the first off clears the second, and the request derives the pair rather than sending both — a switch out of step would otherwise produce a 400 for a combination nobody chose.

The proposals sit above the saving, one line each, with a count of what the checks threw out. Same placement and the same reason as the reordering notice: a figure read before its caveat is a figure nobody agreed to.

And the web suite calls the route handler for real. Every test in apps/web until now read source and asserted on the text of it, which was honest about the two rendering bugs it was written for and could not have seen the one below. test/api.test.mjs sends requests: next/server is redirected to the platform's own Response.json through a stable module.register hook — not --experimental-test-module-mocks, because a suite should not depend on a flag a Node minor can rename — and the core, the rules and both catalogues are the real modules. Eight tests, four mutants, each killed by the one test meant for it.

apps/web/package.json declares "type": "module" as part of this. Loading a .ts route from plain Node otherwise warns on every run that the file "doesn't parse as CommonJS" and is being reparsed — a fair warning, and one worth fixing rather than muting with --disable-warning, since every config in that directory was already .mjs. Build, typecheck and both suites verified after.

The CI step that runs them is renamed from "Core and CLI tests" to "Tests (core, CLI, web, Action)", which is what npm test at the root has been doing for a while. A step labelled for two of the four suites invites a reader of a green build to think the other two are unchecked, and invites somebody adding a suite to write a step that already exists.

Fixed#

applySuggestions on its own returned 200 and applied nothing.

Found by sending it, not by reading the diff. The response came back with a complete report, no suggestions key, and the prompt untouched — the one thing the caller asked for silently did not happen. Nothing about the source looked wrong: the field parsed, the literal-true guard around it was correct, and the branch that would have used it was simply never entered.

That is the same failure as a misspelled field being accepted, which this endpoint already refuses for disableRules and usage.model, and which the CLI already refuses for the matching pair of flags — "a flag that quietly does nothing is the same failure as a typo'd flag being accepted". There was no reason for the HTTP surface to be the lenient one. It is a 400 now, refused before any call to the model, so a malformed request never costs one.

Only a literal true is refused, because only a literal true would have been honoured: applySuggestions: "false" asks for nothing and gets nothing, which is what it says.

Security#

The trimming rules only trimmed in two languages, and the report did not say so.

--reorder was fixed to refuse safely in nine languages. The rules that actually cut tokens still had dictionaries for English and Spanish alone, so:

en   22 →  11   2 rules
es   25 →  14   2 rules
fr   25 →  25   0 rules     ← nothing
de   20 →  20   0 rules     ← nothing

A French or German author ran Trazum, read No rule found anything to trim, and took it to mean their prompt was already efficient. It meant Trazum could not read it. Same defect --reorder had, one layer over: the tool knew something it was not telling anybody.

Two fixes, in that order. The report now names its coverage whenever no rule fires — stated rather than detected, because guessing a prompt's language is one more thing to get wrong and naming the coverage cannot be. And French, German, Portuguese, Italian and Dutch join the six dictionaries, so the trimming and the reordering finally cover the same set of Latin-script languages.

Fixed#

A dictionary translated word by word changed meaning. The first pass at those five languages shipped muito, molto and heel as intensifiers. All three are also quantifiers, and INTENSIFIERS is dropped outright at the aggressive level:

Hai molto tempo per rispondere.   →   Hai tempo per rispondere.

"You have much time" became "you have time". Spanish had this right all along — muy is on the list and mucho deliberately is not — and translating word for word instead of by role lost the distinction. Found by running the five languages through the rules, not by reading the list.

Both halves are now tested. One suite keeps those three words out and asserts the quantifier sentences come back byte-identical. The other counts entries per language per dictionary, because the behavioural test passes on whatever the fixture happens to contain — which is exactly how a two-language hole survived a full suite for this long. Portuguese and Italian dropping to a single firing rule the moment molto came off the list is what surfaced the need for it: the fixtures had been carrying the claim.

Added#

RELEASES.md — release notes for people, and the workflow now publishes a GitHub release from them.

Until now, tagging published to npm and created no GitHub release at all. The tag existed, the page behind it was empty, and anyone following a "what changed?" link arrived at a file list. This changelog is thorough and it is not what you hand somebody who has forty seconds — it is the maintainer's record, written for whoever has to understand a decision two years from now.

scripts/release-notes.mjs extracts one version's section, and the release job pipes it into gh release create. Writing the notes in a pull request beats typing them into a web form at the moment of releasing, which is the moment least suited to writing anything carefully.

Five tests make the file load-bearing rather than decorative: the version in the manifests must have a section, the newest section must be the pending release or the current version, the file must say nothing is published while nothing is published — the exact claim ROADMAP.md got wrong — and the extractor must return one section and fail loudly for a version it has never heard of. All checked against mutants, including one that makes the extractor swallow the next release's notes.

The release job's contents permission widens from read to write, which gh release create requires. Stated rather than slipped in: that job now holds a token that can push to the repository. The checkout is still persist-credentials: false, so the working tree's remote has no token to reuse, and the only step touching the API creates a release from a file already in the commit.

Fixed#

ROADMAP.md filed five versions under "Released" that were never released.

There is no git tag in this repository, the @trazum scope does not exist, and this file — which states at the top that Unreleased means merged-but-untagged — holds every one of 1.1.0 through 1.5.0 right here. Two documents, one of them wrong, and nothing checking either against the other.

It matters beyond tidiness: "Released" is what somebody reads before deciding whether they can install this. The discipline in this repository is not claiming what has not been checked, and this was the least-checked claim in it.

Those milestones now sit under Merged into main, not yet released, which says what is true: the ordering is a useful record, the numbers will not appear on npm, and the first publish collapses all of it into one version. The two things needed before any of it ships are named there, and both belong to the maintainer.

Three tests keep the record honest from here — every version the roadmap calls released must have a changelog entry, nothing under "not yet released" may already be released, and the manifests must carry the newest version the changelog has actually cut. All three were checked against mutants; the first reproduces the original bug by name.

Also on the roadmap: an entry for the five commands merged today, and cost alerting added to Under consideration with the reason it is not scheduled — it needs a service holding other teams' prompt metrics on a schedule, and check --max-tokens in CI already covers the threshold case for anyone content to have the answer arrive on a pull request instead of in Slack.

Added#

trazum eval --export promptfoo — hand the run to your own harness.

Agreement is the question Trazum is qualified to ask, and it is not the question a team needs answered before shipping. Theirs is whether the classifier still hits 94%, whether the JSON still parses, whether the refusal rate moved — and those are assertions about their task, which this tool has no business inventing.

So it builds the part it can get right: a suite in which the only variable is the prompt, with both versions, every case bound to the correct template variable, and the same provider on both sides. defaultTest.assert is left for the team.

It makes no API call and needs no key — the whole point is to hand the run over — and it warns about the things that would quietly make a run meaningless: a ${x} placeholder promptfoo will not substitute, a prompt with three placeholders and one value per case, a provider whose id had to be guessed.

The only assertion seeded is is-json, and only when the prompt shows a fenced JSON block. That is not an opinion about the task; the prompt already demands it.

JSON rather than YAML, which promptfoo reads just as happily. This package has no dependencies and is not acquiring a YAML emitter, and a hand-rolled one is a quoting bug waiting for the first prompt with a colon, a tab, or a line ending in a space.

Fixed#

The JSON detection used findRestatedFormat, which was the wrong question wearing a convenient shape: that function answers "is this prompt wasting tokens restating its own schema?", so a prompt demanding JSON cleanly got no assertion while a wasteful one did — exactly backwards. It now looks for a fenced block tagged json, or an untagged one that parses as JSON, and the report says how many assertions were seeded rather than claiming "no assertions" unconditionally.

Added#

trazum rank <dir> — which of these prompts to fix first.

The obvious shape for this was a complexity score out of a hundred, and it is the wrong shape. A number nobody can reproduce by hand cannot be argued with, and the weights that turn four measurements into one get tuned until the ranking looks right — which is fitting the metric to the answer.

So the ordering is the one quantity that is not a matter of opinion: what optimising each prompt would actually recover, obtained by running the deterministic rules rather than evaluating a formula. The structural measurements are printed beside it as the explanation — tokens per sentence (verbosity independent of length), few-shot examples and what they cost, a restated output format, and the share of the prompt that is protected content.

That last one earns its place: a prompt that is 83% code has far less headroom than its size suggests, and a ranking that hid it would send somebody to spend an afternoon on a file that cannot move.

Source files contribute their marked prompt, never the code around them. One with no marker is skipped and counted, so a repository whose prompts mostly live in code does not show a short list and look complete.

Two tests hold the line against a score reappearing: PromptProfile may not grow a field whose name contains "score", "rating", "grade", "index" or "complexity", and neither may the ranking's JSON.

Fixed#

Two problems in that work, both visible only by running it:

  • A single unmarked source file aborted the whole ranking. sourceFileOf throws for a source file with no // trazum:prompt marker, which is right for optimize — you named that file, and optimising it would rewrite your code — and wrong for a command walking a directory. One stray .ts killed the other thirty-nine.
  • Four prompts showed $0.25 and looked like four equivalent jobs. Three of them recovered a single token, which at 50,000 calls is twenty-five cents. The arithmetic was right and the presentation was not. Rather than invent a threshold nobody could check, the token count is printed beside the money.

Added#

optimize --suggest — rewrites proposed one phrase at a time.

--llm hands the model the whole prompt and takes the whole answer back, which is all-or-nothing in both directions: a result that fails one safety check leaves the author with nothing, and one that passes is a wholesale rewrite they must read end to end before trusting. --suggest asks which exact phrases say something in more words than they need, and returns a list small enough to judge on sight — You should always make sure to → Always.

The model proposes; the prompt decides. Every suggestion is checked against the text before it is shown and dropped rather than reconciled: before must appear byte for byte (a model asked to quote will tidy the punctuation as it goes), it must not touch code, URLs, placeholders or tags, after must not introduce any, it must actually save tokens, and overlapping suggestions are refused because applying both produces text neither described. A phrase occurring several times is rewritten everywhere it appears outside protected content rather than the whole suggestion being refused for one occurrence in a code block.

Nothing is applied unless asked. --apply-suggestions takes them and the headline figures move with the change; on its own it is an error rather than a no-op, for the same reason a misspelled flag is.

The model is asked about the optimised prompt, not the one as written — re-finding what the rules already took spends a call to be told what Trazum knew for free.

Fixed#

The first version of the CLI test for this deadlocked and had to be killed by the timeout. It drove the binary with spawnSync, which blocks the test process's event loop, so the fake LLM server living in that process could never answer the child it was waiting on. A fake server in the test process only works if the test process is free to run.

Added#

trazum blame <file> — when this prompt got expensive, and what change did it.

Git already knows who edited a prompt and when. What it does not know is that three lines added to a system prompt at 50,000 calls a month is a bill rather than a diff. blame walks the file's history, counts the tokens at each commit, and puts both facts on one line — with the net movement priced through the same usage profile optimize uses, and the single worst commit named.

--prompt tracks one marked prompt inside a source file, so refactoring the imports is not read as the prompt growing. Renames are followed. --limit, --json, and the pricing flags behave as everywhere else.

This is the first thing in the repository that runs another program, so it happens in one module written as though it were the whole attack surface: no shell, every path after a -- separator, object names validated as 40 hex digits before being glued to anything, bounded timeout and buffer, and no credential prompting. Six invariants in security.test.js assert all of that, each checked against a mutant — including that git.ts stays the only file importing node:child_process.

Every command now honours -- as the end of options. Without it there was no way to name a file called -x.txt or --output=… on the command line at all; the parser saw a flag and refused before the path reached anything.

Fixed#

Two bugs found while building the above, both by running it:

  • A renamed prompt reported no history before the rename. nameAt asked git log --follow --max-count=1 <sha> -- <today's name>, which returns nothing for commits where that name did not exist — so every revision before a move showed "not present" while the data sat there under the old name. One git log for the whole history now pairs each commit with the path it touched.
  • --limit was ignored. It was accepted by the command's flag list but never added to VALUE_FLAGS, so --limit 6 parsed as a boolean and the count was the next positional argument. It silently walked the default 20 every time.

Security#

--reorder had no safety at all outside English and Spanish. Not a missing feature — a silent failure.

BACKWARD_REFERENCES was one flat list of English and Spanish phrases, applied to every prompt. The module's own documentation says its entire design is about what it refuses; for a French, German, Portuguese, Italian, Dutch, Japanese or Chinese author it refused nothing. "Résumez le texte ci-dessus" was hoisted above the text it points at and reported as a saving. Every test in the suite passed, because every test asked the question in the two languages that worked.

Seven more languages now, grouped per language so the coverage is a thing you can look at rather than infer. Japanese and Chinese match without word boundaries — the boundary test asks whether the neighbouring character is a letter, and in 上記のテキスト it always is, so a boundary-matched CJK list would have read like cover and provided none.

And a fourth refusal, for the scripts still missing. Cyrillic, Arabic, Hebrew, Hangul, Devanagari, Thai and Greek: nothing moves, and the report names the script and says why. A single such instruction inside an otherwise English prompt is enough to stop it — that is the case where a missed reference does damage, and the cost of being wrong is a saving the author can still take by hand. Adding a language is adding an array.

Three tests keep this honest rather than trusting it: the README's list of languages must match the table, no phrase may be capitalised (it could never match), and no covered language's phrases may use an uncovered script. All were checked against mutants.

Fixed#

Two pluralisation slips in the reorder report, both visible in the output above: "1 tokens back, every call" and "Left 1 block where they were".

Changed#

The web app is rebuilt on shadcn/ui, wearing Trazum's own palette.

The components are shadcn's — Radix underneath, so the model and level pickers are properly keyboard-navigable and the toggles announce themselves, which the bare <select> and <input type="checkbox"> never did. The colours are the ones that were already here: --primary derives from --terracotta, --background from --paper, and so on down. Taking shadcn's neutrals would have made this look like every other application assembled from the same registry, and the whole point of theming through CSS variables is that you do not have to.

Two changes are not cosmetic. The result and the diff are tabs rather than a toggle button whose label named the state you were not in — that button was read backwards about half the time. And the endpoint field is the picker the previous release made it, now rendered as one.

Animation comes from react-bits' CountUp, AnimatedContent and ShinyText, rebuilt on requestAnimationFrame and CSS keyframes: upstream builds them on motion, and one animated integer is not worth a 50 KB dependency in a project whose published packages have none. prefers-reduced-motion switches all of it off in one rule rather than component by component.

Fixed#

Two bugs in that rework that compiled, typechecked, and were wrong. Both were found by opening the page, not by reading the diff.

The results summary rendered blank. AnimatedContent waited for an IntersectionObserver, and a reader who scrolls down to reach the Optimise button gets their result mounted above the viewport — so the observer reported "not intersecting" and a 214px card sat there at zero opacity. Content that appears in response to an action is not scroll-triggered content; it animates on mount now, and waiting for the viewport is opt-in.

The Copy and Clear buttons each fell onto their own row. CardHeader is a grid, so the flex-row justify-between on it merged in and did nothing — the two utilities are in different groups, so nothing overrode anything and the class list read as though it should have worked. They use shadcn's CardAction slot, which is what the grid has a rule for.

apps/web has tests for the first time, and they encode exactly these two: a component whose default hides content, and a layout override that is silently ignored. Both were checked against a mutant that reintroduces the bug.

Fixed#

The alert gate raced the analysis it reads, and failed the merge that fixed both alerts. open-alerts and codeql started together: the gate finished one second in, CodeQL uploaded a minute later, so the gate judged the merge against the state of the commit before it and reported two findings at line numbers that no longer existed. A red build for a fix that worked is how people learn to re-run until green.

It now runs needs: codeql, and — because the alert index settles after the upload returns — checks that every row it is about to report carries the SHA being built, retrying for up to 90 seconds and failing rather than passing if it cannot read current state. A gate that cannot see the present has no business reporting green.

The test for it caught nothing at first: /needs:\s*codeql/ matched the comment explaining the dependency. Fourth time in this repository. It strips YAML comments now and asserts against a mutant with the line deleted, so "can this assertion fail?" is answered rather than assumed.

Security#

CodeQL reopened the SSRF alert on the fix below, and it was right a third time. Both earlier attempts hardened the wrong layer. The taint does not start at TRAZUM_LLM_BASE_URL, which an operator sets on their own machine — it starts at POST /api/optimize, whose body carried a baseUrl that the deployed server would then fetch. That is server-side request forgery by construction, and no amount of validating the string fixes it: the host filter reads a name, and a name an attacker registered resolves wherever they choose.

So the request body no longer names an endpoint. It selects one. TRAZUM_ALLOWED_LLM_ENDPOINTS is a comma-separated list the operator writes, and the value that reaches fetch is the entry from that list — the string that arrived over HTTP is compared and then dropped. The list is empty by default, so a deployment that has not thought about this cannot be pointed anywhere at all. Nobody loses the capability: TRAZUM_LLM_BASE_URL and the CLI still go anywhere you like. What changed is who is allowed to choose. The web UI's free-text field is now a picker over what the server actually accepts, because offering a text box that always answers 400 is worse than offering nothing.

A redirect walked past the entire host filter, and had all along. Every check in net.ts reads the URL the caller named, and fetch follows redirects by default — so an endpoint that passed validation could answer 302 Location: http://169.254.169.254/latest/meta-data/ and the request went there anyway, authorization header included. One HTTP response, and the whole filter was decorative. Every server-side call now carries redirect: 'error', plus credentials: 'omit' and referrerPolicy: 'no-referrer'. This one mattered for the CLI as much as for the deployed app, which is why it is fixed in the provider rather than in the route.

And a third door nobody had looked at. countTokensAnthropic takes a baseUrl and sends an x-api-key to it, with no validation whatsoever. Both providers had been hardened twice over while this sat open, because it is called a counter rather than a provider. It goes through the same gate now.

Security#

The alert gate found two things on its first real run, which is the argument for it in one sentence. Both were invisible to every pull-request check.

The SSRF fix did not close the alert, and CodeQL was right. Validating at construction checked baseUrl and then fetched `${baseUrl.replace(/\/$/, '')}/chat/completions` — two different expressions, so the thing checked was never the thing used and nothing on the path from option to fetch was a barrier. A later edit could have moved the check without anything noticing.

The validator now returns the value to use rather than approving one, and the fetch uses what it returned. Re-parsing normalises it too: https://host/v1/../../admin passed as a string and resolved to /admin on the wire; it is resolved before the request now.

And a time-of-check to time-of-use race I introduced an hour earlier. The symlink guard was lstat then readFile, which resolves the name twice — CodeQL opened it as high severity. It is now a single open with O_NOFOLLOW: one syscall, and the handle is the file that was checked.

This repository had already been caught by the identical stat-then-read pattern in the config reader and fixed it the same way. I wrote it again anyway, which is the more useful half of the finding: the guard against a class does not live in anybody's memory.

A third thing turned up while fixing those two: one file had no diff. scripts/measure-token-band.mjs used a raw NUL byte as a hash field separator, which is enough for git to call the file binary. Its three commits — including the one that fixed the SSRF finding above — rendered as Bin 7652 -> 7654 bytes, and nothing anywhere warned that a security fix had gone through unreadable. The byte is now written \0, which produces the same digest, and a test refuses a raw NUL in any source file: the other invariants here all assume somebody can read the code.

Security#

A job now fails the build when main carries an open critical or high alert.

CodeQL's pull-request check reports new alerts in the code that pull request changed. A finding already open on main is not new, so every later pull request goes green beside it — which is exactly what happened: eleven consecutive green runs with a critical SSRF alert open the whole time, found only because somebody opened the Code scanning page and looked.

Green on a pull request is not green on the repository, and nobody should have to remember the difference.

Four decisions in the job worth stating:

  • It reads security_severity_level, not severity. The first is the CVSS-style rating the UI shows as "Critical"; the second is the query's own, where today's critical was merely error. Reading only the second would have let it through.
  • security-events: read. A job that could dismiss an alert is a job that could dismiss the alert it was written to surface.
  • Skipped on pull requests. There it would report the base branch's state and fail somebody's unrelated work for a finding they cannot fix from their branch.
  • It can actually fail. The jq filter was checked against a realistic payload — one critical, one medium, one with no severity field at all — before it was committed. A gate that cannot fail is worse than no gate, because it looks like coverage.

SECURITY.md gains the row, and two tests assert the job still asks the right question and stays read-only.

optimize pointed at a source file used to rewrite your code. Handed src/prompts.ts it optimised the whole file — imports, const client = new OpenAI();, all of it — counted 83 tokens of code the model would never see, and priced it against Claude Opus 5 on a file that plainly imports openai.

Then the capitalisation rule turned import OpenAI into Import OpenAI, which does not compile, and -o wrote it back over the file. That is the worst behaviour this repository has shipped, and it was the default.

It now reads the marked prompt and leaves the file alone — 43 tokens instead of 83, and the output is the prompt rather than mangled TypeScript.

It refuses rather than guessing, in three places:

  • An unmarked source file. Optimising TypeScript as prose does not produce a worse prompt, it produces broken code. The refusal names the marker syntax, so it costs the reader one comment.
  • A file holding several marked prompts — it lists them and asks for --prompt. Optimising "the first one" silently is how the wrong prompt gets rewritten.
  • A marker it cannot read, naming the line and the reason.

Detection now feeds the price, closing the gap trazum where opened: an import names who and never which, so the provider's stand-in model is used and the report says GPT-5 rather than Claude. The layering is unchanged and now has one more rung — a flag beats config, config beats detection, detection beats the built-in default. Reading the code is better than assuming, and worse than being told.

--prompt <name> is new. A plain .txt prompt and stdin go through exactly as before, which is asserted rather than assumed.

Security#

The SSRF filter was one level too far out. openAiCompatible fetches whatever baseUrl it is handed, with no validation of its own. The web route validated a body-supplied URL before calling it, and that was the only thing standing between a public deployment and the internal network — which SECURITY.md claims as a property of the filter, not of one caller remembering to use it.

openAiCompatible and anthropicProvider now validate their own endpoint at construction, so a provider that can never safely work does not exist to be handed around. openAiCompatible is an exported library function: "the caller checks" is a promise about every future caller, including ones outside this repository.

The distinction that makes this safe without breaking anything is who chose the URL:

  • From TRAZUM_LLM_BASE_URL — the operator configuring their own machine. providerFromEnv passes allowInsecure: true, because http://localhost:11434 for Ollama is the documented normal case and refusing it would break every local setup.
  • From an HTTP request body — a stranger naming a host for this server to fetch. Validated, and now twice: the route still turns the reason code into a sentence in the reader's language.

Verified by hand across the shapes that matter: the cloud metadata address, plain localhost, an RFC1918 address and credentials in the URL are all refused; a public https endpoint and an operator-configured Ollama both work.

measure-token-band.mjs followed symlinks. The script posts corpus file contents to the counting endpoint — that is its job, and CodeQL flagging the file-to-network flow describes the job rather than a bug. What it made worth checking was the edge: readdir plus readFile follows a symlink, and the corpus is a test-fixture folder people drop things into. One named few-shot.txt -> ~/.aws/credentials would have been posted to an API without a word. walkPrompts has skipped symlinks since it was written; this script had simply never been held to the same rule.

It now refuses a symlink by name and prints every file it is about to send before sending any of them. The repository tells people not to paste a private prompt into a public issue; posting one to an API deserves the same warning, and consent has to be informed to be consent.

Both alerts had been open on main for hours without appearing on any pull request. The CodeQL check on a PR reports new alerts in the changed code, so a finding on main stays green on every subsequent PR — worth knowing about the tooling, not just about these two.

The report reads like a report. Twelve releases added sections to it and nobody had looked at the whole thing at once. Printed end to end, three problems were obvious:

  • The amounts were unreadable as a set. Four advisories worth $506, $422, $170 and nothing, each with its figure trailing the end of a different-length title. They exist to be compared, and comparing them meant hunting for four numbers in four sentences. The amount now has a column of its own, right aligned, with the detail indented to the title so the prose forms one block instead of stepping around the numbers.
  • Nothing said what to do first. The rules saved $1.25 and the top advisory was worth $506 — a 405× difference the reader had to notice for themselves. A closing line names it: Start here: "This task may not need Claude Opus 5" — $505.80/month, 405× what the rules saved.
  • --reorder printed a heading, a blank line and a shrug when there was nothing to move and nothing refused. The token count above had already said nothing changed. That block now appears only when there is a move or a refusal to report.

Two things caught by looking at the output rather than the diff: the closing line lowercased the advisory title, turning "Claude Opus 5" into "claude opus 5" — a product name mangled to fit a sentence — and it was the one line in the report not wrapped to the common width, so it ran off a narrow terminal.

A test that matched Nothing could safely move. was rewritten to assert the behaviour instead: that the prompt came back unrearranged. A test pinned to a line that says nothing is what keeps that line alive.

On a subscription, Trazum stops printing money. Inside Claude Code, Codex or Cursor you pay the same whatever your prompt costs, so a monthly figure is arithmetic about tokens dressed as cash — and "$184/month" told to somebody on a flat plan is not a rounding error, it is money that does not exist.

The report now says what the saving actually buys there: tokens back per call, and the share of the context window they free. The window is the real currency inside an agent — every token the system prompt holds is one the conversation cannot.

Advisories whose only pitch is money are dropped too, which the first version got wrong. Suppressing the price beside each title left model-downgrade's detail reading "you would go from $843.00 to $337.20 per month" in prose underneath. model-downgrade, batch-api, output-dominated and promo-pricing now go entirely; caching, context overflow, contradictions and redundant examples stay, because latency, headroom and correctness are still real on a plan.

The test for this is blunt on purpose — no dollar sign anywhere in the output. A softer assertion would have passed the version with money in the prose.

Two escape hatches, and the reasoning behind them matters more than the flags: the host says where Trazum runs, not where the prompt goes. Somebody editing a production prompt inside Cursor wants the dollars, so --cost brings them back without leaving the editor, and --tokens-only forces the other direction anywhere. --cost wins when both are given.

Also fixed before it shipped: forced with --tokens-only on GitHub Actions, the report announced that "GitHub Actions bills by subscription", which is simply false. It now distinguishes being told from having detected. And unknown billing is treated as unknown rather than as a subscription — guessing wrong there would hide the product's main output from most of the people using it.

trazum where — which provider a prompt is actually sent to. Pricing seven providers turned the Claude default into a wrong number: a file calling OpenAI was billed against Claude Opus 5 without comment. This reads what the code already says instead of assuming.

Four kinds of evidence, strongest first: model= on a trazum:prompt marker, a quoted model id, a base URL, an SDK import. Every answer names the line it came from — a detection that cannot be checked is a guess, and the dollar figure that follows from it would be a guess too.

Three things it gets right that the first version got wrong, each caught by running it rather than reading it:

  • A base URL beats the SDK it was pointed at. Moonshot, DeepSeek, xAI and Groq are all called through the OpenAI SDK with a different base_url — their documented usage. Treating that as a contradiction refused to price a perfectly ordinary client; pricing it as OpenAI would have been wrong for a large slice of everyone using this.
  • It priced an OpenAI file as Claude. Detection found the provider, found no model, and fell through to the global default — printing "goes to openai" and "priced as Claude Opus 5" three lines apart. It now uses that provider's own model and says that is what happened.
  • Nearest capability, not an exact match. Neither OpenAI nor DeepSeek has a large model, so matching the default's capability exactly found nothing and the fallback fired anyway. A ladder with different rungs is the normal case.

It refuses when a file names two providers, names both, and assumes nothing. Detection sits between config and defaults: a flag beats config, config beats detection, detection beats the built-in default.

With no file it reports the host — Claude Code, Codex, Cursor, GitHub Actions, CI or a plain terminal — and warns when that host bills by subscription, because a monthly saving is arithmetic about tokens there rather than money anybody gets back. That is the first half of the Cursor/Claude Code question the roadmap has been holding: not a full report in a different unit, but no longer quoting a dollar figure to someone on a flat plan without saying what it is.

A third quadratic line-number lookup, found by the hostile-input tests before it shipped — 36 seconds on a file repeating a model id. Same shape as extract.ts, written again after fixing it there; a binary-searched line index this time, because the matches arrive out of order and a forward-only counter cannot work.

The report no longer claims a Claude number for a model that is not Claude. Pricing seven providers left the token estimator where it was — tuned against Claude's tokenizer — while ±15% kept printing beside GPT and Kimi figures. That band was never measured for those families, and stating it was a precision claim nobody had earned:

1,021 → 1,020   -0.1% (estimated — the counter is calibrated on Claude, not GPT-5)

Anthropic models still show ±15%, where the band is at least the claim it was written for. --exact-tokens remains the answer for figures you can budget from.

Documents the multi-provider work that shipped alongside it: a table in the README of what actually differs between providers — cache read rates, cache minimums, whether caching starts automatically, and which providers have no batch API or no caching at all — plus the two things deliberately left out and why.

The Tokenizer per model family entry under Under consideration is heavier than it was, and says so. And the error-band entry now explains once that it will keep being renumbered by anything that ships before it, rather than re-justifying the move each time: it is the only item on the roadmap whose completion is not ours to schedule, and holding shippable work behind it would be the wrong trade every time.

Trazum prices models from seven providers, not one. OpenAI, Google, Moonshot, DeepSeek, xAI and Mistral join Anthropic in the bundled catalogue.

The data was the easy half. The hard half was that the cost multipliers were global constants — one cache-read rate, one cache-write rate, one batch discount for everything — and global made them quietly wrong the moment a second provider existed. They now live on the model, defaulting to Anthropic's values so nothing that worked before changes.

Three of them were not inaccuracies but savings that do not exist, and all three were found by running the catalogue rather than by reading it:

  • Kimi, DeepSeek and Grok have no batch API. The global constant offered them a 50% discount that cannot be bought — $139 a month in the test that caught it. batch: null now means "there is no batch API", which is deliberately different from not having said.
  • Mistral has no prompt caching. A zero cache minimum satisfied 0 >= 0, so the caching advisory fired and offered $100 a month of a feature that does not exist. Introduced by this very change and caught before it left the branch.
  • The batch saving was computed as cost × discount, which equals the saving only when the discount is exactly 0.5. Latent on Anthropic, wrong on the first provider with any other rate.

Gemini and Grok read cache at 25% of input against Anthropic's 10%, so the same prompt cannot be worth the same fraction on both. The advisory now quotes each provider's real rates — and stops naming cache_control to people who do not have it: OpenAI, Moonshot and DeepSeek cache automatically above a threshold, and the report says so instead of naming a parameter that does not exist for them. The advice to move stable content forward is identical either way, because a prefix is a prefix.

A cheaper model means a cheaper model, not a different supplier. Recommendations are now scoped to the current provider. Dropping from Opus to Sonnet is a one-line change; moving to another vendor is a different API, different behaviour and a migration — and this advisory is already caveated as a keyword heuristic rather than a judgement about answer quality. Unscoped, it was telling Claude users to switch to gpt-5-nano.

Deprecated#

  • ModelPricing.tier — replaced by capability, on a vendor-neutral scale of small | mid | large | frontier. Anthropic's ladder used as the generic axis reads as nonsense the moment the model is not Anthropic's: telling somebody on Kimi that their task "looks like haiku complexity" is a label meaning something other than what it says.

    Migration: replace model.tier with model.capability, mapping haiku → small, sonnet → mid, opus → large, frontier → frontier. cheapestOfTier and cheapestOfTierIn still take the old names.

    Per VERSIONING.md, tier keeps working unchanged for the whole of 1.x and is removed in 2.0. Both fields are populated on every model and a test asserts they never disagree — if they drift, half the code ranks one way and half the other, and the difference is invisible until a recommendation is wrong.

Added#

  • multipliersFor(model) — the cache and batch rates that apply to one model, defaults filled in. Anything computing a cost should go through it rather than reading COST_MULTIPLIERS, which remains exported and remains correct for Anthropic.
  • ModelPricing.provider, .caching, .multipliers and .recommendable. The last replaces a hardcoded model id inside cheapestInTier — a model behind a private programme is real and worth pricing, but recommending a switch to something the reader cannot buy wastes their time.
  • An optional provider argument on cheapestOfTierIn, so a caller who genuinely wants "cheapest anywhere" can still ask.

On the prices themselves: they are written from what was known when this landed, not read off each provider's page today, and PRICING_LAST_REVIEWED says when. The tests check them for coherence — output dearer than input, a plausible window, a cache minimum only where there is a cache — never for accuracy, which no test here can do. The local pricing overlay corrects any of them without upgrading the library.

Reordering for the cache is on the web. It is the largest saving Trazum can make — a $0 caching saving against a $184 one on a 1,178-token support prompt — and it was reaching only people who had cloned the repository and built the CLI. The web is the front door; the biggest thing the product does should not require a terminal to find.

Opt-in over HTTP for the same reason it is opt-in on the command line, and the reason is stated in the route rather than left in the diff: every other transformation the endpoint performs deletes text whose absence is local, and this one moves text, where order carries meaning. Nothing about it is less safe here — the same deterministic core, nothing sent anywhere, the prompt returned byte-identical when it cannot act — but "the browser did it quietly" is not something this endpoint should be able to do.

  • Honoured only on a literal true. A string, a number and null are all ignored: the body is untrusted, and a truthy check would let "false" rearrange somebody's prompt.
  • original stays what the caller sent, so the diff the browser draws shows the move instead of hiding it behind the deletions.
  • Refusals come back in the response and render whether or not anything moved. The panel is deliberately not styled like the green savings box — that is a saving to enjoy, this is a change to review — and it sits above the money so the number is read after the caveat rather than instead of it.

Verified against the running server rather than the module underneath it: the endpoint moves and reports, omits reorder entirely when it was not asked for, returns the prompt byte-identical with both refusals named when it declines, and ignores a non-true value three ways. The checkbox and its warning render in both locales.

ROADMAP.md records what is deliberately not coming to the web — the pricing overlay, config-aware defaults and budgets — with the reasoning, rather than leaving three unexplained gaps. A textarea for pasting prices into somebody else's server is not the overlay feature wearing a different hat; it is a worse one, with no review and no provenance.

The queue was renumbered so Released runs unbroken: embedded prompts became 1.3.0 and the web 1.4.0, and the error band moved to last. Its corpus, harness and test all shipped; the measurement needs the official counting endpoint and a key, so it is the only entry whose completion is not ours to schedule. Holding two releases that could ship behind one that cannot would have been the wrong trade, and the file says so rather than renumbering quietly.

check now reads prompts embedded in source files. It read .txt, .md, .prompt and .tmpl; real prompts live in TypeScript template literals and Python strings, so adopting Trazum meant refactoring them into standalone files first — the largest barrier to adoption the tool had.

// trazum:prompt support-system
export const SUPPORT = `You are a support agent.

Customer message: ${message}`;

It reads a marker rather than guessing. Inferring which string in a file is a prompt is a heuristic, and a heuristic inside a CI gate fails builds over log lines and SQL queries. //, #, -- and <!-- --> cover the languages prompts live in.

${x} needed no handling at all: it is exactly the shape segment.ts already protects, so an embedded prompt gets the same cache-prefix analysis, rule protection and --reorder treatment as a {{x}} template, with no second code path to drift.

  • Each prompt is budgeted on its own, not summed into the file. Four prompts in a file are four things to govern, and the imports around them are not tokens the model will see.
  • The id is path-prefixed — src/prompts.ts#support-system, or src/prompts.ts:12 for a bare marker — so existing budgets globs cover embedded prompts without new syntax.
  • Source files are scanned without being opted into. Requiring config to discover a marker somebody just wrote is how eval came to be fully implemented and completely undiscoverable. An unmarked source file is dropped silently.

A marker Trazum cannot read fails the build. A prompt assembled by concatenation has no text until it runs; Trazum declines it and names the line rather than governing the fragment it can see. The author marked that prompt to have it governed, and a green build saying otherwise is the same lie as "0 failures" from a run that measured nothing.

Scanned character by character rather than with a regex, and the hostile-input tests earned their place immediately: the obvious line-number lookup was quadratic in the number of markers — 15.5 seconds on a file holding 20,000. Caught before it shipped rather than after.

One more found in review, by CodeQL: <!-- trazum:prompt greeting--!> produced the name greeting--!>, because --!> is the comment end bang the HTML parser also accepts and only --> was being stripped. Fixed for both terminators, and the class closed behind it — a name is now constrained to an identifier charset rather than "whatever is left on the line", falling back to the file:line id when it is not one. The name is printed in reports and matched against budget patterns; it should never have been arbitrary text.

diff for embedded prompts is still to come; check is the gate and came first.

The ±15% error band now has a corpus, a harness and a test. It is printed on every report, appears in both READMEs, in the estimator's own doc comment and in VERSIONING.md as part of the frozen API — and estimateTokens was tested for exactly three things: zero on empty input, monotonic growth, and never returning NaN. Nothing measured its accuracy, and every dollar figure Trazum prints descends from it.

Eight corpus samples, chosen to exercise the branches the estimator actually has: prose in English and Spanish, Japanese and Chinese, code with fenced blocks, few-shot Input/Output pairs, a punctuation-heavy Markdown table, and dense numerics. scripts/measure-token-band.mjs (npm run measure:tokens) counts them against the official endpoint and subtracts the message envelope, so the figures describe the text rather than the request around it.

token-band.test.js asserts the band per sample. Three deliberate choices in it:

  • It does not pass quietly while unmeasured — it skips out loud and names the command. "0 failures" from a check that measured nothing is the most misleading thing this suite could report, which is the same reasoning that makes check treat an unbudgeted run as an error rather than a pass.
  • It requires the docs to admit the band is unverified until ground truth exists, so ±15% cannot harden from a design target into a fact nobody established. tokenizer.ts and the README now say so.
  • It carries a digest of the corpus, because a fixture describing text that has since been edited passes while describing something else.

Both paths were exercised before committing: a synthetic fixture confirmed the band assertions fire per type with an actionable message, and that editing the corpus afterwards fails on the digest. The synthetic fixture was then deleted — fabricated numbers are exactly what this test exists to prevent, and committing them to make a suite look green would be the same failure wearing a lab coat.

Needs the maintainer once: ANTHROPIC_API_KEY=... npm run measure:tokens. The endpoint is free and does not run the model. Commit what it writes and the assertions go live; if the bands differ materially by type, the reports stop printing one number for all text.

A tag now publishes the release. Publishing is the one action in this repository that cannot be undone — npm allows unpublishing for 72 hours and then the version number is spent for good — and until now it was also entirely manual. A tag matching v*.*.* runs full verify, reports exactly what each tarball would contain, and then publishes both packages.

Trusted publishing (OIDC), not a stored NPM_TOKEN. That is the decision in this change rather than an implementation detail. A long-lived publish token would be the highest-value credential the project holds, sitting in secrets permanently for something used a few times a year — and unlike every other secret here, a leak is not recoverable by rotation alone, because whatever was published under it stays published. OIDC needs id-token: write and stores nothing. Provenance comes free with it, so a consumer can verify a tarball was built from this repository at this commit.

Three refusals, each a mistake with no correction afterwards, and each with a test:

  • The tag and the manifests must agree. publish.test.js already asserts every manifest carries the same version; the workflow checks that the shared version is the tagged one.
  • verify runs before anything is published, and it is the same verify a pull request runs. A release gate that checks less than the pull-request gate lets through exactly what the tag was for.
  • workflow_dispatch is dry-run only — every publish step is gated on a tag.

@trazum/core publishes first, because the CLI depends on it at an exact version and the other order leaves a window where installing the CLI fails.

Still needs the maintainer once: the @trazum scope does not exist on npm, so it has to be created and this repository configured as a trusted publisher. docs/releasing.md is new and has the exact fields, the cutting-a-release checklist, and what to do when a release goes wrong — including the field that is easy to get wrong: npm's Environment must read release, or the OIDC claims do not match and the publish is rejected with an error about the token rather than about the mismatch.

Until it is done a tag push runs every check and fails at the publish step — which is the right failure, having published nothing.

Issue templates, so a tester has somewhere to land. .github/ had CODEOWNERS, Dependabot and workflows but no way for anyone to report anything. Two forms: a bug report, and "a rule changed what my prompt asks for" — the second one separate and labelled correctness, because Trazum's entire claim is one sentence and a rule that saves tokens while quietly changing the meaning is the failure the product exists to prevent, not a smaller version of a good outcome.

Both forms ask for a reproduction and both warn, before anything else, not to paste a prompt containing anything private: Trazum runs entirely on your machine and never sends a prompt anywhere, but an issue is a public web page. Security reports are routed to a private advisory rather than an issue, and blank issues stay enabled — the reports nobody anticipated are usually the interesting ones.

Property tests for reorderForCache, over 400 generated prompts. The hand-written fixtures each ask one question about one prompt, and a fixture list only asks the questions it encodes — which is how two quadratic patterns, a CRLF bug and a leading-blank-line bug all shipped in the same module and were caught one at a time afterwards.

Eight properties, checked across every generated case: content is conserved (no word deleted or invented), a refusal returns the prompt byte-identical, a move always grows the prefix and the figure reported matches what the advisories' analyser computes, moved blocks keep their relative order, backward references and placeholder blocks never move, the transformation is idempotent, the author's line ending survives, and the prompt neither opens with a blank line nor ends differently from how it was given.

Generation is seeded rather than random, so a failure names a case you can reproduce by reading the seed out of the message.

All eight pass. Three failed on first run and all three were the assertions being wrong rather than the code: indexOf from zero finds the first copy of a repeated block, so a duplicate read as an out-of-order move; the generator emitted mixed line endings and then asked whether endings were preserved; and a moved block legitimately brings its own indentation to the front when the placeholder sits on the first line. Each is now written to ask what it meant to ask.

A piped --reorder said nothing about what it had done. Redirecting the output takes the "prompt and nothing else" path, which is right for every other transformation — they delete, and the diff shows it. This one moves text, and piping it made both the move and the refusals invisible. That is precisely what the module promises not to do: "a saving Trazum chose not to take is one the author cannot evaluate."

One line now goes to stderr, so stdout still carries the prompt and nothing else:

trazum: moved 1 block (~1,001 tokens) into the cacheable prefix. Run without redirecting output for the reasons.
trazum: nothing could safely move; 2 blocks left in place. Run without redirecting output for the reasons.

The version comment added yesterday named a version that does not exist. The README's SHA pin read # 1.1.0 against manifests reading 1.0.0 — no such tag, no such package. The test written to stop exactly this class of drift only required #\s*v?\d, so it passed. It now compares the comment against package.json, and fails on the string it was shipped with. A test that admits only the shape of an answer will accept a wrong one.

The README recommended a tag that did not exist. Davmunrey/Trazum@v1.0.0 was the copy-pasteable Actions example for a whole release, and no such tag was ever pushed — so anyone following the quickstart got a workflow that could not resolve its own action.

It was also the wrong shape. SECURITY.md says every third-party action is pinned to a commit SHA because a tag is a movable pointer, and security.test.js enforces that on every workflow in this repository. The README was telling readers to do the thing the project refuses to do itself.

Both examples now pin to a commit SHA with the # 1.1.0 comment Dependabot reads, and a new test extends the SHA-pin rule from what this repository runs to what it tells other people to run — the docs had drifted for a release with nothing checking. The test fails on the old README, which is the only evidence that it checks anything.

trazum optimize --reorder moves the stable instructions in front of the placeholder. Since 0.2.0 the cache-prefix-reorder advisory has pointed at the largest saving Trazum knows about and no command could take it. Prompt caching is a byte-for-byte prefix match, so everything after the first {{placeholder}} is re-read at full price on every call. Measured on a 1,178-token support prompt: 14 tokens cacheable as written, 1,174 after rearranging the same content — the difference between a $0 caching saving and a $184 one at 50,000 calls a month on Opus.

Opt-in, and deliberately not part of aggressive. Every other transformation deletes text whose absence is local. This one moves text, and order carries meaning: "Summarise the text above" is correct where it sits and nonsense in front of the text it points at. aggressive promises "read the diff"; this asks whether the order mattered, which is a different question.

What it refuses, which is most of the module:

  • A block containing a backward reference (above, the following, anterior, …) stays put — and so does everything after it, because moving a later block past a pinned one changes their order relative to each other. The phrase list is generous in both locales on purpose: a false positive costs a saving that was available, a false negative silently changes what the prompt asks for.
  • Only whole blank-line-separated blocks move, so a sentence is never severed from the paragraph that qualifies it and the placeholder's own line travels with it.
  • Nothing moves without a placeholder, or below the model's cacheable minimum.

A refusal returns the prompt byte-identical and names the phrase responsible: "no saving here" and "there was a saving and it was not safe to take" are different answers, and only the second one is actionable. --diff compares against what you wrote rather than against the rearrangement, so the move is not hidden behind the deletions; --json carries the whole decision under reorder, refusals included. check rejects the flag — it is a gate, and a gate does not rewrite. reorderForCache is exported from @trazum/core for callers who want the decision without the CLI.

Fixed#

Two defects in the rejoined seams, both found by writing the fixtures that had been missing rather than by a report:

  • A placeholder on the first line left the rearranged prompt opening with a blank line. With no head for the moved blocks to sit after, the usual leading gap put whitespace at byte zero — which changes the cache prefix for nothing.

  • A CRLF prompt came back with mixed line endings, because the seams were rejoined with bare newlines regardless of what the author used. In a byte-for-byte prefix match a changed byte is a changed price, and in a repository it is a diff on every line nobody asked for. The prompt's own line ending is now preserved, as is whatever it ended with — collapsing runs of blank lines remains the whitespace rule's job, not this one's.

  • Two quadratic patterns in the new module, found by pointing the existing ReDoS suite at it rather than by reading it. split(/(?<=\n)(?=\s*\n)/) re-consumed a run of blank lines at every position inside it — 13.9s on 120 KB of newlines — and /\s*$/ on a prompt holding a long whitespace run that does not end in one took 31 seconds at 200 KB, inside the 400 KB the HTTP API accepts. Both are now a linear scan and a trimEnd.

    The suite drives optimize, which never reaches reorderForCache, so nothing covered it: a fixture list only asks the questions it encodes. reorderForCache now has ten fixtures of its own, each sized so the old pattern is well past the budget rather than near it.

The cacheable-minimum bar is on the resulting prefix, not on the amount moved — minPrefixTokens, not minTokens. Those are different questions, and asking the second one refused a real saving: a prompt whose 1,265-token head already cached gained nothing from 359 stranded tokens, because 359 is below Opus's 512-token minimum. It reported "nothing could safely move", which is not what happened.

The declined list in the report is capped at three lines and now says how many it did not print. A report that shows three of nine reads as "three".

--reorder and --markdown-out were both accepted and undocumented — absent from --help in either locale. The parity test named four required flags and passed the whole time; it now derives the list from what the binary actually accepts, by reading the allow-list the CLI prints when it rejects an unknown flag. --markdown-out had been undocumented since 0.11.0.

A post-1.0 roadmap. Next was empty after 1.0.0 — honest, since every planned item had shipped, but the file's stated purpose is that "the ordering is a commitment", and it was committing to nothing.

Four entries, and two of them exist because writing the roadmap turned up things worth saying out loud:

  • 1.2.0 — Releasing without remembering. A tag-triggered workflow, using npm trusted publishing rather than a stored NPM_TOKEN: a long-lived publish token would be the highest-value credential this project holds, sitting in secrets permanently for something used a few times a year. It must refuse to publish a version that does not match the tag.
  • 1.3.0 — The error band, measured. ±15% is printed on every report and asserted nowhere. estimateTokens is tested for zero-on-empty, monotonic growth and not-NaN; nothing checks its accuracy, and every dollar figure descends from it. It is also one number for all text, which the CJK case suggests is not true.
  • 1.4.0 — Prompts where they actually live. Trazum reads .txt/.md/ .prompt/.tmpl, so prompts embedded in TypeScript or Python require refactoring an application before Trazum can be adopted at all.
  • 1.5.0 — The front door catches up. The web app optimises and nothing else, five releases behind the CLI. Last on purpose: it changes how the product looks rather than whether its numbers are right.

Release automation was written as 1.1.0 and is now 1.2.0, because writing the entry established that it cannot ship: publishing needs the @trazum scope to exist on npm, which is the maintainer's to create. Holding the queue behind a prerequisite outside the repository would have been worse than reordering it, and ROADMAP.md says so in the file rather than only here — the ordering is a commitment, so a change to it owes a reason.

The Tokenizer per model family entry under Under consideration now says it is pending the error-band measurement rather than reading as a pure dependency-cost decision. Measuring the band is what settles whether a real tokenizer is needed.

Also fixes a doubled --- left in ROADMAP.md by the 1.0.0 edit.