The v0.10.4 release (run 31755436916) failed at the very last step of `verify`,
after all 69 other jobs had passed — full test matrix, all eight builds, smoke
and soak:
publish-vt-evidence: wrong evidence marker in binaries/vt-results.tsv
The results format was bumped to `cbm-virustotal-results-v2` in #1596, but
publish-vt-evidence.sh still demanded `-v1`. It was the only straggler: the
gate, the selector, the notes renderer and all three contract fixtures were
already on v2.
It survived because that PR had removed the script's only caller, leaving it
dead code that nothing exercised. Restoring the full-surface scan restored the
caller too, and the stale expectation surfaced in the worst possible place — at
the end of a real release rather than in a dry-run, since `verify` does not run
in dry-runs.
Also adds the guard that would have caught it: every marker publish-vt-evidence.sh
validates must be one the gate actually writes. Revert-checked — reintroducing
v1 fails with "publisher expects marker never written by the gate:
cbm-virustotal-results-v1".
The scan itself passed. This is purely the evidence-publishing step.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two things, both found by the dry-run and by sweeping what it exposed.
1. The three-candidate change was incomplete. The dry-run failed at
stage-release-candidates.py, which carries its OWN copy of the transform
validation that the previous commit only fixed in the selector:
stage-release-candidates: candidate linux-amd64/debug-stripped has
invalid transform: 'strip-debug'
Sweeping for that assumption found it in six places, not two:
stage-release-candidates.py, select-release-candidates.py,
verify-release-selection.py, append-vt-notes.sh and two contract-test
fixtures — as hardcoded 16s, `len(TARGETS) * 2`, two-entry VARIANTS tuples
and two-key truth tables. All are now derived from len(VARIANTS).
Field prefixes needed care: the variant NAME keeps its hyphen because it is
the on-disk directory, while the evidence columns use underscores, so
`debug-stripped` reads `debug_stripped_sha256`. Every lookup now goes through
an explicit FIELD_KEY map instead of interpolating the variant directly.
The selection contract test now covers the truth table EXHAUSTIVELY: three
variants x two tolerated classifications is exactly eight combinations, and
there are exactly eight targets, so every case is exercised once.
2. Full-surface VirusTotal scanning is restored. This PR had moved scanning
upstream to the candidates and deleted the post-package pass, which silently
narrowed coverage from everything we ship to executables only. The 42 runtime
files across the 14 containers — install.sh, install.ps1, LICENSE,
THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the unpacked UI assets —
were still extracted, structurally verified and strings-audited, but no
longer scanned at all. install.sh and install.ps1 are the highest-consequence
non-executable bytes we publish; users pipe them straight into a shell.
The verify job scans every extracted object again, under the same policy.
Re-submitting the selected executables alongside them is close to free
because VirusTotal is content-addressed and answers for identical bytes from
its own record — the same property that made the analysis-id equality check
untenable two commits ago.
The gate-chain contract asserted the opposite ("duplicate post-package
VirusTotal path remains"). That assertion is inverted: the pass is required,
and it is not a duplicate, since it covers a strictly larger set. README and
SECURITY.md updated from "archive containers are checksummed rather than
redundantly rescanned" to state the full covered surface.
All five release/VT contract tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Release evidence from run 31744302624 shows the tolerated Microsoft `!ml`
verdict is close to a coin flip per byte image rather than a property of the
code. Across the eight targets the stripped and unstripped candidates of the
SAME linker output disagreed on four, and in both directions:
linux-amd64 stripped microsoft-ml unstripped clean
darwin-arm64 stripped clean unstripped microsoft-ml
linux-arm64 stripped clean unstripped microsoft-ml
If the classifier were keying on something intrinsic to our code the siblings
would agree; they do not. So each variant is close to an independent draw, and
5 of 16 candidates drew the flag.
Two draws is not always enough. On that run linux-amd64-portable came back
microsoft-ml on BOTH candidates, leaving no clean binary to ship for that
target. A third independent draw at a ~31% observed per-candidate hit rate takes
the both/all-flagged case from roughly 1-in-10 per target to roughly 1-in-30.
The third candidate is `--strip-debug` (Apple: `-S`): debug information removed,
symbol table kept. Behaviourally identical to the other two — same linker
output, only metadata differs — but a distinct byte image, which is all
VirusTotal needs to scan it as its own file. Verified on the real v0.10.4
candidates: linux-amd64 gives three distinct hashes (294,634,656 /
294,623,208 / 293,746,096 bytes) and darwin-arm64 likewise, with the ad-hoc
signature verifying after strip.
Selection is unchanged in spirit and now ordered: smallest artifact first
(stripped, debug-stripped, unstripped), take the first CLEAN one, and only if
every candidate drew the tolerated verdict ship the smallest flagged one. A
hard verdict on any candidate still blocks the release before selection.
Cost is 24 objects per release instead of 16.
Derivation enforces that all three hashes differ — identical candidates would be
one draw wearing three hats, and the selector would believe it had alternatives
it does not have. Public claims in README, SECURITY.md and docs/index.html
updated from "both stripped and unstripped" to the three candidates.
All three release contract tests pass, including the native derivation test
which exercises the real strip and codesign path on this host.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The build+VT dry-run (run 31744302624) blocked all eight targets on:
BLOCKED: objects/3eb486a4... completed with only 48/75 decisive engines (< 50)
That object is the windows-arm64 STRIPPED candidate with ZERO detections. It
was classified "hard" — the same bucket as a real malware verdict — purely
because 48 engines answered instead of 50. Its own sibling scanned clean with
66.
How many engines return a decisive result is VirusTotal fleet availability on
the day, not a property of our binary. These artifacts are ~300 MB and many
engines skip or time out at that size, so the count sits near the floor and
moves run to run: on this run one object landed at 48 and the other fifteen
spread 59-68. A floor there makes shipping a lottery decided by someone else's
infrastructure, which is exactly what our CI-determinism rule forbids, and it is
not something we can influence by fixing anything on our side.
Classification now depends only on what engines FOUND:
hard - two or more engines, any non-Microsoft engine, any label that is
not `!ml`, or anything suspicious. Still blocks the release.
microsoft-ml- exactly one Microsoft `!ml`. Tolerated and disclosed.
clean - nothing flagged.
The decisive-engine count is still recorded per candidate in the results
manifest and printed as a NOTE when it is below the reference value, so the
evidence is unchanged; only its power to block is gone. The selector keeps
rejecting an incoherent response (total < completed).
Public claims updated to match in README.md, SECURITY.md and docs/index.html —
all three promised "at least 50 decisive engines" as a publication requirement,
which would no longer have been true.
Contract test updated: a clean verdict now stands regardless of engine count,
and the below-reference NOTE must still be reported. All three release contract
tests pass (vt-gate-policy, release-gate-chain, candidate-derivation).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The previous commit framed accepting VirusTotal's own analysis id as giving up
freshness. That was the wrong way round.
VirusTotal recognising our bytes is desirable: the evidence this pipeline
publishes is the hash-keyed file report — append-vt-notes.sh asserts the URL is
exactly .../gui/file/<sha256>/detection — so the report we gate on is the same
one a reader gets by looking that SHA-256 up themselves. Requiring a freshly
minted analysis id would have contradicted the evidence we publish next to it,
and made our verdict unreproducible by anyone outside the release job.
Comment-only; no behaviour change. The build+VT dry-run on 094929fd exercises
identical logic.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The dry-run for this branch (run 31729540899) failed in `select-package`, the
new job, after every build had completed and before smoke — the exact shape of
failure that would have taken a release down:
BLOCKED: VirusTotal response analysis id does not match the submitted
analysis: objects/70fbbc37...
That object is the linux-arm64 UNSTRIPPED candidate, 295,476,856 bytes.
VirusTotal is content-addressed, and for bytes it already holds it may answer
with its own canonical analysis rather than the one this upload created. A large
release binary is exactly the artifact most likely to be already on file, so
requiring the ids to be equal would have recurred on most releases rather than
being a one-off.
The equality check was guarding the right idea at the wrong level. What has to
hold is that a verdict describes THESE bytes, and parse_completed already
enforces that a few lines below against meta.file_info.sha256 and
file_info.size — a strictly stronger binding than the id. It is also what keeps
a tuple's two candidates apart: stripped and unstripped differ in hash by
construction (70fbbc37... vs 28b3890f... on linux-arm64), so neither can be read
as the other whatever ids VirusTotal hands out. The wrong-hash and wrong-size
contract cases pin that and still block.
The property deliberately given up is freshness: an older cached analysis of
identical bytes is now accepted. For identical bytes that is the same content
risk, and it buys a gate that is deterministic instead of one that fails
whenever VirusTotal recognises our own binary. The submitted id is still
recorded in the results manifest as evidence.
The selection policy is unchanged and needs no change: stripped clean ships
stripped; stripped microsoft-ml with a clean unstripped ships the clean one;
both microsoft-ml ships stripped; a hard verdict on either sibling still blocks
the release for analysis.
Contract test updated to assert the passing direction, and revert-checked: with
the fix reverted it fails with the production message, with it applied it passes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three comments still described the original design: "Three attempts over ~30ms
total" (it has been six attempts over ~630ms since the backoff landed), and a
hard-coded list of waits plus a "~0.6s ceiling" rationale that is now only the
unsanitized half of the story.
Rewritten so the prose derives from CBM_SPAWN_RETRY_ATTEMPTS instead of
restating it, which is what let the first one go stale. Comment-only; no
behaviour change. subprocess suite 31/0/0, clang-format clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The previous commit widened the spawn-retry budget for sanitized builds, and it
did nothing on TSan — the leg it was written for. `subprocess_run_spawn_failure`
failed again on the very PR that was meant to fix it.
CBM_SANITIZED_BUILD comes from SANITIZED_DEFINE, which keys off $(SANITIZE).
TSan does not use that variable — it has its own TSAN_SANITIZE — and CFLAGS_TSAN
never included SANITIZED_DEFINE. So the macro was undefined on that leg and
every sanitized-budget branch compiled to its NATIVE value while running an
instrumented, several-times-slower binary.
The comment above SANITIZED_DEFINE already describes this exact failure for
trap-UBSan: "the build system is the single source of truth for is this binary
instrumented; compiler-specific probes miss clang's feature-check spelling and
every non-ASan sanitizer". That lesson was recorded and the TSan leg was never
wired up to it. Nor would compiler probes have saved this: clang spells thread
instrumentation __has_feature(thread_sanitizer), not __SANITIZE_THREAD__.
CFLAGS_TSAN now defines it unconditionally, which is honest — that flag set
exists solely to build an instrumented binary.
Checked the neighbours: MSan (scripts/msan.sh passes SANITIZE=) and the diag
lane (passes SANITIZE= too) both already get the define. TSan was the only gap.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The exponential backoff shipped in v0.10.3 (10/20/40/80/160/320ms, ~0.6s) fixed
the ordinary case. `subprocess_run_spawn_failure` then kept failing on
`test-tsan (macos-14)` — twice on the same SHA, on a PR whose entire diff was a
shell contract plus one line in test.sh, so causation was impossible.
ThreadSanitizer runs several times slower and holds far more process state, so
the pressure window it creates is simply longer than 0.6s. The budget was right
for a normal build and short for a sanitized one.
Raising it for everyone would be the wrong fix: an unsanitized machine that is
genuinely out of capacity should fail fast rather than hang for seconds. So the
extra patience is scoped to the builds that need it — three more doublings,
about 5s, under CBM_SANITIZED_BUILD or __SANITIZE_*__ only. Same shape as the
daemon announce backstop in test_daemon_frontend.c, and for the same reason.
Also fixes a bug in the backoff itself: the shift was capped at a hard-coded 6,
so with a budget of 9 the final three waits would have flattened to 320ms each
instead of continuing to double — the extra attempts would have bought a third
of the intended time. The cap now follows the budget.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A duplicated install left a Codex config.toml carrying a CLOSING
`# <<< codebase-memory-mcp SessionStart <<<` with no opener. Every later
install then failed that client with op=legacy_hook_cleanup and aborted the
whole activation, so one bad file made the installer permanently unusable for
that client. Reported by @PsyTech-Rob.
We are the only writer of these markers, which makes the imbalance our own
residue. Refusing to touch a file we cannot parse is the right default in
general — it is the wrong default for a mess we made.
Removal now strips the stray line. The bounds come from the marker scan that
already ran, so nothing is guessed and no surrounding content is touched; the
user's own keys either side survive verbatim.
A WRITE still refuses on an imbalance, deliberately: with only one marker there
is no defensible region to replace, and guessing could destroy user content.
That asymmetry is the whole design — heal where the answer is unambiguous,
refuse where it is not.
Both orientations are tested (orphan opener, orphan closer) and revert-checked:
without the fix each returns the same -1 that aborted the reporter's install.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`ui_enabled` governs a loopback HTTP listener on 127.0.0.1:9749, and the only
way to turn it off was hand-editing ~/.cache/codebase-memory-mcp/config.json.
The key was absent from CONFIG_KEYS, so `config list` could not show it and
`config set` rejected it outright. A reporter spent two debugging sessions
finding the switch, and described the surface as looking impossible to turn off.
A network surface a user cannot discover how to disable is not acceptable
whatever its default, and "documented in the source" is not discoverable.
ui_enabled and ui_port now appear in `config list`, and get/set/reset work on
them. They live in a separate file (cbm_ui_config_load/save) rather than the
key-value store the other keys use, so this routes rather than merely listing
them — set validates (true/false; a port in 1..65535) and reports that the
daemon must restart for the change to take effect.
The test asserts DISCOVERABILITY rather than the current values: both keys must
be present in the table that `config list` and `config set` walk, because a key
missing there is a key the user can neither find nor change.
Reported by @PsyTech-Rob.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The previous commit pinned the linux/amd64 asset hash directly, which
verified the asset this job needs but hard-coded the platform: the
uname-based selection was replaced by a fixed filename, so moving the job
to another runner OS or architecture would have needed a code change and
a second pinned hash.
Pin the SHA-256 of the release's own checksums file instead, and verify
whichever asset the runner selects against it. Upstream publishes that
file for the whole release, covering linux, darwin and windows on both
amd64 and arm64, so one pinned value now covers every platform and the
uname-based selection is restored.
Match the asset by exact filename, since a substring match would also
accept the .sbom.json and .sigstore.json lines for the same asset, and
fail closed when an asset is absent from the checksums file rather than
installing it unverified.
Verified end to end against the pinned release: the checksums file
matches its pinned hash; asset selection resolves to a listed asset for
linux/amd64, linux/arm64, darwin/amd64 and darwin/arm64; the linux/amd64
asset verifies and extracts to a statically linked x86-64 ELF; and
appending one byte to the downloaded asset makes verification FAIL, so
the gate binds rather than passing vacuously.
Co-Authored-By: Andrew Hundt <ATHundt@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The publish-mcp-registry job fetched mcp-publisher from the `latest`
release and piped curl directly into tar. Whatever upstream published at
that moment therefore executed inside the job that holds the MCP Registry
publish credential, with no opportunity to verify it first.
Pin the release to v1.8.1, download to a file, verify its SHA-256 against
the checksum published in registry_1.8.1_checksums.txt for that same
release, and only extract once the hash matches. The job runs on
ubuntu-latest, so the linux/amd64 asset replaces the uname-derived
selection.
Reported by Andrew Hundt in #1245.
Verified: fetched the pinned asset (7,339,841 bytes, matching the release
asset size), confirmed its SHA-256 against upstream's checksums file, and
extracted a valid statically linked x86-64 ELF.
Co-Authored-By: Andrew Hundt <ATHundt@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@gotspatel's Claude Desktop log showed the entire failure as:
Server transport closed unexpectedly, this is likely due to the process
exiting early
for what was a specific, nameable refusal. v0.10.3 added a JSON-RPC error for
bootstrap failures (#1539), but every EARLIER exit on the client path still
wrote to stderr only — and no MCP client surfaces stderr. So the guarantee "a
server that cannot start always says why" held for exactly one failure mode.
The worst of them was the endpoint-creation exit, which is precisely where an
ownership or ancestry refusal lands: the very failure most likely to stop a
fresh install was the one that said nothing at all.
Client-path exits now report through the same path as bootstrap failures, and
the endpoint failure carries the validation detail — so instead of a closed
transport the client receives the directory and the rule that refused it.
The reporter was extracted rather than duplicated (main_report_client_failure),
so a future exit added on this path gets the behaviour by calling one function
instead of remembering to hand-roll a JSON-RPC frame.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On 0.9.0 -> 0.10.2, `update -y` auto-confirmed deleting other projects' indexes,
removed two .db files (~59 MB), and THEN aborted on an interactive prompt it
could not answer in a non-interactive shell. The run failed; the data was
already gone. Reported by @PsyTech-Rob.
The current code is already correct: both activation flows prompt early but
delete late, after the binary swap and agent configuration, and both return
before the deletion when configuration fails. #1538 also removed the specific
prompt that aborted. So this commit fixes nothing — it stops the fix from being
undone, which is the part that was missing. The reporter made exactly this
point: removing one bad prompt addressed the instance, not the rule.
Contract: in each flow that deletes indexes, cbm_remove_indexes() must appear
after the agent-configuration call. It keys on the CALL rather than a local
result variable, because the two flows spell that result differently, and it
fails loudly if a flow is renamed rather than passing vacuously.
Verified by reverting: moving the deletion ahead of configuration reproduces
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1558's shape and the contract goes red.
v0.10.3 split the activation refusal into two messages so a reservation failure
would stop blaming "active CBM sessions" for something no session caused. That
half worked. The other half did not: the new message tells the reader to "check
the errors above", and nothing is above.
The detail that names the failing component is recorded on the daemon side
(ipc_validation_detail) and was only ever read by `daemon status`. So the CLI
refusal replaced a message that blamed the WRONG thing with one that blamed
NOTHING, and two reporters — #1537 on macOS, #1416 on Linux — were left with a
refusal, a reboot that changed nothing, and no way forward.
The refusal now prints the check that actually refused:
error: activation could not reserve exclusive access; no activation was
committed.
error: this is NOT a running-session problem — nothing needs to be closed.
The check that refused was: <path>: ancestor '.cache' is not a usable
private-directory parent
The test asserts the PROPERTY rather than the wording: the message must never
point at evidence it does not show. A test pinned to the current text would
have passed happily while the dangling pointer was live, which is exactly how
this shipped.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`install` configured EVERY detected client. Someone who wanted Claude and Codex
had to revert the OpenCode and Cursor integrations by hand — and the next
`install`/`update` silently recreated them, so the choice could not be made to
stick. Reported by @PsyTech-Rob.
`--clients=claude,codex` restricts it. Omitting the flag keeps today's
behaviour, so nothing changes for anyone not asking for this.
The vocabulary is what makes the flag usable rather than decorative. 26 clients
ship here, several with tokens nobody would guess — factory-droid, mistral-vibe,
copilot-cli, kilocode. So:
- `install --clients` prints every token with its display name and an example;
- an unknown token FAILS and prints that same list. Silently treating a typo as
"that client was not detected" would configure nothing and report success,
which is the failure mode this project keeps having to fix;
- validation runs before anything is written, so a typo cannot strand a
half-configured install.
The test pins the whole table rather than a sample: every token must resolve, so
a client added to detection but forgotten in the selector's list is caught here
instead of by a user who cannot select it.
Per the maintainer's decision `install` keeps its name — splitting it into a
separate `setup` command (#1566) stays a deliberate future decision rather than
something folded into a fix batch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`install`/`update` bundle two separable jobs: place the binary (and put its
directory on PATH), and configure the agents. For someone who installed through
mise, Homebrew or nix, the first job is not redundant — it is harmful. It drops
a SECOND copy into ~/.local/bin that shadows the managed one depending on PATH
order, and appends to a shell rc file no package manager asked us to touch:
Would install binary -> /Users/user/.local/bin/codebase-memory-mcp
Added /Users/user/.local/bin to PATH in /Users/user/.zshrc
They wanted the agent configs refreshed and nothing else. Reported by @SC-CTS.
`--skip-config` (binary, no configs) has existed for a long time; this adds its
missing mirror. Per the user's decision:
- **Infer by default.** A recognised manager path means the binary is not ours;
we configure the agents and leave the binary and PATH alone.
- **`--skip-binary` / `--force-binary`** override the inference either way.
- **`update` REFUSES** on a foreign binary and names the command that will work
(`mise upgrade`, `brew upgrade`), rather than reporting success while changing
nothing — the dishonest-success pattern this project keeps having to fix.
- **PATH is only touched when WE placed the binary**, which is a bug in its own
right regardless of the rest.
Detection is POSITIVE evidence — the path must look like a known manager's — and
the first implementation got this wrong in the tempting way: "anything outside
our install dir". The test suite failed immediately and was right to. A test
binary runs from build/, and an ordinary `install --dir=/opt/cbm` also lives
outside the default; both would be misread as foreign, and `update` would then
refuse to update an installation we own. The asymmetry settles it: a false
positive costs a working update, while a false negative merely leaves today's
behaviour for an unrecognised manager, which --skip-binary covers explicitly.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
v0.10.3 published, then failed:
MCPB package '...codebase-memory-mcp-darwin-amd64.mcpb' is not publicly
accessible (status: 404)
publish-mcp-registry and publish-final both needed only publish-registries, so
they ran in PARALLEL — and publish-final is the job that un-drafts the release.
The registry validates every package URL it is handed by fetching it, and a
draft release's assets are not publicly readable. The registry lost the race by
five seconds; the identical URL served 200 once the release went public, and the
job passed on a plain re-run.
The registry now needs publish-final. This keeps the documented intent exactly:
publish-final still does NOT depend on the registry, so a registry-preview
outage can never block shipping — the registry simply runs after the assets it
validates exist to the outside world.
The gate-chain contract now pins BOTH directions, because either one alone is a
bug: the registry must depend on publish-final, and publish-final must never
depend on the registry. Verified by reverting the ordering — the contract goes
red with the exact v0.10.3 failure, and green again with it restored.
This is the second latent defect the .mcpb path produced on its first real
release (after the security-strings audit treating a manifest as a binary).
Both were invisible until a stable release ran; an end-to-end bundle gate is
worth adding before the next one.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
An elevated install created its own staging file and then refused it:
error: failed to stage install candidate: activation transaction I/O
failed: owner-not-current-user
We compared the object's owner against TokenUser. Windows stamps newly created
objects with TokenOwner, and for a member of the Administrators group that
defaults to BUILTIN\Administrators ("System objects: Default owner for objects
created by members of the Administrators group" — default on Server, common on
hardened clients). So on any elevated account the check rejected a file written
seconds earlier by the same process. Reported by @spam0115 (#1580); @gotspatel
hit the same wall from the other side (#1582) — a fresh install whose MCP server
exits during initialize, from `C:\Users\Administrator`.
The fix reads TokenOwner and accepts that SID too. This is stricter-or-equal,
never looser: TokenOwner is the ONE SID this process stamps on its own
creations, so the check still refuses anything a different account owns. It does
NOT accept "whatever an administrator owns" — that would be the loose version,
and the neighbouring activation_windows_owner_is_trusted already covers the
pre-existing-directory case where a broader rule is appropriate. On a
non-elevated account TokenOwner == TokenUser and nothing changes at all.
Windows-only code, so it is unverifiable on the macOS host: this needs the
Windows leg to compile and exercise, which is the next step before merge.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The formula still pinned 0.8.1 — five releases and two months behind — so
`brew install` handed out a June binary while GitHub, npm and PyPI all served
the current one. Nothing in the release pipeline touches this file, which is
how it drifted that far without anyone noticing.
Version and all four SHA-256 digests come from the published v0.10.3
checksums.txt.
Worth doing separately: have the release workflow rewrite this formula from
checksums.txt after publish. A channel nothing updates is a channel that rots,
and the failure mode is silent — brew keeps working, it just installs something
old.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The EAGAIN retry shipped in v0.10.3 was too small to matter. It waited a flat
3 x 10ms, which covers a momentary dip and not the real thing: a CI runner
building and testing in parallel stays process-starved for hundreds of
milliseconds at a stretch. `subprocess_run_spawn_failure` kept failing WITH the
retry in place — macos-15-intel on a release matrix, then macos-14 under
ThreadSanitizer, twice on the same SHA, which is what moved this out of the
flake bucket. A fixed short delay just samples the same congested instant
repeatedly; doubling walks out of it.
Now 10/20/40/80/160/320ms — about 0.6s of total patience. That is invisible
next to spawning a process that does real work, and a machine still refusing
after it is genuinely out of capacity, where failing fast beats hanging.
The previous attempt shipped a constant with no way to prove it worked, so this
adds the seam that was missing: CBM_ENABLE_TEST_SEAMS builds can force N
simulated refusals, and two tests pin the behaviour deterministically — fewer
refusals than the budget must still spawn, more must fail rather than retry
forever. Both are revert-checked. The seam compiles out entirely in production
(cppcheck correctly called the always-false branch dead code, so it is gone
rather than suppressed).
Two things the tests caught that review had not:
- The macOS primary path is posix_spawn, not fork; injecting only into the fork
fallback exercised nothing on macOS. Both paths now carry it.
- cbm_fork_with_retry ended with an unconditional fork() OUTSIDE the loop, so
the final attempt could never be reached by a test. Every attempt now goes
through the same branch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
OpenCode reads either opencode.json or opencode.jsonc. We always wrote the
.json name, so a user whose real config is opencode.jsonc got a SECOND config
file that OpenCode ignores: the MCP server never appeared, while the installer
reported success. Silent, and indistinguishable from "the tool is broken".
The install target is now whichever file already exists, .jsonc first since it
is the one we used to miss. With neither present, .json is created exactly as
before, so fresh installs are unchanged. Other clients in this file (pochi,
kilo) already resolve .jsonc paths — OpenCode was the gap, not the exception.
Reported by @iandol in discussion #1560, who noticed their config was .jsonc
while the installer named .json. Verified by revert: the test fails without
the fix and passes with it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.10.3 release failed its `verify` gate:
--- Auditing: scan-...--manifest.json ---
BLOCKED: Unauthorized URL in binary: https://github.com/DeusData
BLOCKED: Unauthorized URL in binary: https://deusdata.github.io/...
=== BINARY STRING AUDIT FAILED ===
The blocked URLs are OURS — the .mcpb bundle manifest's own homepage and
documentation fields.
security-strings.sh already knows that URL and dangerous-command auditing are
"designed for compiled artifacts" and skips them for text, keeping the
credential and base64 audits that are universally meaningful. Its detection
just had no case for JSON: `file` reports "JSON data", which matched none of
the shell-script/ASCII-text patterns, so a manifest was audited as if it were a
compiled binary. An MCP manifest is exactly the shape that misfires — it
carries project URLs by design, and a `command` field by specification.
This is the first release to ship .mcpb bundles, which is why it surfaced now:
it is the first time a manifest reached the scanned object set. The bundles
were noted as end-to-end unproven until a stable release; this is that proof
doing its job.
Two changes:
- Classify structured-text artifacts by extension FIRST (.json/.yaml/.toml/
.md/...), then by `file` type with JSON/XML added. The extension check also
covers hosts without file(1), where the existing `command -v` guard silently
left every artifact classified as binary.
- Allow-list our own org root and documentation site, so a compiled binary
embedding the docs link (the graph UI does) passes on its own merits rather
than relying on the text exemption.
Tests: the manifest case, plus a control proving the exemption is narrow — the
credential audit still runs on structured text. The existing negative control
(an unauthorized URL in a real binary is still BLOCKED) is unchanged and still
passes, so the binary audit is not weakened.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Codex sanitizes the environment of stdio MCP subprocesses and passes through
only the names listed in `env_vars`. Our generated block had `command` and
`args` and nothing else, so a user with a custom CBM_CACHE_DIR got a
Codex-spawned server reading the DEFAULT cache while their account daemon used
the configured one. The two disagree during initialization, the connection
closes, and Codex exposes no cbm tools at all — a total failure that looks like
a broken MCP server rather than a missing environment variable.
The name is listed unconditionally rather than only when the variable happens
to be set during install: `env_vars` names variables to forward IF PRESENT, so
listing it costs nothing when unset, and it keeps working for someone who sets
CBM_CACHE_DIR after installing — a case install-time detection would silently
miss, producing exactly this bug again with no way to tell.
Reported by @anton-matosov, who verified the corrected block end-to-end before
filing.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The installed SKILL.md carried an unquoted `description` whose value contains
"Triggers on: ". In YAML a colon-space inside an unquoted scalar is a
nested-mapping indicator, so strict readers — js-yaml's `load`, the frontmatter
parser in `npx skills` — reject the whole document. The skill does not load
wrongly; it does not load at all, and nothing in the install output says why.
Quoting the scalar fixes it without touching a word of the text.
The test checks the RULE rather than this string: for every skill, any
frontmatter value containing ": " must be quoted. Pinning the current wording
would let the next skill someone adds reintroduce exactly this bug. Verified by
reverting the fix — the test goes red, and green again with it restored.
Reported by @xezzon.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two contributors independently sent the same fix for #1550 while ours was
already in CI (#1553 by @musichen, who also reported the bug, and #1564 by
@henri-edh). Rather than close both as duplicates, this takes the part of
theirs that ours was missing.
@henri-edh's comment names the actual mechanism: Pi loads an extension via
jiti.import(path, { default: true }) and rejects anything that is not a
function, so a named-only export hands the loader the module namespace object
and the install fails with "Extension does not export a valid factory
function". Ours said "Pi calls the default export" — true, but it does not tell
the next reader why a named export cannot simply be added alongside, which is
exactly the change someone would try. Their wording is now ours.
The assertion is tightened the same way, from `export default function` to
`export default function (pi)`, so a default export with the wrong shape cannot
satisfy it.
Co-Authored-By: musichen <musichen@users.noreply.github.com>
Co-Authored-By: henri-edh <henri-edh@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The reporter came back with the output that named this:
exact executable identity could not be verified (cache-private) -
/Users/.../.cache/codebase-memory-mcp: ancestry component validation
failed (errno 2)
Two separate defects, and the second is why nobody could see the first.
**The refusal.** The daemon's private-directory ancestry walk rejected any
group-write bit on any ancestor — `(mode & 0022) == 0`. That is the identical
rule #1535 removed on the activation side, in the sibling gate the consolidated
strictness decision covers but that never got changed. A group-writable ~ or
~/.cache is ordinary: WSL2 ships 0775, so do several distro skeletons and any
site with a shared primary group. So the daemon refused to run, permanently,
with nothing the reader could close or restart.
World-writable is still refused (any local user could swap a path component),
and the private directory itself is still chmod'd to 0700 and verified after
the walk — which is what makes admitting the ancestor safe: the thing that
actually holds data stays owner-private either way. Group-writable ancestors
are logged, so the weaker check is never silent. Both directions are tested;
a fix that admitted everything would "fix" this by deleting the protection.
**The invented errno.** When that check failed, the branch set no detail, so
the caller fell back to printing `errno` — which nothing on the path sets. The
reporter was handed ENOENT for a permission refusal and went looking for a
missing file that was there all along; it cost a round trip to find out the
number meant nothing. The branch now names the component and the rule.
Reported by @listepo, who ran the diagnostics that turned this from a guess
into a one-line root cause.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The local Windows leg caught this within hours of merging #1532: on a machine
that is not domain-joined, USERDOMAIN is the literal string WORKGROUP, and
`WORKGROUP\test` is not a resolvable principal —
WARN: build-dir DACL stamp (pre-wave) failed (user=WORKGROUP\test ...):
WORKGROUP\test: No mapping between account names and security IDs was done.
so the grant fails outright and the tree keeps its inherited
`Authenticated Users:(M)`. The guard then correctly reports that cross-account
mutation is still possible, and the whole leg stops. Since most Windows
machines are not domain-joined, that is the common case, not the exotic one.
#1532 was fixing a real bug in the other direction (a bare name resolves
against the machine first, so a host named like its user grants to an empty
principal). Both failures are name RESOLUTION failures, so this stops resolving
names: the account is identified by SID, which is unambiguous everywhere. The
SYSTEM and Administrators grants in these same commands have always used that
form — the current user was the odd one out. Name lookup survives only as a
fallback where PowerShell is unavailable.
Applied to all three scripts carrying the stamp (run-tests-parallel, soak-test,
memlab), since they share the failure and would otherwise drift apart.
Co-Authored-By: Kiborgik <Kiborgik@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`test-msan` failed the two backpressure cases while every other lane was green,
and the same suite passes under MSan locally (12/12). The wait that fired is a
LIVENESS BACKSTOP, not a race budget: the daemon child either announces on its
ready pipe or it does not, and the test asserts the announcement itself, never a
timing window.
A backstop only does its job if it never fires on a legitimately slow start.
30s was already the second value here — the first was below the daemon's own
12s cohort-claim budget and SIGKILLed healthy startups on oversubscribed
runners. Under MemorySanitizer, with instrumented libc++ and origin tracking,
startup runs several times slower than that same budget it has to clear, so 30s
stopped being a backstop and started being a verdict.
Sanitized builds now get 180s, matching the existing sanitizer-aware pattern in
test_cs_lsp_bench.c. This costs nothing when the daemon is healthy: a passing
run returns the moment the byte arrives, whatever the ceiling. Nothing about the
asserted state changed, and the non-sanitized bound is untouched.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A reporter ran `uninstall`, then could not install again: every attempt said
"active CBM sessions and operations could not be stopped safely". A reboot
proved no such session existed, and the remedy line v0.10.2 added made it worse
— it told them to run `codebase-memory-mcp daemon status`, a binary they had
just uninstalled.
Both halves were wrong for their case:
The guard funneled two unrelated outcomes into one message. A cohort that is
BUSY means real sessions hold it and closing something is the fix. A reservation
that FAILED — lock I/O, leftover coordination state, permissions — means nothing
is running at all, and telling that reader to close sessions sends them after
processes that do not exist. Same misattribution class as #1416 and #1535: a
non-session failure wearing a session costume. They are now separate messages,
and the failure one says plainly that nothing needs to be closed.
The remedy also has to survive the situation it prints in. Install, and any
retry after uninstall, runs with no cbm on PATH; advice that starts with
`codebase-memory-mcp ...` is unusable exactly when it is needed. The busy
message now leads with the actionable step (close the editor/agent holding it)
and mentions the command only as an aid where a binary still exists; the
failure message asks for a cache listing, which needs no cbm at all.
A recorded transaction refusal still outranks both generic messages, so the
#1416 attribution is unchanged.
This does NOT yet explain why their reservation fails with nothing running —
that needs their environment, which is being requested on the issue. What it
does is stop sending them after the wrong thing while we find out.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`posix_spawnp` and `fork` both fail with EAGAIN when the process table or a
per-user limit is momentarily full. Both paths treated that as a hard failure,
so a burst of process creation turned into CBM_PROC_SPAWN_FAILED — a git probe,
an LSP server, or an index worker refusing to run for a reason the user can
neither see nor act on, on a machine that is merely busy.
This is the production half of a flake we have been re-running for two days:
`subprocess_run_clean` failed spawning `/bin/sh -c "exit 0"` on macOS runners in
three separate CI runs, always under a loaded release matrix, always green on a
quiet rerun. Chasing it as a test race would have been chasing the symptom — the
test is right, and a user on a loaded laptop hits the same edge.
EAGAIN/ENOMEM now retry three times over ~30ms; everything else keeps failing
immediately, and an exhausted system still fails after the backoff rather than
spinning. exec-class errors (ENOENT/EACCES/ENOEXEC/...) are untouched, so the
documented "missing binary exits 127, spawn_failed means the mechanism failed"
contract still holds. The fork fallback gets the same treatment: it fails for
the same reason under the same pressure, and the fallback must not be less
robust than the primary path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`trace_path(include_evidence:true)` added `strategy` and `confidence` per hop in
the default tree output, and returned `cols: ["name","hop"]` under
format:"json" — no error, no warning, just two fields missing from the response
shape the schema and `--help` both promise.
include_evidence was only ever implemented in bfs_to_tree_table; bfs_to_tree_json
never took the flag. So the callers most likely to want the evidence — the ones
asking for structured output to consume programmatically — were exactly the ones
who silently got none of it. Reported by @asemraza.
The json path now emits both columns with the same semantics as the tree path,
including the placeholder case: the root hop has no inbound edge and non-CALLS
edges record no strategy, so those rows carry nulls rather than dropping fields
and leaving callers with ragged rows. Tree says "-", json says null — the same
promise in the form each consumer can test.
The existing evidence test covered only the tree path, which is why this went
unnoticed; it now asserts both formats.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1545 fixed the defect that let concurrent instances quarantine each other's
healthy databases, but shipped without tests — and this is a data-loss path, so
it gets coverage before it ships.
Three cases, one per verdict, chosen so a regression in either direction fails:
healthy -> OK (the baseline the other two are measured against)
bad root_path-> CORRUPT (a verdict that never says CORRUPT would protect
broken databases instead of users)
no handle -> TRANSIENT (a file we could not open tells us NOTHING about
its contents; answering CORRUPT there is how a
database nobody had read got renamed and rebuilt)
Co-Authored-By: LynxBay <LynxBay@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two regressions we shipped ourselves, both with the same shape: a change that
was correct in the abstract broke a concrete thing users had working.
**`update --ui` became a hard error (#1544).** v0.10.2 removed the ui/standard
chooser and deleted the flags that drove it, so a command in scripts, aliases
and muscle memory started failing with "unknown update option" where v0.10.1
had printed the install-script handoff. Retiring a CHOICE is fine; retiring the
WORDS is a break users get nothing for. Both flags are accepted again, do
nothing, and print one note explaining that every build now includes the UI.
A genuinely unknown flag is still rejected — the point is compatibility, not
silence. Reported by @ehendrix23.
**The generated Pi extension could not load (#1550).** Since #1392 the
extension has been generated from the tool registry, emitting
`export function register(pi)`. Pi loads an extension by calling its DEFAULT
export as a factory, so the file failed to load — and a Pi extension that fails
to load takes every `pi` command with it, `pi doctor` included, not just cbm's
tools. It now default-exports the factory. Reported by @musichen.
The existing generator test passed throughout: it asserted WHICH tools the file
lists, never that the file Pi loads is loadable. The new test pins the export
shape, which is the property that actually matters to the consumer.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
core.filemode=false hid the +x here, and release.yml invokes the
registry-entries generator directly — a 644 mode would fail the publish
job the first time it runs.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every release now carries .mcpb one-click-install bundles alongside the
archives, and the MCP Registry entry lists them with per-file sha256:
- package-release.sh (canonical) builds codebase-memory-mcp-<target>.mcpb
for darwin/windows and the STATIC linux builds — manifest.json + the same
staged (stripped, gated) binary + LICENSE + THIRD_PARTY_NOTICES.md. The
glibc-dynamic linux targets stay archive-only: a dynamic binary defeats
the one-click promise.
- _build.yml / release-draft: bundles flow through provenance attestation,
checksums.txt, cosign signing and the release asset list; checksums.txt
is also preserved as a same-run artifact for the registry job.
- verify: the canonical scan matrix grows to 14 containers; MCPB manifests
are validated (parse, binary server, entry_point member, command binds
the entry point). Bundle binaries dedupe to the archive scan objects, so
the VT gate gains only the three distinct manifest.json files.
- publish-mcp-registry: gen-mcpb-registry-entries.sh appends one mcpb
package entry per bundle (release-asset URL + fileSha256 from the
attested checksums) to server.json before mcp-publisher runs.
Idempotent; a checksums file without bundles is a hard failure.
- contracts: Step 0o pins the bundle shape at its producer on every leg,
Step 0p pins the registry entries against the live server.json, and the
extractor contract covers the 14-container matrix incl. broken-manifest
fail-closed cases. The linux test image gains zip for the packager.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Five field reports in the 24 hours after v0.10.0 all pointed at the same thing:
gates that were right in principle refused real, ordinary setups, and then
failed to say why. Per the consolidated strictness decision, each gate keeps the
protection that matters and drops the part that was refusing legitimate users —
and every refusal now names what it refused and how to proceed.
**Daemon image gate: npx and every ephemeral install path (#1539, #1383).**
The admission check treated "the peer's image hashes differently" and "the
peer's image cannot be examined at all" as one failure. The second is what
`npx codebase-memory-mcp` always produces (ephemeral cache path,
unfingerprintable), so every npx-invoked client was rejected — and, because the
client never reported it, agents saw a transport that closed mid-handshake with
zero bytes on stdout. Reported by @wassolles with the admission path already
read and the fix space mapped.
An unverifiable image is now admitted: the rendezvous HELLO immediately above it
has already proven semantic version, build fingerprint, and protocol/store/
feature ABI, and the image check was trading that real proof for an unavailable
one. It logs daemon.client_image_unverifiable_admitted so the weaker check is
never invisible. A fingerprint MISMATCH — the tamper case the gate exists for —
still rejects hard. Separate test seams keep the two modes testable apart.
**Client bootstrap failures are no longer silent (#1539).**
An MCP client that cannot reach the daemon now emits a JSON-RPC error on stdout
naming the reason, plus the same text on stderr. Previously the reason sat in
bootstrap_result.message and the process exited having written nothing at all.
**POSIX activation: group-writable ancestors (#1535, discussion #1526).**
activation_directory_secure required no group or other write bit on the install
directory AND every ancestor. WSL2 ships ~ and ~/.local at 0775, as do several
distro skeletons and any site using a shared primary group, so install.sh failed
for a large fraction of Linux users — reporting a policy refusal as "activation
transaction I/O failed", which sent reporters after disk errors and filesystem
types. Root-caused by @AmirF194 in a clean ubuntu container; @shochdoerfer and
@iandol confirmed independently.
World-writable ancestors are still refused (any local user could swap a path
component mid-transaction). Group-writable ancestors are now warned about and
admitted. The LEAF directory stays strictly owner-private — that is where the
binary is published, and group write there would let another account replace the
executable between validation and exec. Refusals now name the directory, its
mode, and which rule refused.
**The obsolete ui/standard chooser (#1538, from discussion #1526).**
v0.10.0 consolidated to one archive per platform with the UI always embedded,
but `update` still offered a variant choice: "ui" could only 404, and "standard"
quietly WAS the UI build. Reported by @iandol upgrading 0.9.0 -> 0.10.0. The
chooser, its --standard/--ui flags, and the ui- URL plumbing are removed, along
with the CBM_VARIANT=ui remnant in the npm installer.
Already-released 0.9.x binaries cannot be fixed retroactively, so the release
workflow now publishes byte-identical ui-*-named alias assets — their updaters
work again with no user action. The aliases are uploaded AFTER the VirusTotal
gate: they are the same bytes as archives it already cleared, and uploading them
earlier would duplicate every object in the scan set and the provenance manifest.
**macOS install noise and attribution (#1537).**
install.sh silenced the "No such xattr: com.apple.quarantine" line, which is
what happens when a curl-downloaded archive carries no quarantine attribute —
harmless, and it became the title of a bug report about an unrelated failure.
The session-stop refusal now points at `daemon status` to list the client
processes actually holding the daemon, instead of asserting sessions exist and
leaving the reader to guess. Reported by @listepo.
**Riders.** hatchling is pinned in pkg/pypi (an unpinned backend resolved fresh
inside `python -m build` is what emitted Metadata-Version 2.5 and broke the
v0.10.1 publish); SECURITY.md's supported-versions table moves to 0.10.x.
Tests: separate seams for unverifiable vs mismatched peer images with a test per
outcome; activation refusal must name directory + mode + rule; a group-writable
ancestor must stage successfully. The update tests drop the flag that no longer
exists. Verified against each reporter's environment shape.
**Open security alerts (all three, OSSF Scorecard).**
- HIGH, binary artifact: an 8.8 MB compiled Go ELF wrapper had been committed at
pkg/go/codebase-memory-mcp by accident. Removed, and both it and its .exe
sibling are gitignored so `go build` in that directory cannot repeat it.
- HIGH, GHSA-2v37-7h3g-55p8: nanoid < 3.3.17 loops forever when a custom
generator is called with size 0. It reaches us transitively (postcss -> vite),
so it is pinned through the existing graph-ui overrides block rather than
promoted to a direct dependency; the lockfile resolves 3.3.18.
- MEDIUM, unpinned pip command: the publish step installed build/twine by
version only, leaving the whole transitive graph resolved at run time.
pkg/pypi/requirements-publish.txt now hash-pins the complete toolchain (316
hashes), generated on a linux/amd64 python:3.12 image so the wheels match what
ubuntu-latest resolves, and the step runs pip with --require-hashes. Verified
by installing from it in that same image.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.10.1 publish failed twice, for two stacked defects:
1. `python -m build` resolves the UNPINNED hatchling backend fresh inside its
isolated build env, and current hatchling emits Metadata-Version 2.5 —
which the pinned twine==6.2.0 rejects as "'2.5' is not a valid metadata
version". Deterministic, and a time bomb: v0.10.0 published cleanly days
ago on the same pins. Verified locally on identical artifacts: twine 6.2.0
rejects, twine 7.0.0 passes. The pin moves to 7.0.0 and `twine check`
now runs at build time so a metadata regression fails BEFORE upload.
2. The job was not idempotent, breaking its own design comment ("if publish
fails, the release stays in draft so we can re-run"). Attempt 1 published
npm 0.10.1 and then died at twine; the re-run 403'd on its own success
("cannot publish over the previously published versions") and the release
wedged in draft. npm publish now skips when the exact version already
exists on the registry, and twine uploads with --skip-existing — both
registries treat immutable prior success as done, not as a collision.
Workflow-only diff. Unblocks re-dispatching the wedged v0.10.1 release.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The detector's 100x bound blocked the v0.10.1 release twice on macos-15-intel
with 101.1x — and control runs on a quiet arm64 host measure the SAME curve
as 57-68x, deterministically. The resolve path has never been linear for this
fixture shape: per-function cost grows 0.5ms -> 1.7ms from 100 to 2000
functions, the audited short-name-lookup/no-negative-memo superlinearity now
tracked as #1527. A bound calibrated as "linear plus generous overhead" on a
curve that was never linear measures host constants, not complexity.
The bound moves to 200x — mid-way in log space between the known ~60-100x
curve and clear quadratic near 400x — so a real quadratic regression still
fails by 2x margin while known behavior stops producing release-blocking
false alarms on slower hosts. The measured table is pinned in the test
comment; when #1527 fixes the curve, the bound comes back down to ~40x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three regressions shipped in 0.10.0 share one failure shape: an empty result
with a success status, indistinguishable from "nothing found" for the LLM
clients that are cbm's primary consumers.
1) structuredContent {} on the whole tree-format surface (#1522 bug 1).
#1488 replaced the duplicated payload with an EMPTY structuredContent
object while every tool still declared a blanket permissive outputSchema.
Spec-honoring clients (Claude Code among them) treat structuredContent as
THE result when a schema is declared, so search_graph, trace_path,
query_graph, get_architecture, search_code, and detect_changes all rendered
as literally "{}" on their DEFAULT format, on every platform. The corrected
contract: no tool declares an outputSchema (tool output is
format-parameter-polymorphic — no static schema is truthful), JSON-object
payloads keep their parsed structuredContent, errors keep
structuredContent.error, and text-shaped payloads carry NO structuredContent
key at all — which also preserves #1375's no-duplication win.
2) Frontend queue overflow killed the session (found by the #1522 sweep).
Any 7+ requests pipelined in one stdin burst — an agent issuing parallel
tool calls does exactly this — overflowed the 8-frame frontend queue, which
failed the whole session: rc=1 with ZERO bytes of output, every buffered
response lost. A full queue is now backpressure: the stdin reader blocks
until the worker drains (bounded by the same stop/fail flags every teardown
path already sets); only a single frame larger than the entire 12 MiB byte
budget — which could never be admitted — remains a hard failure.
3) config get printed "" with exit 0 for every unset and every unknown key
(#1522 bug 2). list printed stored-or-DEFAULT while get printed
stored-or-EMPTY, and no subcommand validated key names, so a typo was
indistinguishable from a correctly-read setting. One config-key table now
drives help, list, get, set, and reset: get prints the stored value or the
key's real default (the same fallback the runtime readers use), and unknown
keys error with exit 1 on get, set, and reset alike.
Tests — each RED on the pre-fix tree and RED again on revert:
* test_mcp.c: text results carry no structuredContent key; tools/list
declares no outputSchema; the tool-table guard now binds all three
branches (absent / parsed-object / error) for every registered tool.
* test_daemon_frontend.c: the over-capacity contract flips from
"session fails" to "backpressure without loss" — the held first request
plus all 32 over-capacity frames are answered and the run closes cleanly.
* test_cli.c: the config command contract — defaults, round-trip, reset,
and unknown-key rejection on all three subcommands.
* smoke-test.sh: Phase 3z rewritten to the corrected structuredContent
contract, new Phase 3z1 (default-format replies usable in schema-honoring
clients, no outputSchema advertised), 3z2 (24 pipelined calls all
answered), 3z3 (config defaults + unknown-key rejection) — all asserted
against the SHIPPED artifact, where #1488's smoke phase previously pinned
the empty-object behavior as correct.
Verified end-to-end on the locally built production binary: all 15 tools
declare no schema; text tools return ABSENT structuredContent, object tools
populated, error envelopes intact; 7/24/64-deep pipelined bursts all answered
with rc=0; config get prints real defaults and exits 1 on unknown keys.
hook_augment (structuredContent.projects) and index_resilience
(structuredContent.status) consume object payloads and are unaffected.
Fixes#1522.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
test_py_lsp_bench.c:262 failed the v0.10.0 release run TWICE on genuine
attempts (ubuntu-24.04-arm 3/3), while the SAME sha passed that shard in the
morning's dry run. A wall-clock bound on a shared 4-vCPU runner is not a
property of the code, and this repo's testing doctrine says exactly that:
perf tests assert a ratio; a time bound may exist only as a liveness backstop.
This test's perf content already IS ratio-form (resolution ratio with a
sanitizer-aware floor). The 150/1500 ms budget was the anti-pattern half.
It becomes a 30 s liveness backstop so a catastrophic hang still fails the
suite rather than the 900 s shard wall clock, and absolute timings stay where
they belong: scripts/benchmark-*.sh, which record rather than gate.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both Linux soak-quick legs in the dry run failed with:
FAIL: RSS 17592186044293MB > 200MB ceiling
17592186044293 MB is 2^44-123 -- a NEGATIVE ~123MB wrapped through size_t. The
diagnostics snapshot reported mi_process_info()'s current_rss verbatim, and on
Linux mimalloc never sets that field: it keeps mimalloc's internal
committed-page counter, which this project deliberately tunes low
(purge_decommits=1, purge_delay=0) and which can transiently go negative under
purge. macOS and Windows are untouched -- mimalloc reports real RSS there --
which is exactly the observed split: both Linux legs red, both macOS legs green.
mem.c already learned this lesson in full (its comment block documents the same
trap for cbm_mem_over_budget): cbm_mem_rss() is /proc-statm-primary on Linux and
mimalloc-correct elsewhere, and cbm_mem_peak_rss() reconciles peak >= current.
The diagnostics writer now uses both instead of the raw mi fields, so the soak
leak gates measure actual process RSS on every platform.
Verified: diagnostics suite passes; compiles and formats clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first full dry run after #1508 failed in all three smoke-linux-portable
legs: _smoke.yml still expanded a variant matrix and extracted
codebase-memory-mcp-ui-<os>-<arch>.tar.gz — a name the build no longer
produces. PR CI never sees this job (pr.yml calls the smoke wrappers directly),
so the miss only surfaced in the dry-run/release path this workflow serves.
The matrix loses its variant dimension, all three legs extract the unsuffixed
archive, and the positional/SMOKE_VARIANT plumbing is replaced by
SMOKE_REQUIRE_UI=1: these legs smoke the SHIPPED artifact, so a binary serving
no embedded UI is a defect here, exactly like scripts/ci/smoke-artifact.sh.
Verified locally: venue-parity and smoke-fixture contracts pass; YAML parses.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Root cause of the Cyrillic guard failure, found via the finalize diagnostics:
seal_existing rc=-1. The primary store open normalizes the path through
cbm_path_for_file_api (UTF-8 -> wide -> UTF-8, the form SQLite's Windows VFS
needs), but four maintenance opens passed the raw UTF-8 path:
cbm_store_seal_existing_path_for_replace (the one this path hits)
cbm_store_prepare_path_for_replace
cbm_store_backup_path (both source and staging)
cbm_store_dump_to_file
So a generation was WRITTEN through the normalized path and then could not be
RE-OPENED through the raw one. On the second index into a cache the seal step
failed, finalize returned PERSIST_FAILED, and the user was told "Check repo_path
exists and contains source files" about a repo that had just indexed cleanly
(nodes=12 edges=21 in the same worker log).
On POSIX cbm_path_for_file_api is a no-op memcpy, which is why the full 3-OS
ladder and every non-Windows test passed -- the defect is Windows-only and only
manifests on a path the raw UTF-8 -> SQLite open mishandles. It surfaced on the
Cyrillic case first because the guard indexes the same repo TWICE (once for the
count, once on retry), making it the first case to re-open an existing
generation; the earlier ASCII/latin1 cases index into fresh caches.
All four now use cbm_path_for_file_api exactly as the primary open does. Also
upgrades dump_to_file from sqlite3_open to sqlite3_open_v2 with explicit flags.
The finalize.prepare_failed / rename_failed diagnostics from the previous commit
stay: a silent PERSIST_FAILED that blamed the repo cost a day of remote bisection
and must never be silent again.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Cyrillic guard failure came down to this: the worker indexes the repo
perfectly (gbuf.dump nodes=12 edges=21, DB written, sealed, sidecars staged) and
then finalize fails somewhere between stage_sidecars and prepare_live -- and
every failure edge on that path returns PERSIST_FAILED without a word. The user
then reads "Pipeline failed. Check repo_path exists and contains source files"
about a repo that indexed flawlessly.
A silent failure that misattributes itself cost a full day of remote-only
bisection: five instrumentation commits in the TEST were needed to even locate
the failing function. That is a product defect independent of the original bug.
Every edge now logs before returning -- destination-not-regular, seal rc,
sidecar-cleanup errno (both branches), and the final rename with stage and
destination paths. The next guard run names the exact edge and value instead of
requiring a sixth probe.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The CLI probe reproduces the empty pipeline (rc=1, same one-line hint), so the
remaining record is the worker's own log -- which the supervisor unlinks on a
clean exit. CBM_PROFILE=1 flips that exact branch to retention
(index.supervisor.profile_log) with no product change, and the probe now prints
every retained log tail from its isolated cache.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The retry with a fresh cache fails identically, so this is path-dependent, not
order-dependent — the daemon-lifetime theory is dead. The MCP result compresses
the pipeline's own diagnostics into one hint string; the documented CLI
entrypoint prints them. Third fresh cache, same repo, output verbatim.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The worker log shows the discriminating shape: the supervisor reaps the worker
CLEAN (exit 0, 2.3s) yet index_repository reports an empty pipeline -- and the
test daemon now serves the UI on fixed port 9749, which main's build does not
do here (no pack beside the test binary keeps its UI off, while the embedded
build auto-enables). Sequential per-case daemons therefore contend on one port,
and Cyrillic is simply the SECOND case in the run order.
An in-process retry against the same repo with a fresh cache separates the two
remaining explanations: a passing retry convicts the previous case's daemon
lifetime (order), a failing retry convicts the path itself.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The previous round narrowed Cyrillic-on-x64 to: correct project name derived,
server responsive, but the pipeline reports 'Check repo_path exists and
contains source files' and only _config.db + logs/ land in the cache. The
supervisor wrote worker logs, so the actual error is already on disk in the
failing job -- print their tails plus the repo listing instead of guessing
across another cycle.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Cyrillic fails deterministically on the x64 runner and passes on the arm64 VM
and on the main-based sibling PR at identical path depth, so neither script nor
path length explains it. The remaining unknown is what index_repository
actually said and whether anything landed in the isolated cache; print both.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Cyrillic case fails deterministically on the GitHub runner (two genuine
attempts, fresh temp roots each time) while latin1/CJK/Greek pass around it and
the SAME commit passes all cases on the real-Windows VM. The guard records an
'error' field for exactly this situation and then never prints it, so the CI log
shows nodes=None and nothing else. Surface it verbatim; the next run carries the
diagnosis instead of a guess.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Same restoration as the pypi wrapper, same reasoning: serialization of
concurrent publishers, killed-publisher reconciliation via locked readiness,
the crash-phase matrix, and multiply-linked-leaf rejection are properties of
the LOCK design, not of how many files publish.
Adaptations for one file: the mid-publication pause hooks the binary rename
itself (the runtime lock is held until publish returns, so the serialization
property is exercised exactly as before); the "partial sidecar publish" crash
phase is gone with the sidecars; backup-member expectations drop to the single
member, including cleanup-interrupted leaving 0 rather than 1; and the
hardlink-rejection guard now targets the binary leaf.
go vet + gofmt clean; full Go suite ok including the restored
TestConcurrentRuntimePublishersAreSerialized,
TestKilledRuntimePublisherIsReconciledByLockedReadiness (5 crash phases),
TestRuntimePublicationCrashHelper and
TestRuntimeReadinessRejectsMultiplyLinkedLeaves.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Deleting these with the multi-file runtime set threw away properties that do not
depend on how many files publish: two wrappers racing to publish the SAME cache
still must serialize, a killed publisher still must be reconciled by locked
readiness rather than trusted, a failed publication still must restore the prior
binary, and a complete foreign winner still must never be deleted.
Restored from main and adapted: the staging fixtures publish one binary instead
of binary+integrations+pack, and the assertions that read sidecar bytes are gone
with the sidecars. Each surviving test keeps its lock/race logic intact
(7/10/5/3 assertions respectively, all still reading published binary bytes).
Revert-checked against production, not just green: disabling
_runtime_backup_target_name makes test_publication_failure_restores... fail and
restoring it passes. pypi suite: 28 tests, OK.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The new contract hit the WSL-alias trap on its first Windows CI run: BASH is a
shell variable, NOT exported, so os.environ.get("BASH", "bash") silently fell
back — and native Windows resolved bare 'bash' to the WSL stub, which fails
every invocation with 'Windows Subsystem for Linux has no installed
distributions' before the extractor runs at all.
This is the exact pattern the venue-parity contract bans; it slipped past that
rule's regex because the literal sits inside a .get() fallback rather than a
bare argv element. Fixed the repo's documented way (df2088a7): the shell wrapper
passes "$BASH" in as argv and Python uses it verbatim.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Ran the format gate before adding --dir rather than after, so the ternary
landed unformatted and lint/lint went red. Reformatted with the pinned Homebrew
LLVM clang-format; the diff is confined to that block and cli.c still compiles.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Deleting this contract with the UI-pack architecture threw away coverage of
machinery that SURVIVED: the extractor still builds the scan-set and association
manifests that check-virustotal.sh reads. Without it the gate could report a
confident green over an incomplete set.
It immediately paid for itself. The extractor was BROKEN by my own edit:
NameError: name 'variant' is not defined (process_tar)
TypeError: add_association() missing 'variant'
Removing the variant from validate_namespace's return left the call sites still
threading it. python3 -m py_compile passes on that -- it is a runtime failure --
and the extractor runs ONLY in the release and dry-run VirusTotal jobs, so
nothing else in the tree or on the 3-OS ladder executes it. It would have
shipped and broken every release verification.
The contract is rewritten rather than adapted: the original was woven through
the pack format (16 archives, 88 associations, per-pack asset math), and a
half-adapted version that passes for the wrong reasons is worse than none. It
now asserts what still exists:
- the exact 8-archive matrix, 32 member associations, nothing dropped
- archive containers never enter the scan set as members
- deduplication is exact: 8 byte-identical LICENSE members collapse to ONE
scan object while 8 distinct binaries stay 8
- manifest metadata agrees with its own rows, and every staged object matches
its recorded SHA-256
- fail-closed on a surplus member, a missing member, and a short matrix
Registered as Step 0l so it runs in every venue, not just when someone
remembers.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows long-path guard failed with:
error: unknown uninstall option: --dir=C:\...\install_Ωμέγα\...\bin
Restoring cli.c from before the externalization dropped --dir from uninstall
while install kept it. That is worse than a test failure: anyone who installed
outside ~/.local/bin had no supported uninstall path at all — uninstall would
look in the default location, find nothing, and leave the real install in place.
Parsing mirrors cbm_cmd_install (--dir=PATH and --dir PATH, both rejecting an
empty value), and the resolved directory now feeds the target path with an
explicit truncation check instead of an unchecked snprintf into a fixed buffer.
Confirms the earlier cascade read: with the extended-path probes fixed,
daemon_stability, hook_augment, cli_non_ascii_arg and ui_drive_listing all went
green on their own. non_ascii_path was the only real failure, and it was hiding
two distinct product regressions behind one red guard.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
test_non_ascii_path.py caught a real product regression I introduced:
[FAIL] long-path existing target was overwritten despite --no
(target existence probe is not extended-path safe)
Restoring cli.c from before the externalization also reverted three probes from
cbm_path_info_utf8() back to plain stat(). On Windows stat() resolves through
the ANSI code page, so an extended-length or non-ASCII target reports "absent" —
and a non-force `install --no` then overwrites the exact bytes the user asked to
keep. Silent data loss, not a cosmetic difference.
That fix rode along in the externalization commit but is orthogonal to it: the
wide-path helpers it depends on (win_utf8.h, compat_fs.c) were kept. Only their
call sites in cli.c were lost.
Two of main's five cbm_path_info_utf8 sites lived in asset code that is
correctly gone; the two restored here are the install target probe and the
uninstall binary probe.
Not yet explained: test_daemon_stability.py also went red in the same run
("daemon did not accept the UI configuration"). Its code path is identical to
main's apart from the intended capability predicate, and the guard that failed
before it completes an install into a long path it should have refused, so this
may be downstream state. Pushing this fix to find out rather than guessing.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pr-smoke was green but test-windows-guards failed, and the shape of the failure
named the cause:
[PASS] non-ascii/latin1_accents nodes=12 edges=21 definitions=5
[FAIL] non-ascii/cyrillic nodes=None edges=None definitions=None
ASCII and latin1 pass, Cyrillic does not -- latin1 survives a Western ANSI code
page and Cyrillic does not. But the product code was never the problem: the
wide-path helpers (win_utf8.h, runtime.c extended paths) were KEPT. I had
reverted this test to a base that predates them, so it lost windows_extended_path()
and the explicit `executable=` argument to subprocess.run. With
lpApplicationName NULL, CreateProcess applies the legacy MAX_PATH limit while
deriving the module name from the command line, so on CI's long guard-bundle
path the process never started and every count came back None.
Same mismatched-base class as the package wrappers: production kept the fix, its
test was reverted past it.
Restored from main and adapted rather than dropped -- it still covers
extended-length publication AND non-ASCII indexing. Only the sidecar
assertions are gone (one self-contained file to publish now), and the probe
uses --version instead of the removed --verify-runtime-assets.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel
