mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-06 17:49:47 +00:00
a35671e962f1f2fb4e94deaf55e71cf05f6b3828
118
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Martin Vogel
|
8fd93a2e08 |
test(smoke): pin the Claude PostToolUse matcher that covers the graph tools
Smoke check 8d required Claude's PostToolUse matcher to be exactly Read. The
drift note widens it on purpose to Read|mcp__.*codebase-memory-mcp__.*, so a
graph tool call can reset the search count, and the macOS PR smoke failed on
it. 8d now requires exactly the new matcher, and besides a Search matcher it
also refuses a catch-all ("*" or empty) matcher, as its comment promised.
Checked against real installs: the previous binary's Read-only settings fail
8d, the new install passes, and the new install plus a catch-all PostToolUse
entry fails.
Refs #509
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
10c7fc3206 |
fix(pkg): remove the archive size caps from the npm, PyPI and Go wrappers
v0.12.0 could not be installed through npm, pnpm, pip or the Go wrapper on any platform (#2657): each wrapper refused an archive member above 256 MiB, and every v0.12.0 binary is 331-335 MB. The v0.11.0 binaries were already about 300 MB, so the Go wrapper, which has had the cap since v0.10.0, was broken before; npm and PyPI gained it in v0.12.0. The archive is verified against the release's checksums.txt before it is unpacked, so a byte cap only ever refused what we ship. - npm, PyPI, Go: delete every byte-size cap (compressed archive, per member, aggregate, checksum manifest, download size). The checksum, the exact member set, path and link refusal, the member count, HTTPS and the --version check stay. - All three honour CBM_DOWNLOAD_URL like install.sh, for testing: https, or plain http to an exact loopback authority with no proxy and no redirect. - Smoke: scripts/smoke-wrappers.sh installs the three wrappers from the checkout against the release fixture and runs the binary each one installed. smoke-test.sh Phase 13w runs it; SMOKE_REQUIRE_WRAPPERS=1 makes it required on the PR smoke lanes and in the release smoke, which gates publishing. Other install paths found broken while checking every channel: - Homebrew: the formula was still at 0.10.3 and installed the dynamic Linux build, which needs glibc 2.38 (Ubuntu 22.04, Debian 12 and RHEL 9 lack it). It now installs 0.12.0 with the static -portable Linux assets and tracks the newest release in the version contract like the other surfaces. - Go: `go install .../pkg/go/cmd/codebase-memory-mcp@vX.Y.Z` never worked (no pkg/go/v* tags, so Go fetched the whole repository, over its 500 MB module limit), and @latest built main's wrapper, whose version constant lags each release. The release now also tags pkg/go/vX.Y.Z, and the wrapper installs its own module version (the constant only for local and untagged builds). - npm: a stable release also moves the `next` dist-tag to itself; `next` still pointed to 0.9.1-rc.1. - Docs: winget and Chocolatey were listed as install channels but are not published; they are removed until they are. Windows arm64 ships and is now listed. Fixes #2657. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
810d677d41 |
test(smoke): follow the #1180 Hermes home and Claude detection
The client-detection fix moved the Windows Hermes home to %LOCALAPPDATA%\hermes and stopped counting a bare ~/.claude directory as a Claude Code install. The smoke script followed only part of the way. Hermes: 8x and 8x-i read the new HERMES_HOME_DIR, but 8x-ii still looked for the pre_llm_call hook in ~/.hermes/config.yaml, and the uninstall absence checks 9m and 9n still named ~/.hermes. On Linux and macOS HERMES_HOME_DIR is ~/.hermes, so the stale paths pointed at the right directory and nothing showed. Only on Windows do the two differ: 8x-ii failed on a hook that install had written to the new home, and 9m and 9n passed without looking at any file install had produced. All three now use HERMES_HOME_DIR. The assertions are unchanged. Claude Code: the fixtures of 9b-2, 9b-3, 9b-8 and both Phase 13 legs created only an empty ~/.claude, which is no longer a Claude Code home. Wherever no claude executable is installed, 9b-2 then finds no .claude.json entry and 13e finds no agent config, and 9b-3, 9b-8 and the install.ps1 leg stop exercising the Claude paths without saying so. The Linux test container reproduces both failures. The Windows job never got that far, because 8x-ii stops it in Phase 8. The fixtures now seed the .claude.json user config through one helper, the same seed the unit tests use. 9b-1b pins the new rule end to end: where the empty-home control shows no Claude Code, a home with only a bare ~/.claude gets no Claude config. A venue that has a claude executable skips it, with the reason at the case. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
2b329fe75d |
fix(cli): client detection for OpenCode, Hermes, Claude (#1167, #1180)
Three detection defects kept `codebase-memory-mcp install` from configuring agents people run, and made it configure one they never installed. OpenCode without a config file or a CLI on PATH (#1167, #1180). OpenCode was detected only from its global config file or directory, OPENCODE_CONFIG_DIR, or an `opencode` found on PATH. It now also counts as installed when its data directory exists: ~/.local/share/opencode on every OS, %USERPROFILE%\.local\share\opencode on Windows (opencode.ai/docs/troubleshooting, "Storage"; the same page names it as the global config location of older installs). The OpenCode CLI and the Desktop app both create it -- the Desktop's bundled opencode-cli server uses the same XDG paths (anomalyco/opencode packages/core/src/global.ts creates the config and data dirs at startup; packages/desktop/src/main/sidecar.ts only redirects XDG_STATE_HOME) -- so detection no longer depends on PATH. The install target ~/.config/opencode/opencode.json(c) -- %USERPROFILE%\.config\opencode on Windows per the same docs page -- is unchanged and right for CLI and Desktop alike. A PATH longer than 4 KB, which hides every CLI from find_in_path(), is handled separately by PR #2381. Hermes home on Windows (#1180). Native Windows Hermes -- the install.ps1 CLI, the MSIX package and the Desktop app -- defaults HERMES_HOME to %LOCALAPPDATA%\hermes and falls back to a legacy ~/.hermes only while that directory is absent (NousResearch/hermes-agent website/docs/user-guide/windows-native.md, "Data layout"; apps/desktop/electron/data-paths.mjs, resolveDesktopHermesHome). cbm probed and wrote ~/.hermes only, so a Desktop install was never detected and a Hermes found on PATH got its config.yaml and skill in a home it never reads. On Windows the home now resolves to <home>/AppData/Local/hermes unless only ~/.hermes exists, using the home-derived AppData convention of the existing Zed, goose, Kilo and VS Code probes; detection, install and uninstall all follow it. HERMES_HOME still wins, and macOS/Linux keep ~/.hermes. Claude Code from a bare ~/.claude (#1180). Any ~/.claude directory counted as a Claude Code install, so a user who only had the folder got hooks, skills and MCP entries for a client they never installed. Detection now needs what Claude Code itself leaves behind: settings.json in the config dir, .claude.json in the home (in CLAUDE_CONFIG_DIR when set), or the claude CLI. Proof on macOS against synthetic HOME/cache/runtime dirs with a minimal PATH (`install --dry-run --skip-binary`, then a config-only install): bare ~/.claude main: Claude-Code fix: none ~/.local/share/opencode only main: none fix: OpenCode In the OpenCode case the fixed install writes the codebase-memory-mcp entry to ~/.config/opencode/opencode.json; main writes nothing. Controls (a launched Desktop layout, a real Claude user) are detected by both. The Windows Hermes home is covered by a platform-parameterised resolver test that runs on every host plus a _WIN32-bound end-to-end detection and plan test. Tests (cli suite, 335 passed): seven new tests; five fail on main three times out of three (the _WIN32-bound Hermes test and the look-alike control are green on macOS by design). Reverting the Claude+Hermes half and the OpenCode data-dir probe separately turns exactly their own tests red. Nine existing Claude fixtures that used a bare ~/.claude as the install marker now write .claude.json; two Hermes tests and smoke-test 8x expect %LOCALAPPDATA%\hermes on Windows. Refs #1167 Fixes #1180 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
  Anton StandrikandClaude Fable 5.1
|
fa138acf44 |
fix(test): arm smoke runtime cleanup at init, retire the daemon first
Review on #2199: cbm_test_runtime_init ran more than a hundred lines before the only EXIT trap, so under `set -e` a failure in the fixture mktemp or its cygpath conversion left the private root behind, and the fixture trap ran smoke_rmtree ahead of the runtime cleanup. Arm `trap 'cbm_test_runtime_cleanup "$BINARY"' EXIT` immediately after a successful init, as soak-test.sh and memlab.sh do. The fixture trap that replaces it now retires the private daemon first and removes the fixtures second, so no earlier cleanup step stands between the exit and the runtime cleanup; on Windows the retirement is also what unblocks the fixture rm. tests/test_smoke_runtime_isolation_contract.sh pins the early trap to the window between the init call and the first fixture; the pin fails against the previous head. Part of #1696. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Anton Standrik <astandrik@yandex-team.ru> |
||
|
Anton Standrik
|
183cd727be |
fix(test): isolate smoke harness daemon runtime
scripts/smoke-test.sh retires the account daemon from seven call sites, but its wrappers sandbox only HOME/TMPDIR/CBM_CACHE_DIR and only CBM_RUNTIME_DIR moves the daemon rendezvous, so every retirement reached the operator's live daemon. Source scripts/test-runtime.sh so every product process runs under a harness-owned runtime and cache, and clean that root up from the EXIT trap. Add tests/test_smoke_runtime_isolation_contract.sh, which fails before this change, and wire it into scripts/test.sh. Part of #1696. Signed-off-by: Anton Standrik <astandrik@yandex-team.ru> |
||
|
Martin Vogel
|
92e7e6a913 |
fix: keep large thread-locals off the static TLS block
Indexing broke on x86-64 Linux: the index worker refused to start and
SIGKILLed its own process group, so every venue reported nothing but
"index worker ended with killed (exit=-1, signal=9)".
glibc places a thread's static TLS block inside the stack allocation it
makes for that thread. This branch's new thread-local caches -- a
512-slot field cache (~24 KB), per-depth tree cursors (~4 KB) and two
parked CBMArenas (~8 KB; an arena carries blocks[256] + block_sizes[256])
-- took static TLS from 47,905 to 93,513 bytes. The 64 KB parent-death
watchdog thread then could not be created at all, and a worker without
process-tree containment correctly refuses to index.
Four layers, because there were four things wrong:
- The big thread-locals move to the heap behind a thread-local POINTER,
allocated once per thread and reused, each with a correct fallback if
allocation fails (the field cache and cursor pool are caches; a miss
is slower, never wrong). Static TLS is back to 52,233 bytes.
- PARENT_WATCHDOG_STACK_SIZE 64 KB -> 256 KB. "The watchdog only polls,
a tiny stack suffices" was true when written and false once TLS
reached ~50 KB, because the TLS block comes out of that same
allocation.
- cbm_thread_create retries with the default stack on EINVAL. A stack
size is a hint about what a thread needs; the platform refusing the
hint is not a reason to fail to create the thread.
- tests/test_thread_stack_tls_contract.sh (Step 5a of scripts/test.sh)
keeps static TLS a quarter clear of the smallest requested stack,
reading that smallest size from the sources so a new small stack
tightens the gate automatically.
Two diagnostics that made this findable at all, and that stay:
smoke-test.sh prints the worker log its failure message names (CI deletes
that sandbox with the job, so the one artifact naming the cause was
unreadable), and worker_containment_unavailable() now reports WHICH of
its conditions failed, with pid/pgrp/ppid.
Verified in the amd64 container, the venue nothing on the local ladder
covers: baseline main indexes 9039 nodes/15067 edges, this branch before
the fix refuses to start the worker, and after it indexes 9039/15067.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
b51411753e | Merge branch 'main' into codex/fix-1689-codex-agents-cleanup | ||
|
Martin Vogel
|
619c683480 |
fix(mcp): settle the lean-output branch on current main
Fix-forward after rebasing onto main (545 commits, 26 conflicted-file resolutions). Where main had shipped a behaviour in the meantime, main wins; the PR keeps its output contract on top of it: - search_code: the scan runs under main's supervised, deadline- and output-bounded runner and feeds the branch's streaming classifier from the runner's output file; the POSIX wrappers map grep's no-match to 0, so every non-zero exit fails closed (no exit-1 special case). - trace_path: strategy/confidence precede args in header and rows (#1542), in both the tree table and the json legs. - cypher: relationship expansion is never capped (a cap falsified aggregates) and unlabeled MATCH scans every candidate; `truncated` is raised only by a real ceiling — a clamped hop range is probed one hop beyond the cap and reports truncated only when a candidate exists there. The plain BFS and the trail BFS both probe one row past max_results so saturation is observable instead of silent. - search_code full mode and the final json use main's U+FFFD sanitizer instead of '?' replacement. - search_graph json keeps an empty `groups` array for semantic-only calls; list_projects json echoes offset/limit next to next_offset and accepts main's include_details spelling for the stats projection. - discovery budget: main added get_file_outline, compare_graphs, manage_adr set_sections and the debug/include_details parameters; their descriptions are trimmed to the lean style and the ceiling moves from 15 to 18 KiB. Tests that asserted a JSON default now request format:"json"; the fail-closed search test uses an unreadable regular file (a missing or non-regular indexed path is deliberately not a scan operand on main); the non-regular fixture gives its real sources File nodes, as every indexed project has; list_projects total counts listable projects (the 0-byte ghost is never a row, so it is not a page either). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
3909e5a0e1 |
feat(mcp): make tool output lean and lossless
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
39c74bdd5c |
test: converge hook tests and smoke on the exec-form registration
The rebase onto main brought the Bash-enabled PreToolUse matcher (Grep|Glob|Bash), which the exec-form tests still counted as Grep|Glob. The Windows lifecycle test seeded the .cmd shell spelling as the converged identity; the shell-free registration converges every owned shell spelling onto one exec-form entry per matcher instead. The test now asserts that, plus their removal on uninstall, and hands cbm_install_agent_configs the '/'-spelled binary path cbm_cmd_install registers (cbm_mkdtemp returns a native '\' temp root on Windows, which uninstall never derives). Smoke 8aq proved Claude's gate registration by grepping the shim name out of settings.json; the exec form carries no shim name, so it now checks the binary-plus-'hook-augment' PreToolUse hook. 9d's shim-name marker would otherwise pass vacuously after uninstall; the 'hook-augment' argument is an owned marker there too. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
be66a2d2c6 |
smoke(8al-node): load the Pi extension with the modules the Pi host provides
The 8al-node probe imports the generated Pi extension in bare Node with an empty node_modules. That was a sound stand-in for the Pi host only while the extension imported nothing but node: builtins; the first package import (`typebox`, Pi's schema library for tool parameters — a real Pi dependency, verified against pi-coding-agent 0.84.4 on #1809) can never resolve there, so the probe fails on a correct extension and would pass a wrong one just as readily. Give the probe what the host gives the extension: a Proxy-backed `typebox` stub in the probe's node_modules (any Type.X(...) returns a descriptor, so the probe never tracks which helpers the emitter uses — it only exercises the tool lifecycle), plus an explicit allowlist: every import specifier in the generated file must be a node: builtin or a package on PI_HOST_PACKAGES, otherwise the probe fails naming the specifier. That turns the guarantee the empty node_modules gave by accident (no surprise dependency) into an asserted contract. Verified locally by running only the 8al-node block against extensions generated by two binaries: main@5802ccdd extension, old probe: OK; new probe: OK #1809 extension, old probe: ERR_MODULE_NOT_FOUND typebox (the CI red) #1809 extension, new probe: OK #1809 extension + an injected 'lodash' import, new probe: FAIL 8al-node: ... packages the Pi host does not provide: lodash Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
astandrik
|
4c1b23477b |
fix(codex): keep global activation pointer
Signed-off-by: astandrik <astandrik@yandex-team.ru> |
||
|
astandrik
|
4cafe8eb22 |
fix(codex): stop installing global AGENTS guidance
Signed-off-by: astandrik <astandrik@yandex-team.ru> |
||
|
Martin Vogel
|
51c48e2093 |
Merge pull request #1858 from DeusData/fix/issue-1834-pi-stdin-transport
fix(pi): send tool arguments over stdin |
||
|
Martin Vogel
|
d706c33ac2 |
Merge pull request #1337 from abendrothj/bash-tool-hook-augment
feat(hook-augment): augment Bash tool searches via graph |
||
|
Martin Vogel
|
ee32816e3c |
fix(pi): send tool arguments over stdin
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
72ee88054f |
feat(cli): add Grok Build as an install target
Grok Build (xAI's grok CLI/TUI) already reached codebase-memory-mcp through its Claude/Cursor compat layer, but a Grok-only user got nothing. Detect $GROK_HOME (default ~/.grok) or a grok CLI and install: - an owned [mcp_servers.codebase-memory-mcp] managed block in config.toml; a same-name foreign table is left byte-identical because a second header would make Grok reject the whole file, while the known owned shape written by `grok mcp add` is adopted. No env forwarding: grok 1.0.5 passes the full parent environment to stdio servers (verified), unlike Codex (#1562). - an owned rules file under $GROK_HOME/rules/ (always scanned, all projects) - the shared skill under $GROK_HOME/skills/ - Scout/Verify/Auditor agents with named-server mcpInheritance and the tier allowlist spelled out as the server__tool ids Grok's search_tool/use_tool dispatcher accepts (inheritance filters per server, never per tool) - hooks withheld: Grok's passive events discard stdout and PreToolUse honors only deny/rewrite decisions, so there is no context-output contract. Verified against grok 1.0.5: inspect discovers the server, agents, skill and rules; mcp doctor handshakes and discovers 15 tools. --clients=grok token, README/npm/llms/landing-page matrices (43 -> 44 surfaces), and smoke coverage. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
 Jake Abendroth
|
36b7b18b4d |
test(smoke): expect Bash Claude hook matcher
Align the exact Claude PreToolUse smoke contract and documentation with the Bash-enabled search matcher. Signed-off-by: Jake Abendroth <contact@jakea.net> |
||
|
Martin Vogel
|
8eabe191d2 |
fix(install,daemon): unbreak npx clients, group-writable homes, and legacy updaters
Five field reports in the 24 hours after v0.10.0 all pointed at the same thing: gates that were right in principle refused real, ordinary setups, and then failed to say why. Per the consolidated strictness decision, each gate keeps the protection that matters and drops the part that was refusing legitimate users — and every refusal now names what it refused and how to proceed. **Daemon image gate: npx and every ephemeral install path (#1539, #1383).** The admission check treated "the peer's image hashes differently" and "the peer's image cannot be examined at all" as one failure. The second is what `npx codebase-memory-mcp` always produces (ephemeral cache path, unfingerprintable), so every npx-invoked client was rejected — and, because the client never reported it, agents saw a transport that closed mid-handshake with zero bytes on stdout. Reported by @wassolles with the admission path already read and the fix space mapped. An unverifiable image is now admitted: the rendezvous HELLO immediately above it has already proven semantic version, build fingerprint, and protocol/store/ feature ABI, and the image check was trading that real proof for an unavailable one. It logs daemon.client_image_unverifiable_admitted so the weaker check is never invisible. A fingerprint MISMATCH — the tamper case the gate exists for — still rejects hard. Separate test seams keep the two modes testable apart. **Client bootstrap failures are no longer silent (#1539).** An MCP client that cannot reach the daemon now emits a JSON-RPC error on stdout naming the reason, plus the same text on stderr. Previously the reason sat in bootstrap_result.message and the process exited having written nothing at all. **POSIX activation: group-writable ancestors (#1535, discussion #1526).** activation_directory_secure required no group or other write bit on the install directory AND every ancestor. WSL2 ships ~ and ~/.local at 0775, as do several distro skeletons and any site using a shared primary group, so install.sh failed for a large fraction of Linux users — reporting a policy refusal as "activation transaction I/O failed", which sent reporters after disk errors and filesystem types. Root-caused by @AmirF194 in a clean ubuntu container; @shochdoerfer and @iandol confirmed independently. World-writable ancestors are still refused (any local user could swap a path component mid-transaction). Group-writable ancestors are now warned about and admitted. The LEAF directory stays strictly owner-private — that is where the binary is published, and group write there would let another account replace the executable between validation and exec. Refusals now name the directory, its mode, and which rule refused. **The obsolete ui/standard chooser (#1538, from discussion #1526).** v0.10.0 consolidated to one archive per platform with the UI always embedded, but `update` still offered a variant choice: "ui" could only 404, and "standard" quietly WAS the UI build. Reported by @iandol upgrading 0.9.0 -> 0.10.0. The chooser, its --standard/--ui flags, and the ui- URL plumbing are removed, along with the CBM_VARIANT=ui remnant in the npm installer. Already-released 0.9.x binaries cannot be fixed retroactively, so the release workflow now publishes byte-identical ui-*-named alias assets — their updaters work again with no user action. The aliases are uploaded AFTER the VirusTotal gate: they are the same bytes as archives it already cleared, and uploading them earlier would duplicate every object in the scan set and the provenance manifest. **macOS install noise and attribution (#1537).** install.sh silenced the "No such xattr: com.apple.quarantine" line, which is what happens when a curl-downloaded archive carries no quarantine attribute — harmless, and it became the title of a bug report about an unrelated failure. The session-stop refusal now points at `daemon status` to list the client processes actually holding the daemon, instead of asserting sessions exist and leaving the reader to guess. Reported by @listepo. **Riders.** hatchling is pinned in pkg/pypi (an unpinned backend resolved fresh inside `python -m build` is what emitted Metadata-Version 2.5 and broke the v0.10.1 publish); SECURITY.md's supported-versions table moves to 0.10.x. Tests: separate seams for unverifiable vs mismatched peer images with a test per outcome; activation refusal must name directory + mode + rule; a group-writable ancestor must stage successfully. The update tests drop the flag that no longer exists. Verified against each reporter's environment shape. **Open security alerts (all three, OSSF Scorecard).** - HIGH, binary artifact: an 8.8 MB compiled Go ELF wrapper had been committed at pkg/go/codebase-memory-mcp by accident. Removed, and both it and its .exe sibling are gitignored so `go build` in that directory cannot repeat it. - HIGH, GHSA-2v37-7h3g-55p8: nanoid < 3.3.17 loops forever when a custom generator is called with size 0. It reaches us transitively (postcss -> vite), so it is pinned through the existing graph-ui overrides block rather than promoted to a direct dependency; the lockfile resolves 3.3.18. - MEDIUM, unpinned pip command: the publish step installed build/twine by version only, leaving the whole transitive graph resolved at run time. pkg/pypi/requirements-publish.txt now hash-pins the complete toolchain (316 hashes), generated on a linux/amd64 python:3.12 image so the wheels match what ubuntu-latest resolves, and the step runs pip with --require-hashes. Verified by installing from it in that same image. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
f59d24fd10 |
fix(mcp): make every tool reply client-usable — structuredContent, pipelining, config get (#1522)
Three regressions shipped in 0.10.0 share one failure shape: an empty result
with a success status, indistinguishable from "nothing found" for the LLM
clients that are cbm's primary consumers.
1) structuredContent {} on the whole tree-format surface (#1522 bug 1).
#1488 replaced the duplicated payload with an EMPTY structuredContent
object while every tool still declared a blanket permissive outputSchema.
Spec-honoring clients (Claude Code among them) treat structuredContent as
THE result when a schema is declared, so search_graph, trace_path,
query_graph, get_architecture, search_code, and detect_changes all rendered
as literally "{}" on their DEFAULT format, on every platform. The corrected
contract: no tool declares an outputSchema (tool output is
format-parameter-polymorphic — no static schema is truthful), JSON-object
payloads keep their parsed structuredContent, errors keep
structuredContent.error, and text-shaped payloads carry NO structuredContent
key at all — which also preserves #1375's no-duplication win.
2) Frontend queue overflow killed the session (found by the #1522 sweep).
Any 7+ requests pipelined in one stdin burst — an agent issuing parallel
tool calls does exactly this — overflowed the 8-frame frontend queue, which
failed the whole session: rc=1 with ZERO bytes of output, every buffered
response lost. A full queue is now backpressure: the stdin reader blocks
until the worker drains (bounded by the same stop/fail flags every teardown
path already sets); only a single frame larger than the entire 12 MiB byte
budget — which could never be admitted — remains a hard failure.
3) config get printed "" with exit 0 for every unset and every unknown key
(#1522 bug 2). list printed stored-or-DEFAULT while get printed
stored-or-EMPTY, and no subcommand validated key names, so a typo was
indistinguishable from a correctly-read setting. One config-key table now
drives help, list, get, set, and reset: get prints the stored value or the
key's real default (the same fallback the runtime readers use), and unknown
keys error with exit 1 on get, set, and reset alike.
Tests — each RED on the pre-fix tree and RED again on revert:
* test_mcp.c: text results carry no structuredContent key; tools/list
declares no outputSchema; the tool-table guard now binds all three
branches (absent / parsed-object / error) for every registered tool.
* test_daemon_frontend.c: the over-capacity contract flips from
"session fails" to "backpressure without loss" — the held first request
plus all 32 over-capacity frames are answered and the run closes cleanly.
* test_cli.c: the config command contract — defaults, round-trip, reset,
and unknown-key rejection on all three subcommands.
* smoke-test.sh: Phase 3z rewritten to the corrected structuredContent
contract, new Phase 3z1 (default-format replies usable in schema-honoring
clients, no outputSchema advertised), 3z2 (24 pipelined calls all
answered), 3z3 (config defaults + unknown-key rejection) — all asserted
against the SHIPPED artifact, where #1488's smoke phase previously pinned
the empty-object behavior as correct.
Verified end-to-end on the locally built production binary: all 15 tools
declare no schema; text tools return ABSENT structuredContent, object tools
populated, error envelopes intact; 7/24/64-deep pipelined bursts all answered
with rc=0; config get prints real defaults and exits 1 on unknown keys.
hook_augment (structuredContent.projects) and index_resilience
(structuredContent.status) consume object payloads and are unaffected.
Fixes #1522.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
39e02544cc |
fix(smoke): UI presence is the caller's claim, not every lane's assumption
Making REQUIRE_UI unconditional broke all three pr-smoke legs: FAIL 15a: SMOKE_REQUIRE_UI=1 but this binary serves no embedded UI assets The PR lane runs scripts/build.sh WITHOUT --with-ui on purpose, so an npm frontend build does not land on every product PR. Demanding embedded assets there asserts a property that lane deliberately does not produce. The guard still exists where it means something: scripts/ci/smoke-artifact.sh builds --with-ui, packages the real archive and smokes the EXTRACTED result, so it exports SMOKE_REQUIRE_UI=1 and a frontend-less binary fails there. That is the lane whose whole job is release fidelity. Deliberately NOT fixed by adding --with-ui to pr.yml: that would put `npm ci && npm run build` on three runners for every product PR to re-prove something the dry-run and release lanes already gate. PR CI cost is not free and this branch should not quietly raise it. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
a4336dc40a |
feat(release): ship one archive set and tolerate a single Microsoft !ml verdict
Completes the collapse to a single shipped composition and replaces the
zero-tolerance VirusTotal gate with a narrow, disclosed policy.
Packaging and installers
- package-release.sh loses --variant; archives are codebase-memory-mcp-<os>-<arch>
with exactly four members. install.sh/install.ps1 lose --ui/--standard.
- The extractor drops CBMUIPK pack parsing and --archive-scope; its scan-set and
association manifests (which the gate depends on) are unchanged otherwise.
- npm/PyPI/Go wrappers: the runtime "set" is one file again. The Windows lock
and race fixes from #1495/#1496 are kept; only multi-file set membership goes.
This also fixes `pip install` on Windows, which rejected the fifth archive
member against a hardcoded four-name allowlist.
- The wrappers' post-download probe moves from --verify-runtime-assets (removed)
to --version, which proves the same thing: the binary executes.
VirusTotal gate
- Exactly ONE detection is tolerated, and only when the engine is Microsoft AND
the label ends in `!ml`. Two or more engines, any non-`!ml` label, any other
vendor, any suspicious verdict and every infrastructure error still block.
- A tolerated object prints TOLERATED:, never OK:, and its counts are recorded
in vt-results.tsv exactly as a blocked one would be.
- append-vt-notes.sh mirrors the policy. It previously hard-failed on any
malicious count, so loosening only the gate would have passed the scan and
then died at note publication. The notes now DISCLOSE a tolerated detection
and link to SECURITY.md rather than claiming "0 malicious" for everything.
Rationale for the tolerance is in the gate itself: the verdict is not a property
of our bytes. It inverts across architectures and link modes, moves between
sibling artifacts of one build, and lands in different variant buckets for the
same source. The same `!ml` family hits llama.cpp, GitHub's own `gh`, Microsoft's
own Go toolchain and Anthropic's Claude installer.
The zero-tolerance contract becomes test_vt_gate_policy_contract.sh, asserting
the full matrix: 1x Microsoft !ml passes and reports TOLERATED; a Microsoft
signature label, a non-Microsoft engine, two engines, a suspicious verdict and
every malformed-response case still block. Its tripwire is narrowed to the
reverted endpoint-verification mechanism rather than the words "false positive",
so it no longer fires on a deliberate in-gate policy branch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
d58afe562d |
revert(release): re-embed runtime assets into the single shipped binary
Externalizing the integration templates (#1492/#1493) and the UI bundle
(#1501/#1503) was done to reduce the Microsoft `Wacatac.B!ml` surface. It did
not work: across dry runs the flagged artifact count stayed at ~3 and the
detections merely moved between artifacts.
Dissection of run 31286803592 shows there is no structural cause to fix. The
verdicts split across every axis at once — linux-amd64 (dynamic) flagged while
linux-amd64-portable (static) is clean, but linux-arm64 (dynamic) clean while
linux-arm64-portable (static) is flagged. The two macOS binaries have identical
segment structure and split clean/flagged. Siblings from one build landed in
different variant buckets (.B vs .C). Entropy is low everywhere
(code_vectors.bin 4.166, grammar tables 3.464 bits/byte, against 7.5-8.0 for
packed payloads), so the packed-payload hypothesis is excluded too.
So the complexity bought nothing, and installation goes back to being
self-contained: one binary that carries its own UI and agent integration
templates, with no adjacent data file that has to resolve before `install`
works. Only the UI-capable composition ships from now on, under the historical
unsuffixed archive name.
Removed: src/ui/asset_pack.{c,h}, asset_pack_stub.c, asset_manifest_stub.c,
scripts/pack-ui-assets.mjs, src/cli/integration_assets.{c,h},
assets/cbm-integrations.json, scripts/gen-integrations-hash.sh, the
--verify-runtime-assets probe (nothing adjacent left to verify), and the
composition gates A6/A7 whose property is now deliberately inverted.
Restored: scripts/embed-frontend.sh, src/ui/embedded_{assets.h,stub.c}, the
compiled-in hook/adapter template bodies, and the embed/EMBED_OBJS build path.
Kept from the reverted commits, re-applied by hand where a wholesale file
restore would have dropped them:
- cbm_module_path_utf8() in both self-path sites. GetModuleFileNameA renders
through the ANSI code page and mangles non-ASCII install paths.
- the /__cbm/ui-readiness HMAC proof, secure_random and cbm_hmac_sha256, so
`daemon start --open` still waits for a genuine CBM listener.
- X-Content-Type-Options: nosniff on served assets.
- the MinGW noexecstack gate, -lbcrypt, and the cppcheck/zip CI fixes.
Archives are now codebase-memory-mcp-<os>-<arch>[-portable] with exactly four
members (binary, LICENSE, installer, THIRD_PARTY_NOTICES.md). That restores the
names every static package manifest already points at — aur, chocolatey,
homebrew, scoop, winget and glama were all broken by the -ui- rename.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
8018561cfe |
fix(release): externalize runtime assets and harden VT verification
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
365841304c |
fix(install,smoke): carry cbm-integrations.json wherever the binary goes
pr-smoke failed on all three platforms: the binary's install/uninstall render templates from the integration asset, and every place that stages the binary without the asset made those operations fail closed with "integration assets missing". The asset must travel with the binary in EVERY layout, not just the release archive. Five staging paths were missing it: - scripts/build.sh — stage cbm-integrations.json next to the freshly built binary, so `build/c/codebase-memory-mcp install` works straight out of a build tree (dev, and the base for the smoke fixture). - scripts/smoke-local.sh — include the asset in the fixture tarball and its required-sidecar check. Member set and ORDER now mirror package-release.sh exactly (binary, cbm-integrations.json, LICENSE, install.sh, notices); the fixture was smoking an archive layout we never actually ship. - install.sh / install.ps1 — after installing the binary, copy the asset beside it in the install dir. `install` already publishes a verified copy to ~/.cbm/assets/<version>/, but a later install/uninstall run from the install dir resolves the asset NEXT TO THE BINARY first, so without the adjacent copy that lookup misses and a re-install or uninstall fails on a machine that just installed successfully. Best-effort atomic rename, same shape as the existing install.sh/ps1 sidecar copy. install.ps1 stays pure ASCII. - scripts/smoke-test.sh Phase 14 — this phase hand-stages the binary into a fresh HOME without going through install, so nothing populates ~/.cbm/assets; stage the asset next to each staged copy so the uninstall it drives resolves. Verified: scripts/smoke-local.sh on the standard binary now runs clean through all 16 phases — zero "integration assets missing" / hook_script_uninstall / agent-cleanup-failed lines (was 5). windows-bundle, smoke-fixture, exec-bit and no-embedded-scripts contracts pass; install.ps1 has zero non-ASCII bytes. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
f27f0a2a6c |
perf(mcp): stop repeating the payload in structuredContent (#1375)
Every non-JSON tool result shipped its payload twice: content[0].text, and an
identical copy as structuredContent {"text": <entire payload>}. Measured on a
20k-node query_graph, the reply was 2.05x the payload it carried. Half of every
large answer was redundant bytes — half the 10 MiB transport budget, and double
the tokens billed to every LLM caller on every call.
Measured, same query, same fixture:
before 8,529,990 bytes content.text 4,154,932 structuredContent.text SAME
after 4,265,047 bytes content.text 4,154,932 structuredContent {}
Exactly 50.0% smaller with content.text byte-identical: the payload is fully
delivered, only the second copy is gone. The 10 MiB ceiling now also admits
roughly twice the rows before #1375's limit applies.
Nothing is lost. structuredContent exists to carry STRUCTURE, and a string
re-wrapped in a one-key object has none — a client reading
structuredContent.text learned exactly what content[0].text already told it. The
empty object still satisfies the declared outputSchema, which is
{"type":"object","additionalProperties":true} and never required a text field.
Two cases are deliberately NOT changed:
* JSON payloads. structuredContent stays the PARSED object. That is the
spec's structured+serialized pattern rather than waste, and it is what our
own hook-augment consumes (structuredContent.projects from list_projects).
* Errors. structuredContent.error is kept: bounded, small, and the only
machine-readable form of a failure a client gets.
Guarded at both levels, because the defect was invisible per-tool — each result
looked reasonable alone, and only measuring the wire showed half of it was
redundant:
* tests/test_mcp.c enumerates the TOOL TABLE itself, so a new tool is covered
the moment it is registered, with no test edit. A guard pinned to
query_graph would not have caught search_graph, and would not catch whatever
is added next. It fails if no tool produced a non-JSON payload, so it cannot
report a green it never earned — which it did on the first attempt, catching
that {} args make every tool error out and assert nothing.
* scripts/smoke-test.sh Phase 3z asserts the same property on the SHIPPED
binary. This is a wire-format contract: what a real client receives from the
real artifact is not something a from-source test can prove.
Both revert-checked: restoring the duplication reddens the unit guard at the
strcmp, the plain-text guard, the search_graph expectation, and the smoke phase
(which names search_graph and get_architecture).
tests/test_mcp.c expectations that pinned the old duplicating shape are updated
rather than relaxed — the format changed on purpose.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
452372eec5 |
fix(mem): route ordinary malloc through mimalloc on Linux (#1360)
The comment above MIMALLOC_OVERRIDE_DEFINE claimed Unix "relies on static-link-order override". That was never true. mimalloc emits strong malloc/free definitions only when MI_MALLOC_OVERRIDE is set — the define its source gates alloc-override.c on — and the build set it for MinGW only. MI_OVERRIDE, which the build does set everywhere, is this project's own prod/test marker that mimalloc never reads. With the override body compiled out there were no strong symbols for link order to prefer, so ordinary malloc went to libc. Measured, not inferred. The shipped v0.9.1-rc.1 artifacts report 0/6 allocator-owned size classes on linux-arm64 glibc AND musl-static, and the same A/B on ubuntu-arm64 here: main warn mem.allocator.not_owned owned_classes=0/6 with this fix info mem.allocator.owned classes=all Every purge/reclaim option cbm_mem_init sets was therefore inert on Linux, applying only to the bound sqlite/tree-sitter populations — the same class of defect as #581, where committed memory ratcheted for months because nothing asserted the wiring on a real artifact. macOS stays off deliberately and permanently: enabling the override there compiles alloc-override.c's forwarding definitions, and under the two-level namespace this binary's free becomes mi_free while system libraries keep allocating from the system allocator, so the first pointer crossing that boundary aborts with "mi_free: invalid pointer". ELF's flat namespace has no such split, which is why Linux can have this and macOS cannot. Three parts: * Makefile.cbm switches MI_MALLOC_OVERRIDE and a new CBM_MEM_GLOBAL_OVERRIDE together in one place, for MinGW and Linux. The latter tells our own sources what to expect and goes into CFLAGS_PROD only, because MIMALLOC_CFLAGS_TEST builds mimalloc with -DMI_OVERRIDE=0 and no override define: a test binary has no global override BY CONSTRUCTION and must never be told to expect one. Deriving that expectation from the platform instead would make every Linux test run warn about a correctly configured build. * mem.c splits the startup audit by what the build actually asked for. Where no override was requested, ordinary malloc reaching libc is the design, so it reports the measured ownership at INFO and names what IS bound. A warning that fires on every run of a correct build is not a tripwire, it is noise — it trains readers to ignore the one line that catches #581. The genuine warning still fires wherever an override was requested and did not take. * smoke-test.sh Phase 1b pins the SHIPPED artifact's wiring per platform and fails in BOTH directions: Windows/Linux must own all classes, macOS must not. This cannot live in a unit test — a from-source test build never has the override — which is exactly why the defect survived so long. Verified on ubuntu-arm64: prod owns all classes; the test build reports bound-populations-only at INFO (0/6) and the mem suite stays green; smoke Phase 1b passes, and fails with the expected message when the same tree is built with the override forced off. On macOS: prod reports bound_populations_only at INFO, smoke Phase 1b passes, mem suite 51/51. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
afd67c51b0 |
refactor(cli): install scripts own updating; the binary carries no URL
Completes the move started on Windows. `update` prints the install script's
command and exits 0 on every platform; the download/extract/chmod/exec sequence
is compiled out of release builds entirely, and the MCP background thread that
ran cbm_popen("curl ... api.github.com ...") on first tool call is gone.
The installers now place themselves beside the binary, which closes the last
gap: `update` referenced a raw.githubusercontent URL purely because an
install.sh-installed user had no local copy to point at. Two details make that
safe.
The source is the install script from the archive we JUST checksum-verified,
never "$0" -- which does not exist under `curl | bash`, and would pin the user
to the OLD installer forever. Since the script ships inside the verified
archive, it inherits that verification instead of needing its own.
And it is published by atomic rename, never written over the live path. Bash
reads a script incrementally by byte offset, so overwriting the file it is
executing continues reading NEW bytes at the OLD offset: silent, bizarre
corruption. cp to a temp name then mv -f swaps the directory entry while the
running shell keeps its original inode. Windows follows the same rule via
Copy-Item + Move-Item even though PowerShell parses up front.
Deliberately NOT done: spawning the new installer to delete and replace its
predecessor. Fetch remote code -> drop to disk -> spawn -> self-delete is a
textbook stager, and self-deletion is among the most heavily weighted
heuristics there is. It also gains nothing -- `"$DLBIN" install` already runs
the NEW release's install logic, because the binary owns the install.
Net: zero install or download URLs remain in the shipped binary, and the
allow-list entry is retired with it. Both scripts scan CLEAN (0/61) with the
self-placement code in them.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
9ecabe0a6f |
refactor(cli): move self-update out of the product on every platform
Windows already handed updating to install.ps1, because a running .exe cannot
replace its own image. The remaining platforms now do the same, for a different
and better reason.
An in-process updater is structurally a downloader: it fetches a remote archive,
verifies it, extracts it, marks the result executable and runs it. That composite
lived in every shipped artifact -- along with the release URLs it fetched -- to
serve a command most users run a handful of times. The install scripts already
do all of it, are idempotent (so re-running one IS the update), and run while cbm
is not running.
`update` now prints the exact command for the running platform and exits 0:
unix bash "<dir>/install.sh" (or the curl one-liner when not adjacent)
windows powershell -ExecutionPolicy Bypass -File "<dir>\install.ps1"
Both scripts ship inside their release archive, so the printed path resolves
next to the binary. Flags are still parsed and validated, so `update --dry-run
--ui` keeps rejecting typos rather than silently accepting them.
The updater itself is EXCLUDED from release builds rather than left to dead-code
elimination: build_update_url, download_verify_install, extract_and_install_binary,
checksum fetch, detect_os/detect_arch and the cli_download_* helpers now compile
only under CBM_CLI_ENABLE_TEST_API, which release builds do not define. The C
suite still covers the flow through the activation test seam. Verified against
the release binary: `releases/latest/download` is gone entirely.
Also removes the MCP background update check. It ran on the first tool call and
did:
cbm_popen("curl -sf --max-time 5 -H 'Accept: application/vnd.github+json' "
"'https://api.github.com/repos/.../releases/latest' 2>/dev/null")
An embedded shell command plus a startup network callout, in every session, to
tell users a newer version exists. It shelled out, depended on curl being
installed, and phoned home from every agent session. The install scripts report
versions; this did not need to be in the server.
Smoke follows the same collapse. Phases 6c and 14 asserted a Windows-specific
handoff and a POSIX in-process replacement; they now assert one platform-neutral
contract selected by UPDATE_SCRIPT, and 14a's byte-identity check compares the
driver against ITSELF before and after -- comparing against "$BINARY" passed on
Windows only because Windows has no signing step, while the POSIX fixture
ad-hoc re-signs its copy and so differed before `update` ever ran.
The Linux glibc guard is RELOCATED, not retired. The standard linux asset links
glibc 2.38+ and breaks Debian 11, RHEL 8 and Ubuntu 20.04, so the installer must
fetch the static "-portable" build. That constraint moved from the binary to
install.sh, and the smoke now guards it where the behaviour lives.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
0a22fb14df |
fix(windows): absorb transient file locks when retiring a running image
Release run 30385988475 lost three Windows smoke jobs to one phenomenon on two
different surfaces. A Windows file that was just written or just executed can
refuse deletion and rename for a moment while a scanner reads it or while the OS
finishes reaping a child. Both conditions clear on their own within moments.
It reads as nondeterminism because it is timing: windows-2025 passed both amd64
variants while windows-latest -- the SAME image, windows-2025-vs2026 -- failed
both, and windows-11-arm failed too.
PRODUCT: uninstall abandoned a live installation.
With one binary per platform, uninstall must remove the very image it is
executing, and on Windows a running image can only be renamed, never
overwritten. That rename got a single attempt, so a transient sharing violation
surfaced to the user as
error: failed to remove ...; completed configuration/index cleanup may remain
with the install left half-torn-down. install.ps1 already retries this exact
operation ten times for this exact reason; the C path now does the same, and
only for sharing/access/lock violations, so a file that is genuinely held still
fails closed instead of spinning.
Proven with a failure-injection seam rather than by racing a real scanner:
3 injected failures must still succeed, 64 must still fail AND leave the target
in place. Deterministic, so it does not need CI to learn whether it works.
HARNESS: the smoke killed itself silently.
Phases that install a binary then `rm -rf` the directory hit the same lock. Under
`set -euo pipefail` that returns non-zero and ends the run WITHOUT printing
anything -- the logs jump straight from "OK 13h" to job cleanup with no FAIL
line, which is what made this expensive to read.
None of those 16 removals is an assertion; they are fixture cleanup, and
_smoke.yml already states the principle ("a temp dir on an ephemeral runner must
never fail the job") that smoke-test.sh did not honour. They now go through
smoke_rmtree, which retries so the disk is actually reclaimed -- these fixtures
hold ~300 MB binaries and runners are disk-tight -- then warns and continues.
Verified: Windows VM full smoke ALL PASSED (previously died after 13h), macOS
full smoke ALL PASSED, activation_transaction + cli 259 passed on Windows, all
10 shell contracts pass, lint-ci clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
2740208f40 |
fix(cli): report a failed install --dry-run plan check in the exit status
The Windows smoke guards that an invalid PATH seam must fail closed rather than
silently falling back to the live HKCU\Environment\Path. That guard went green
in a way that meant nothing: the seam still refused, and install still printed
"PATH configuration failed", but the process exited 0.
Windows release builds used to compile a separate managed-install path. With one
binary per platform they compile the shared cbm_cmd_install, which since
|
||
|
Martin Vogel
|
a54ea95719 |
fix(windows): ship one binary — remove the launcher stub flagged as a dropper
Windows shipped a PAIR: a small permanent launcher (codebase-memory-mcp.exe)
plus the real product binary (codebase-memory-mcp.payload.exe). The launcher
existed for exactly one reason — a running .exe cannot replace its own image
on Windows, so an in-process self-update needs a second resident binary to do
the swap.
That stub is statically indistinguishable from a dropper: a small, unsigned,
zero-prevalence PE whose whole job is verify-and-execute another binary.
Defender's ML scored it Trojan:Win32/Wacatac.B!ml and blocked the v0.9.1-rc.1
release at the VirusTotal gate. It is not fixable in our code on x64 —
bcrypt-free, stripped, VERSIONINFO'd, minimal-resource and even
resource-FREE builds on CI's own MSYS2 CLANG64 toolchain were all flagged,
while the product binary scans clean on every platform.
So remove the stub and move self-update OUT of the process into install.ps1,
which runs while cbm is NOT running: Windows' image lock only blocks a
process from replacing ITSELF. now prints the exact PowerShell
command (with the Unblock-File hint for Mark-of-the-Web); install.ps1 is
idempotent, so re-running it IS the update — it stops the daemon, renames the
running binary aside (the one mutation Windows permits on a running image),
publishes the new one, and sweeps retired copies.
Windows now matches Linux and macOS: ONE binary per platform.
* packaging, install.ps1, npm and PyPI wrappers all carry a single binary
* the launcher/payload ABI contract and ~2500 lines of stub state machinery
are deleted
* every daemon start, CLI call and hook fire loses a process spawn, a named
pipe handshake and an stdio relay
* test_windows_bundle_contract.sh is rewritten as an INVERTED contract: it
now asserts no shipped surface can reintroduce a launcher/payload pair,
and that install.ps1 retires the running binary before publishing
Verified: VirusTotal 0/67 on the packaged binary and 0/58 on install.ps1 (no
certificate involved); macOS and Linux full suites green; Windows guards all
green including the new update-handoff contract; npm 10/10; PyPI 3/3.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
6d8b39466b |
fix(smoke): Phase 15 probes the UI a fresh profile can actually serve
Every ui-variant smoke job failed on every platform in dry run 30226483380 — and it turns out a ui-variant Phase 15 had never legitimately passed anywhere: the old probe (a) never passed --ui=true, but the HTTP UI is a persisted DEFAULT-OFF setting, so a fresh profile (every runner, every smoke HOME) can never serve; (b) fed stdin an empty file, but the UI does not pin the process — stdio EOF ends it cleanly (rc=0) before any poll sees it serve; (c) gave a 10s window to a server measured to bind ~6s after launch on a FAST host; (d) POSTed an MCP initialize at /rpc, which speaks the UI's own narrow query protocol, not MCP. The rebuilt probe mirrors the drive-listing guard's proven invocation (--ui=true --port=N equals-form, held-open stdin), waits 30s, and verifies the API surface with the stateless GET /api/ui-config. Validated red->green against the release-shaped artifact: the full darwin ui artifact lane now passes 15a/15b legitimately. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
ad713bce93 |
fix(mcp): guarantee valid UTF-8 in tree-format output
Dry run 30206811293: macos-15-intel release smoke failed B3 with a visibly correct 'semantic: 50' table — BSD grep returns no-match for the ENTIRE output when any line carries a NUL, control byte, or invalid UTF-8 (verified: printf poisoned input -> grep -qE fails), so one bad byte in one row silently unmatches every anchored check. Close the class at the emission choke point: control bytes and invalid UTF-8 force the quoted form; quoting escapes controls as \u00XX and replaces invalid sequences with U+FFFD (RFC 3629 validation, NUL-safe). Tool output is valid UTF-8 by construction. Unit test proven RED-on-revert (raw bytes emitted) and green with the fix. The smoke's B3 failure path now also dumps od -c bytes so a future red names the exact byte. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
3fc93d12da |
ci: unify smoke/soak/test venues onto canonical leg entries
The staged venue-unification bundle: _smoke.yml onto the shared wrappers with an extracted-artifact input (CBM_SMOKE_ARTIFACT_DIR), soak-legs.sh as the one canonical soak entry (quick + query-leak legs across all venues), protected per-user TEMP roots shared across venues, clean-disk preflights before every VM/Docker run, win.sh and run.sh routed through the canonical scripts, soak.yml retired, and the venue-parity contract (whitelist walker: workflows may provision or call canonical entries, nothing else). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
62821375c4 |
fix: harden daemon release paths and verification
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
f7a8e373f4 |
fix(qa): make crash and tolerance gates fail loudly
The fuzz harness swallowed the target's exit status through '|| true', so a SIGSEGV counted as a pass; the status now propagates (a planted crash yields 139) and payloads derive from a logged, replayable seed, with a missing python3 failing the gate instead of generating zero mutations. The Windows guard runner classified unknown exit codes as skips; the contract is now explicit (0 green, 1 red, 2 precondition-skip, anything else a failure) and an all-skip run fails as verifying nothing. The smoke suite gains crash-class detection on the phase-9b tolerance paths and phase 11 kill handling (rc >= 128 or a missing jsonrpc banner fails), a free-port pick plus readiness poll for the UI phase, and a phase-15b failure that actually exits nonzero. The soak reader resynchronizes on late JSON-RPC responses by draining to the matching id, and the analysis fails when fewer than 60 percent of snapshot attempts produced rows — a vacuous analysis previously passed. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
e4400b6f49 |
fix(smoke): retire fixture daemons before every install-fixture cleanup
Phase 9b's fixture installs spawn ephemeral daemons whose mapped generation backings pin files in the fixture HOME; the cleanup rm then races the daemon's asynchronous drain and loses nondeterministically on Windows (Device or resource busy on a generations backing — the same class the final-phase daemon retirement already fixed for the smoke cache, two rounds green then one red on identical code). The retirement is now a shared helper with a cheap not-running probe, called before every fixture cleanup that follows an install and by the final phase; a daemon that will not retire stays a hard failure. The 9b fixtures also route LOCALAPPDATA under the fixture HOME: their installs staged launcher pairs into the real runner profile, outside the fixture's isolation and the harness's cleanup reach. Verified: full VM smoke green end to end (all 9b subphases, phase 17 through the shared helper). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
664058e86b |
test-infra(win): CI-shaped VM harness, managed-install smoke gate, calibrated suite tier
The 40 hidden Windows failures fixed in the previous commit reached CI unseen because the local Windows leg never validly ran the C suite: MSYS2's shared /tmp fails the secure-ancestry validation (refusals, not signal) and the ssh driver piped everything through `tail -40`. This closes those holes and hardens the gates that let an install failure scroll past. - vm-run-tests.sh (new, used by win.sh test / ubsan-test / trap-ubsan-test): gives the suites CI's exact protected per-user temp root (owner-stamped, protected current-SID DACL, mirroring the workflow's pwsh step), shapes the runner's build directory like a real user checkout (drive-root trees inherit an Authenticated-Users Modify ACE that profile-rooted checkouts do not have, which the activation source-directory policy correctly refuses; DACL re-rooted two-step — directory protect then child /reset, since inheritance flags are directory-only and a /T re-root leaves files with empty deny-all DACLs), streams the FULL output, and refuses to report success unless the runner printed its completion summary. - smoke: Phase 8 no longer swallows the install exit code. On Windows it now requires the managed install to SUCCEED and to publish an exact two-link canonical launcher — a staging refusal used to scroll past as tolerated noise while the downstream config assertions kept passing. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
6ab5e3fa79 |
fix(smoke): retire the account daemon before the harness ends
The account daemon legitimately outlives its last client for a moment while it drains, holding its log and config DB open. POSIX rm does not care, but on Windows an open file blocks deletion, so the caller's cleanup of the smoke cache raced the daemon's asynchronous shutdown and failed with "Device or resource busy" after every phase had passed — first visible on windows-latest once the agent-config phases stopped failing earlier, and reproduced identically on the local Windows VM. The smoke now ends with an explicit phase: stop the daemon through its own lifecycle command, wait (bounded) until status reports not-running, and give Windows one beat for the final handle close. A daemon that will not retire is now a smoke FAILURE in its own right instead of a cleanup accident. Verified red-to-green on the VM and green on macOS. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
34f9ad1b3d |
fix(daemon,win): CI-green fixes and a daemon-stability guard
Fixes surfaced while bringing the shared-coordination-daemon branch green across macOS, the Linux containers, and the real Windows-ARM64 VM, plus a new stability guard that caught one of them. - daemon(win): the CLI teardown transition latched a failed release as permanent, but a Windows participant-state release must briefly try-hold the shared startup/legacy gates and legitimately collides with a concurrent one-shot's teardown; the release is retriable by contract (it always retains the transition), so parallel one-shot commands no longer report "CLI coordination cleanup failed" despite succeeding. Found by the new stability guard's churn section. - cli/main(win): install/update/uninstall and the Augment hook-script removal derive the managed launcher's identity from its plain drive-form path (cli_windows_plain_utf8), not the \\?\ extended-length form, so agent-config ownership matches on Windows and uninstall no longer leaves the MCP entry or owned hook scripts behind. - security: the `daemon start --open` browser launch is now shell-free (ShellExecuteW on Windows, cbm_exec_no_shell elsewhere) instead of system(); the CORS origin check spells out its two literal loopback URLs so the static URL audit sees a complete value. Both clear the Layer-1 allow-list audit. - coordination: version-cohort lock retries now sleep a per-process jittered interval — fixed-period retries can phase-lock two participants so one starves. The activation-quiesce test's observation window now covers worst-case candidate staging (three tamper-defense hashes over a ~1 GB sanitizer binary exceed the old 30 s budget on container I/O). - smoke/soak: the agent-config smoke matches Windows config paths through their escaped-backslash quoted form; the soak parser skips the update-available banner before the JSON summary and gained a one-shot CLI admission-churn phase inside the RSS/FD leak window. The build-dir safety fixture creates a real symlink/junction (MSYS2 ln -s otherwise deep-copies) so its traversal-refusal contract is genuinely exercised, and the symlinked-agent-roots test demotes its planted links to an unprivileged owner when run as root so the refusal is really tested. - tests(win): a new daemon-stability guard exercises the parameter surface, hook fail-open with its rate-limited notice, start-twice and occupied-port handling, busy-stop refusal against a live MCP session, kill -9 crash recovery, and sequential/parallel one-shot churn. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
0baddf7887 |
daemon: permanent lifecycle, daemon-backed CLI/hooks, real-Windows hardening, long-path launcher transactions
Daemon lifecycle and Windows correctness, verified on a real Windows 11 ARM64 VM through the maintained test-infrastructure/vm drivers, plus the macOS and Linux arm64 suites and the container lint gate. Daemon lifecycle: - daemon start/stop/status subcommands. `daemon start` launches a PERMANENT daemon (spawn shape is byte-exact argv; survives idle periods and session ends) and reports an already-active daemon instead of failing. Permanence is honored at every stop latch: last-committed-client disconnect, host initial-client window, coordinator release, and application final-session close — a permanent daemon also keeps admitting new sessions after its last one closes. - daemon stop refuses while sessions are active and lists the blocking peers (pid/role) that must finish first; an idle daemon drains through the activation-shutdown machinery with the ACK ordered after connection interrupts. A second stop is idempotent. The wire ops are no-cohort first-frame requests with peer fingerprint authentication, so stop/status never conflict with an exact-build admission gate. - One-shot CLI commands now execute through the daemon (index workers keep their local supervised path). A cold CLI run that had to spawn a temporary daemon prints a hint that `daemon start` removes the per-command startup tax; a warm daemon is recycled silently. - Hooks are connect-only fail-open: with no daemon present the hook emits a visible, rate-limited notice (Claude-dialect systemMessage plus stderr for other dialects) and always exits 0 — augmentation is never allowed to block the caller's tool use. - Version skew: a newer-build client automatically drains an older-build permanent daemon (strict semantic-version triples only; dev builds never auto-drain) and the build-conflict message names `cbm daemon stop` as the manual escape hatch. Windows IPC/runtime (real-VM verified): - ipc(win): persistent pending overlapped ConnectNamedPipe. The accept path used to destroy its listening pipe instance on every 20 ms poll timeout; a client attaching in the teardown window was severed or left on an orphaned pipe object whose HELLO no server handle could ever read, absorbing the connect until the client's own timeout expired. The pending connect now survives poll timeouts and nothing is destroyed while a client could be attaching. - ipc(win): drain-before-close for final responses. Closing a named-pipe server handle can discard a just-sent response before the peer reads it (POSIX stream sockets never lose buffered data on close). A bounded cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX, immediate on interrupted connections) now precedes close in runtime_worker_finish and runtime_reject_inline, so hello-conflict, capacity and disconnect acknowledgements reliably reach the peer. - runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no transport half-close, so close_begin now announces departure with an explicit frame (ordered after APPLICATION_CANCEL, before the local interrupt); the server releases the client's admission on receipt instead of waiting for the handle to close. Admission-drop timing is now identical to POSIX shutdown() semantics on every platform. - runtime(win): client close cancellation. close_begin serializes with request publication under the send lock, best-effort sends the active token's APPLICATION_CANCEL frame, then interrupts local I/O; the server cancels MCP/subprocess work promptly. Contract tests accept both correct outcomes (interrupted transport or decoded CANCELLED). - runtime: activation acknowledgement ordering. The activation ACK is the requester's license to act on "snapshotted and draining", so every connection interrupt is now initiated before the ACK is sent; a session could previously get one more request serviced after the requester observed the ACK. - service(win): deadline-bounded private-file prepare. The conflict-log prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms timer granularity) burned ~1.6 s against permanently obstructed paths, stalling hello rejections past the client's timeout. The retry budget is now a 250 ms deadline; transient share collisions still retry. - subprocess(win): cmd.exe /C payload encoder quotes metacharacters correctly (root cause of the git-on-Windows failure cluster). - watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native Windows stop/unwatch cancellation test with exact-image verification. - httpd: send_all writes in bounded 64 KiB slices. A single giant nonblocking send() on Windows is absorbed wholesale into AFD kernel buffering regardless of SO_SNDBUF, so send deadlines and interrupts could never engage against a slow peer (and the full payload was pinned in nonpaged pool). Slicing restores a deterministic backpressure point; a test hook pins SO_SNDBUF for the deadline and interrupt tests. - ui/http: shutdown lifecycle — interrupt checks, response-wide send deadline, explicit connection states, refusal to free a server while a listener-owned connection is active. Windows long-path support: - Central path-aware wide conversion (canonicalize via GetFullPathNameW and prepend the extended-length prefix for absolute paths >=240) at the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite store opens, and the daemon build-fingerprint/log paths. Deep managed installs (a 64-hex generation directory routinely exceeds MAX_PATH) now index, stage and activate correctly. - activation transaction: its own file APIs and the component-walking ancestry validators now operate in the extended-length namespace; the launcher path is canonicalized (and prefixed when deep) once at entry so every downstream exact-string comparison stays form-consistent. - Executable self-resolution uses the wide APIs (GetModuleFileNameW, GetFileAttributesW) so non-ASCII install paths survive argv[0] resolution. Windows launcher install/uninstall transaction: - FileRenameInfoEx names are NUL-terminated in an over-allocated buffer. FileNameLength governs per the contract, but filter drivers read FileName as NUL-terminated and appended adjacent heap bytes to created names — a flaky, garbage-suffixed rename target. Both the CLI and the launcher rename helpers are fixed. - Uninstall retires state via rename-aside (.cbm -> .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex chars so the bare rename target stays under the FileRenameInfoEx NT-conversion ceiling at guard depths; 64 bits still uniquely identify the generation. - When the running launcher's mapped generation backings pin .cbm against rename, the backings are relocated to activation-<pid>-N .retired tombstones beside the install (a mapped image may be renamed, never deleted; the launcher's liveness-guarded sweep reclaims stale tombstones). Every relocation is recorded, and a FAILED uninstall reverses the moves after restoring .cbm — via MoveFileExW with extended-length paths on both arguments, since the deep generation target is beyond the handle-based rename's bare-path reach — so a restored install keeps its generation backings and stays runnable. - After a committed uninstall the retired tree's backings are relocated out so the tree is shallow enough for the detached cleanup's rd, and the cleanup's working directory strips the extended-length prefix (CreateProcessW lpCurrentDirectory silently ignores prefixed paths). - Files created under Administrators-default-owner directories (CopyFileW destinations, CREATE_NEW tombstones, probe directories) are explicitly owner-stamped so the exact-owner validators hold on runner images; guard fixtures stamp hand-built trees the same way. Diagnostics, tests and infra: - diagnostics: discovery is now an always-delivered JSON control record (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and paths containing spaces; placement honors $TMPDIR with /tmp fallback via a diagnostics-local helper; the soak parser reads the JSON record; documented in docs/CONFIGURATION.md. Red-first coverage for suppressed log levels, TMPDIR-with-spaces, and native Windows output-contract assertions. - tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint parents with production-shaped ancestry (LocalAppData on Windows, via th_secure_runtime_parent_new) — the runtime ancestry validation correctly refuses temp roots whose ancestors grant mutation rights to Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and drive the documented startup-owner publication flow before reading generation-bound endpoint addresses. This turns the 26 Windows failures previously visible in CI's full-test job green without weakening any validation. - tests(win): the launcher guard covers the full permanent-launcher contract including failed-uninstall restore and immediate reinstall after uninstall; new daemon lifecycle and reworked hook-augment guards run the start/recycle/stop flow end to end. - tests: CBM_SKIP_PERF is now actually consumed by the test runner (it was set by CI but never read, so perf suites ran everywhere); four throughput/bench suites are classified as perf, the heavy store_arch suite moved to the slow-timeout tier, and two wall-clock-sensitive assertions were rewritten as invariant checks with coarse hang-detector backstops. - build/test infra: build-dir safety contract, UI dev-proxy security contract, soak daemon-recovery contract, path-safety helper, the Windows VM worktree-sync contract wired into scripts/test.sh, and vm/win.sh guards building its clean embedded-UI product in an isolated BUILD_DIR so it cannot clobber the incremental test build. provision-windows.sh now installs Node.js for the guards UI build. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
897aa7ab1d |
test(smoke): echo the install --dry-run output on the dry-run-mode failure
The Windows dry-run failure prints an error (which passes the install/skill/mcp/agent check) but no "dry-run" indicator, and the smoke never echoed the captured output on THIS branch, so the actual error stayed hidden. Echo it. (Root: Windows install routes through cli_windows_managed_install, which errors on launcher/payload source verification before emitting a dry-run plan - the next run will name which check fails.) Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
d15a74a468 |
fix(smoke): capture daemon diagnostics on an MCP no-response failure
pr-smoke's mcp_run discarded the frontend's stderr (2>/dev/null), so a first-session daemon start/connect failure - which with the daemon architecture surfaces ONLY on stderr, stdout being reserved for JSON-RPC - showed up as a bare "no initialize response (id:1)" with nothing to diagnose (exactly what an isolated ubuntu pr-smoke failure just produced). Keep the frontend stderr and, on an id:1/id:2 miss, dump it alongside the durable cbm-daemon.log and daemon-conflicts.ndjson tails from the cache root. No behaviour change on success. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
5d8a9421fb |
Merge remote-tracking branch 'origin/main' into feat/shared-coordination-daemon
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> # Conflicts: # src/mcp/mcp.c # tests/test_cli.c |
||
|
Martin Vogel
|
ad58e71bea |
fix: stabilize cross-platform daemon coordination
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
cc5f71bdff |
fix(hook,smoke): teach the last two consumers the tree output shape
The PR CI caught two consumers the format migration missed — both parsed
tool output by its old shape and failed silently or with a wrong count:
- hook-augment built its PreToolUse additionalContext by reading the
legacy 'results' object-array from search_graph format:json; after the
json-tree reshape that key no longer exists, so the hook emitted nothing
(caught end-to-end by tests/windows/test_hook_augment.py). The parser
now walks groups -> qn_prefix/file + column-ordered rows. A new local
test feeds the parser the LIVE server envelope, so any future drift
between response shape and hook parser fails in the normal suite on
every platform — not only in the Windows CI guard. Reproduced RED
(ctx NULL, the exact CI failure) before the fix, GREEN after.
- smoke-test.sh carried six old-shape parsers (callers[N], rows[N]{,
clusters[N], semantic[N]{, and the assert_toon_table helper); the tree
headers never matched, so trace verification read 0 callers. All six
updated to the tree contract; the full smoke script passes end-to-end
locally.
cli suite 222 passed; lint clean; scripted smoke ALL PASSED.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
b8a75d142c |
fix: stabilize cross-platform daemon launch
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
cb896a3de4 |
fix: harden cross-platform daemon startup
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |