Brings the branch up to main 9c19b5dd.
Two same-spot insertion conflicts, each resolved as the main version
of the file plus the own diff of this branch:
- src/daemon/application.c: the job struct gained watch_owners on main
and request_started_ms plus the request facts here; both kept.
- tests/test_daemon_application.c: main added the #1818
autoindex-skipped helpers where this branch added the worker resource
failure test; both kept, the test placed after the doc_candidates
test.
Everything else merged cleanly. src/main.c changed on both sides: main
(#2838) lowers the worker CPU priority inside the worker role block,
and this branch strips the publication fence flag before that block
and installs the fence after the worker role options; the hunks are
disjoint and both are kept. tests/test_pipeline.c gained the Next.js
route tests (#2747) on main and the publication gate test here, also
disjoint. No call site needed adapting: no new main code calls a
function this branch re-signed, and main re-signed none that this
branch calls.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A worker resource limit is enforced by the supervisor, but only the worker
replaces the serving database, and it does so before the persistence
artifact export. A limit observed in that window stopped a worker that had
already published, and the response still said serving_index_preserved=true
while the previous index had been replaced.
Publication and resource termination now coordinate through a per-attempt
publication fence: a never-created path next to a reserved unique file in
the private worker directory, which both sides claim by creating it
exclusively (cbm_fopen "wbx": O_EXCL on POSIX, CREATE_NEW on Windows), so
exactly one of them can win.
- The worker claims it through a new pipeline publication gate that
cbm_pipeline_run calls once, after the last pre-publication check and
before the serving database is touched. Losing means the supervisor
already stopped the attempt for a limit: the run aborts before
publication and the previous generation stays in place.
- The supervisor claims it before it terminates the worker for a limit or
a failed RSS probe. Losing means the worker already committed to
publishing: the limit is no longer enforced for that attempt, which runs
to completion, and index.supervisor.resource_limit_after_publication is
logged instead. A claim that can be neither won nor lost is retried on
the next poll rather than risk stopping a committed worker.
- At terminal the supervisor settles the fence into
result.publication_committed. An unquiesced tree keeps the fence, so a
surviving member can never publish after the report.
serving_index_preserved now describes the request, not only its last
attempt: a servable index existed when the request began, no attempt of
the request committed to publishing (an earlier attempt can publish and
then crash into recovery), and that index is still servable. The direct
path and the daemon job both fold every attempt into these request facts;
cbm_mcp_index_worker_resource_response takes them as a third argument, and
NULL claims nothing.
The fence path travels as an optional last pair of the exact worker grammar
(--index-worker-publish-fence), accepted by the worker argv parser and by
the daemon bootstrap role check, stripped by the CLI and installed by both
worker entries.
Limits stay opt-in: without index_max_rss_mb or index_max_duration_seconds
there is no fence, no extra argv, no claim and no extra database read.
Graph content is unchanged either way.
RED (c78c077a plus only the new end-to-end test): a FIFO at the artifact
temp path holds the real worker after the rename, and the duration clock
crosses its limit only once the serving database has been replaced. The
supervisor stops the worker there and answers resource_limit_exceeded with
serving_index_preserved=true over a replaced database. GREEN: the worker
claimed first, the supervisor logs the limit as not enforced, and the
released worker finishes and reports the published index.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main a5b248f47. Eight files conflicted, each a
same-spot insertion where main and this branch added lines next to the
same anchor. Each resolves to the main version plus the change of this
branch:
- src/cli/cli.h, src/cli/cli.c: CBM_CONFIG_IGNORE_WORKTREES and its
CONFIG_KEYS row next to the new doc_candidates key.
- src/mcp/mcp.c: the index_worktree schema property next to the new
doc_candidates property.
- README.md, docs/CONFIGURATION.md: the ignore_worktrees line and table
row next to the rewritten auto_watch entries and the new
doc_candidates and watch_non_git rows.
- src/git/git_context.c: the env-isolation comment after the new debug
stage marker, directly above cbm_popen_git.
- src/cli/hook_augment.c: the new SubagentStart cli_hint ahead of the
parent-graph check of this branch.
- tests/test_mcp.c: both RUN_TEST blocks; main already includes
str_util.h.
Per file, the lines changed against main equal the changes of this
branch against the merge-base.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The review of 2026-10-01 asked for this to be a stated decision rather
than a side effect. Detection walks up from the given path to the
nearest .git entry, so a subdirectory of a linked worktree is treated
as that worktree: index_repository on it is refused, and a session root
or hook working directory inside it is handled the same way. The walk
stops at the first .git; outside any checkout it runs to the filesystem
root one directory at a time. None of it runs while the key is off.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A daemon started with `daemon start` now keeps a project watched after its
last owning MCP session closes, until the daemon stops, the project is
deleted, its directory disappears, or auto_watch is set to false. README
(the watcher paragraph and the index_repository row), the watcher_enabled
row of docs/CONFIGURATION.md and the comment above the index_status watch
provider still said every watch ends with its last session.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Match canonical Windows separators across watch ownership and admission,
drain all session watches when auto_watch is disabled, and retain the
resolved doc_candidates choice for later refreshes. Add deterministic
regressions for the five observed failures and an absent-option guard.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Replace the PATH_ALLOW_BROAD environment variable with a recorded grant,
the shape agreed in review: `codebase-memory-mcp allow-root
--approve-broad <path>` records an explicit human approval for one root
refused as too broad (CBM_WS_DENY_TOO_SHALLOW), mirroring
--approve-sensitive end to end.
- cbm_workspace_grant_add() takes approve_broad next to
approve_sensitive. Each flag lifts only its own verdict, absolute
denials stay unliftable, and a refusal names the flag that would
lift it.
- The approval is stored with a '^' marker, listed by allow-root --list
as "(approved broad)", and honoured by cbm_workspace_root_allowed()
only for CBM_WS_DENY_TOO_SHALLOW and only on an exact path match,
keeping the exact-match rule of the original change.
- cbm_workspace_root_allowed() returns to its previous signature. Every
caller (MCP index_repository and index-job start, auto-index, daemon
sessions, the HTTP UI) reads the same grant store, so stdio and
daemon sessions decide identically and nothing depends on the
environment a daemon was started with.
- The too-broad refusal names `allow-root --approve-broad <path>`, a
command a person runs.
- cbm_workspace_classify_root() and cbm_workspace_verdict_is_overridable()
are unchanged.
The regression test keeps its name and its exact-match, sibling,
absolute and sensitive checks, now against the grant store, and adds
that a broad marker never lifts a sensitive verdict.
Co-authored-by: Mark Murawski <github@kobaz.net>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Watchdogs now sample the OS-charged footprint instead of resident size on
Darwin, and index_max_duration_seconds covers the whole request so a
crash/hang respawn cannot reset the clock.
Signed-off-by: 刘冲 <mail@liuchong.dev>
Discovery limits bound what indexing accepts, not what it then costs. A
repository well inside those bounds can still exhaust the host through parser
memory, or simply never finish, and a supervised worker that hangs leaves the
parent waiting with nothing to report.
Add index_max_rss_mb and index_max_duration_seconds, enforced by the parent
against the worker process tree rather than the worker process alone, so a
runaway child cannot hide behind a small parent. Resident memory is sampled
through the platform interface on macOS, Linux and Windows. Crossing a limit
terminates the tree and yields one trusted, structured terminal result that
attributes the failure to the resource that caused it.
Both limits default to off. A measurement that cannot be taken fails the
attempt instead of passing it: a watchdog that quietly stops watching is
worse than no watchdog at all.
The shell fixture that stands in for the supervisor names the two new keys.
The worker accepts only a policy that spells out every key it knows, which is
what keeps a stale supervisor from starting a worker it cannot bound.
Signed-off-by: 刘冲 <mail@liuchong.dev>
Doc-section -> code candidates (doc_link_candidates, #2574/#2610) are the
documentation-in-the-graph feature of this release. They now run by
default for full indexes and not for moderate or fast ones; an explicit
`doc_candidates` argument, or a `doc_candidates` setting the user set,
overrides the default either way in every mode.
- pipeline: the switch is unset/on/off; unset means on only when the
effective mode is full.
- MCP: forwards only an explicit choice (argument, then a user-set config
value) to daemon jobs and workers; daemon auto-index and watcher/UI jobs
now carry a user-set value too (their workers have no config, so
`doc_candidates false` was ignored there).
- fast + doc_candidates on runs the semantic pass's corpus phases and the
doc phase only: candidates, and still no SIMILAR_TO or
SEMANTICALLY_RELATED edges.
- config: `doc_candidates` joins the config key table (default `auto`);
`config set doc_candidates ...` was refused as an unknown key before.
README, docs/CONFIGURATION.md and the MCP schema describe the default.
Proof (macOS arm64, django fb11376):
full 11,304 rows · moderate 0 · fast 0 · full+off 0 · moderate+on 1 ·
fast+on 1 (moderate/fast discovery skips docs/) -- nodes and edges are
identical between on and off in every mode; fast+on adds 0 semantic
edges.
- New tests (RED on the old code): pipeline
pipeline_doc_candidates_default_follows_mode,
pipeline_doc_candidates_explicit_wins_over_mode,
pipeline_doc_candidates_fast_on_legacy_incremental; mcp
tool_index_repository_doc_candidates_follow_mode,
tool_index_repository_doc_candidates_forwarded_only_when_explicit;
daemon_application
daemon_application_programmatic_index_forwards_set_doc_candidates.
- mcp 374 passed (4 Windows-only skips), daemon_application 73 passed,
pipeline/store_arch/cli/runtime_settings/index_policy/pipeline_frozen
green on the release tree.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The doc<->code candidate scoring (#2574, doc_section_worker in
semantic_edges) scores every documentation section against the functions
that share a key term with it, key terms being those held by at most
funcs/20 functions. Its cost therefore grows with sections x functions.
Measured on 11 corpora (10-09 main vs 4d9592b5, semantic_edges): under a
second up to ~100k functions, +12.6 s on java (325k), +14.6 s on csharp
(451k), +174 s on the Linux kernel (784k functions, 28k sections), almost
all of the kernel's regression against v0.11.0.
A new boolean setting, doc_candidates (default true), and an optional
index_repository argument of the same name switch the scoring off: no
candidate rows are stored and nodes and edges are unchanged. The argument
wins over the setting. The index handler resolves the value once and
passes it to a daemon job or supervised worker as an argument, the same way
as the resource policy, so every route sees one decision; the session
auto-index reads the setting. With the switch off, a delta-route reindex
replaces the candidate rows it would otherwise keep, so turning it off
leaves no stale candidates. The default output is unchanged.
Tests (each fails on its assertion when the switch has no effect):
- pipeline_doc_candidates_off_stores_none_and_keeps_graph: no rows, same
node and edge counts as the default run, which has rows;
- pipeline_doc_candidates_off_clears_rows_on_delta: a closure-repair
reindex with the switch off removes the earlier rows;
- tool_index_repository_doc_candidates_switch: argument false -> none;
setting default -> rows; setting false -> none; argument true over
setting false -> rows.
Proof, release-built binaries on macOS arm64 (M5 Pro 18c/48 GB):
- django, main 4d9592b5 vs this change with defaults: nodes (63,441),
edges (307,584) and doc_link_candidates rows (11,304) hash-identical;
- django with doc_candidates false: nodes and edges hash-identical, no
candidate rows;
- Linux kernel with doc_candidates false: semantic_edges 17.9 s (191.9 s on
main, 17.6 s before #2574), whole index 210.9 s (v0.11.0: ~234 s), no
candidate rows.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Since #1948 a non-git root can be polled by a file-tree scan when
watch_non_git is on. index_status still reported such a root as
strategy "none", which the enum documents as "not a git project: never
polled", so a root that is actively polled looked unwatched.
publish_status now maps a tree-polled root to the new
CBM_WATCHER_STRATEGY_TREE, rendered as "tree" (the value the
watcher.baseline log already uses). "none" keeps its meaning: a
registered non-git root that is never polled. CONFIGURATION.md lists
the four values.
Test: daemon_application_index_status_reports_tree_strategy_issue2167
fails without the production change and passes with it; suites
daemon_application, watcher and mcp: 507 passed, 4 platform skips.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The background watcher only watches the project an open MCP session is
rooted in, and drops the watch when the last session for that project
closes. A repository indexed with `cli index_repository` (or from a
session rooted elsewhere) is never watched, but README promised
"Auto-sync keeps it fresh after that" for every index_repository target
and the MCP instructions said "Indexes auto-refresh". Nothing in the
status output said which projects were actually watched, so a stale
CLI-indexed repo looked like a watcher bug.
What gets watched is unchanged. Instead:
- index_status gains a `watch` object: `watched`, plus `strategy`,
`poll_interval_ms` and `last_scan_at` when watched, or a `reason`
(not_session_project, cli_session, auto_watch_off, watcher_disabled,
not_registered, no_watcher) and a re-index hint when not. It does not
touch the #1561 `freshness` key.
- The watcher publishes per-project strategy, cadence and last completed
scan time through atomics and exposes them via
cbm_watcher_project_info(); the daemon answers index_status through a
session-scoped provider.
- README, docs/CONFIGURATION.md, docs/index.html, the index_status tool
description and the MCP instructions now state the real scope.
`watcher_enabled` already shows in `config list` since v0.11.0 (680bc72e).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A cache root whose resolved path runs through a directory owned by another
non-root account (typical on shared/HPC filesystems where ~/.cache is a
symlink into a shared tree) is refused by the private-directory walk. The
refusal itself is correct: the owner of an ancestor can rename or replace
everything below it after the walk, so it could redirect the cache. But
posix_directory_transition_secure() set no validation detail, so the caller
printed "ancestry component validation failed (errno 17)" -- a stale EEXIST
from mkdirat() that named neither the directory nor the reason.
- The refusal now names the directory, its owner uid and account name, the
caller's uid, and the remedy (a location with a self/root-owned ancestry,
e.g. via CBM_CACHE_DIR). The path is shortened from the left so the
remedy always fits the 384-byte detail buffer.
- On Linux the account name is read from /etc/passwd, not getpwuid(). The
release binary is statically linked against glibc, and getpwuid() there
loads the host's NSS modules at runtime, which can crash on a glibc
version mismatch (a real risk on sssd/LDAP cluster hosts). Accounts that
exist only in a directory service are shown by uid.
- New advice-only check: on Linux, a cache root on NFS, CIFS, SMB2, Lustre
or GPFS (statfs f_type) logs one daemon.cache_root_network_fs warning per
process. SQLite WAL is unreliable on network filesystems. Nothing is
refused.
- docs/CONFIGURATION.md gains a "Shared / HPC filesystems" section.
The ownership policy is unchanged, and there is still no override (same
stance as #1830).
Test seams: one seam reports a directory (matched by dev/ino) as
foreign-owned, and another supplies the statfs magic. The refusal, the
message and the once-per-process warning are therefore testable without
root and on every platform.
Refs #1717
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
README.md:132, README.md:633, README.md:761, and docs/CONFIGURATION.md:203
said one-shot `cli` commands "never start or connect to the coordination
daemon" / read "their own environment without starting the daemon". That
was already contradicted a few lines above it (README.md:126 lists
one-shot CLI commands among the processes that "share a crash-safe OS
admission barrier" with the daemon) and by the code:
main_local_cli_daemon_execute() (src/main.c) bootstraps a client
connection to the shared per-user coordination daemon via
main_client_bootstrap_with_upgrade(), spawning one (bootstrap.daemon_spawned)
when none is running, before dispatching the tool call.
Verified live: `cli --verbose list_projects` against an isolated
HOME/CBM_CACHE_DIR/CBM_RUNTIME_DIR printed "hint: this command started a
temporary CBM daemon", and its cbm-daemon.log showed
daemon.start -> daemon.runtime_stopping (reason=last_committed_client_disconnected)
-> daemon.lifetime_end, i.e. the daemon it spawned exited again once the
CLI command's own connection closed. bootstrap_production_spawn() passes
the calling process's `environ` straight to posix_spawn(), so a CLI
invocation that starts the daemon also seeds that daemon's captured
daemon-owned environment (CBM_DIAGNOSTICS, CBM_LOG_LEVEL, etc.) - it does
not read "its own environment" independently as the docs claimed.
Corrected all four passages to state the real behavior: CLI commands
connect to the shared daemon (starting one if none is running) for the
same admission barrier and per-project locks, hold a `cli_session` that
is never registered with the background watcher, and - if their own
connection is what started the daemon - that daemon exits again once the
command's connection closes and no other session is attached. No opt-out
env var or flag for an in-process/no-daemon CLI mode exists (grepped the
whole tree), so none is documented.
This is a pure documentation fix; no production behavior changed. Kept
clear of PR #2348 (fix/issue-2167), which separately rewrites the
watcher-scope paragraph at README.md:152/237/672 and
docs/CONFIGURATION.md:88 for issue #2167's auto-sync-scope correction -
none of the lines here overlap with that diff.
Refs #2183, #2167
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A project indexed from a directory that is not a git repository was never
refreshed after its first index: init_baseline classified it is_git=false
and poll_project/check_changes returned early for it forever, so the graph
silently went stale. The comment in init_baseline even claimed such folders
were "polled as a plain directory", which was never true.
Add an opt-in config key, watch_non_git (default false, read once at daemon
startup like watcher_enabled). When on, a non-git root is polled on the
existing adaptive cadence (cbm_watcher_poll_interval_ms) by a tree
signature: the indexer's own cbm_discover walk (same skip lists, .gitignore
stack and .cbmignore), sorted by path and folded over (relative path, size,
mtime). A changed signature takes the existing reindex path with the same
commit-after-success discipline as the #937 dirty signature. The baseline
leaves the signature "unknown", so the first poll reindexes once
(at-least-once, covering edits made while the daemon was down).
Runaway guard: only files discovery would index enter the signature, so
cbm's own .codebase-memory output (built-in skip) and the cache directory
(pruned by absolute path) can never make a finished reindex look like a
new change; a scratch folder under an unrelated dirty repository is polled
on its own files and never inherits the ancestor's dirty state.
cbm_file_info_t gains mtime_ns, filled from the stat discovery already
performs, so the signature needs no second stat per file and stays
UTF-8-correct on Windows (discovery's wide stat).
Docs: README and docs/CONFIGURATION.md now say non-git roots are not
watched by default and document watch_non_git. The staleness signal in
index_status is left to #1561's freshness object. Refs #1948.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On WSL2 the Windows drives under /mnt/<drive> are DrvFs (9p) mounts. With
the default automount options they report 0777 for every directory and
ignore chmod, so a CBM_CACHE_DIR there is refused by the private-directory
walk: the ancestor rule (not world-writable) fails first, and the leaf
0700 rule would fail right after. The refusal is correct -- the 0777 is
real -- but it gave WSL users no way forward.
The security gate is unchanged. When the refusing directory is on a 9p
filesystem under a WSL kernel (fstatfs magic 0x01021997 plus
/proc/sys/kernel/osrelease), the ancestor refusal and the leaf
"mode survived chmod" refusal now name the remedy: set
[automount] options = "metadata,umask=22,fmask=11" in /etc/wsl.conf and
run `wsl --shutdown`, or keep the cache on the Linux filesystem. The
ancestor message uses shorter rule text on DrvFs so the remedy fits
the 384-byte detail buffer. Detection compiles only on Linux; other
platforms and non-WSL 9p mounts keep the generic message.
A CBM_ENABLE_TEST_SEAMS switch forces the DrvFs verdict so the daemon_ipc
suite checks the remedy text (and its absence off DrvFs) deterministically.
docs/CONFIGURATION.md documents the WSL requirement.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Agents that use `git worktree add` to run parallel branches end up with one
indexed project per worktree. Each is a near-duplicate of the main checkout,
so the cache fills with redundant graphs and project pickers get noisy.
Add an opt-in `ignore_worktrees` config key. When enabled, a linked worktree
is left alone:
- auto-index on connect skips it (logged as autoindex.skip)
- the daemon's background auto-index skips it
- explicit index_repository refuses with an actionable message naming both
ways forward (index_worktree=true, or turn the key off)
- the hook-augment "not indexed" guidance stops telling the agent to run
index_repository in a directory where it would only be refused
Detection is git plumbing, no subprocess: <path>/.git must be a regular file
holding a "gitdir:" pointer AND that gitdir must contain a `commondir` file.
The commondir check is what separates a linked worktree from a submodule,
whose .git is also a gitlink file but whose gitdir has no commondir. The main
checkout of a repo that has worktrees is never affected.
Defaults to false, so behaviour is unchanged unless the key is set.
Signed-off-by: Nohj <vitalnohj@gmail.com>
Indexing accepts whatever a repository contains. A tree carrying a vendored
monorepo, a generated dump, or a runaway build directory is discovered in
full, and the first sign of trouble is a host under memory pressure with
nothing that attributes it to indexing.
Add two opt-in limits evaluated during discovery against accepted source
files only: index_max_files and index_max_source_mb. Both default to off, so
nothing changes until an operator sets one. Crossing a limit fails the whole
attempt with a structured resource_limit_exceeded result naming the resource,
the observed value and the limit; no partial graph is published, and an
existing serving index keeps answering.
Limits are read from the CLI-managed _config.db and are not MCP request
arguments. A supervised parent replaces any caller-supplied policy before
spawning its worker, and the worker rejects a missing or incomplete contract,
so the CLI, the daemon and the supervised worker all enforce the same
decision.
Both keys reach an operator through the existing config get/set/list/reset
with no new subcommand. `set` suppresses its own generic message for them
because the policy writer names the precise reason -- so that writer speaks
on every failure it can return, including a validated value whose write then
fails on a database that cannot be written. Exiting non-zero in silence is
not an acceptable answer from a CLI.
The two shell regressions that hand-roll the supervisor's worker argv carry
that contract as well. Without it the worker exits before either guard can
observe anything, and the guard would go quietly vacuous.
Signed-off-by: 刘冲 <mail@liuchong.dev>
Follow-up to the PR #1105 review: the unit test proves the config predicate but
would still pass even if the daemon-side gate were removed. Add a process-level
regression that drives the real binary and proves the gate actually prevents the
watcher from being built, started or registered when disabled.
- tests/test_watcher_disabled.sh (wired into scripts/test.sh as Step 5e) drives
the binary against an isolated CBM_CACHE_DIR + tiny git fixture. With
watcher_enabled=false it asserts: watcher.disabled reason=config emitted,
watcher.start absent, no watcher.watch registration, auto_index still runs, and
manual index_repository is served. A watcher_enabled=true positive control
asserts watcher.start + watcher.watch appear, so the test fails if the gate is
removed (verified by neutering the gate in host_state_prepare and observing the
failure).
No assertion is decided by a timeout. The watcher lives in the daemon, which
logs to $CBM_CACHE_DIR/logs/cbm-daemon.log in a fixed order: watcher.disabled
(host_state_prepare) -> daemon.start -> the thread's own watcher.start -> the
join -> daemon.stop. Each run therefore retires its daemon and waits for that
CLOSED lifecycle before asserting, so absence is a fact rather than a race.
Every wait is a bounded poll on an asserted state following the soak harness's
wait_for_daemon_stop doctrine, and an exhausted budget fails loudly with the
reason instead of passing quietly; session stdin is held open through a FIFO
until the last JSON-RPC response lands, not for an interval.
Registration is asynchronous and unreachable until the project DB exists
(application_refresh_watch_locked requires application_regular_db_exists), so a
closed lifecycle alone would not carry the negative — the session could simply
have ended first. The disabled+auto_index run waits for that DB, i.e.
registration's own precondition; the enabled control waits for watcher.watch
itself, so its absence in the disabled runs is evidence rather than noise.
Honors CBM_TEST_BINARY like tests/test_worker_watchdog.sh, so the BUILD_DIR=
sanitizer and container legs test the binary they built.
- docs: CONFIGURATION.md + README.md document that watcher_enabled is read once
when the daemon starts — the daemon outlives individual MCP sessions, so
reconnecting a client is not enough; `daemon stop` retires it. This corrects
the earlier claim that auto_watch is also startup-read: auto_watch is consulted
per registration, and the two keys are now contrasted on read time as well as
on scope (session registration vs whether the subsystem starts at all).
No production code change in this commit: watcher.start and watcher.watch already
exist as production logs.
Signed-off-by: soren <nguiasoren@gmail.com>
Add a persistent `watcher_enabled` config key (default true). When false, the
daemon host skips building the background watcher entirely: the poll thread
never starts and no projects are registered. Manual index_repository is
unaffected, and the config wiring stays unconditional so auto_index /
auto_watch keep working even when the watcher is off.
This is distinct from auto_watch, which only gates per-session registration
while the watcher IS running. watcher_enabled is the master switch that stops
the subsystem from starting at all — the request in #335 (doc-heavy repos where
every save churns the graph baseline).
The key is read once, when the daemon builds its host state. Changing it takes
effect when the daemon next starts; docs/CONFIGURATION.md says so explicitly.
A disabled watcher is not a daemon-startup failure — the daemon still comes up
and still owns IPC, HTTP/UI, project locks, and manual indexing; only the
watcher and the in-memory store backing it go unbuilt.
- cli.h/cli.c: CBM_CONFIG_WATCHER_ENABLED key + cbm_config_watcher_enabled()
predicate (NULL-safe, default true), registered as a CONFIG_KEYS row next to
auto_watch. The table is no longer only a print list: since #1522/#1558 it
also backs key validation, so `config set/get/reset watcher_enabled` would be
rejected as an unknown key without that row, and the default recorded there
(true) is the one the table comment asks to keep in sync with the runtime
reader's fallback.
- daemon/host.c: gate watcher construction in host_state_prepare() and the
thread create in host_background_start(); log watcher.disabled when skipped.
Teardown needed no change — stop NULL-checks, join is gated on
watcher_started, and cbm_watcher_free is NULL-safe.
- test_cli.c: cli_config_watcher_enabled_default_and_persist pins the config
layer — default on, disable/re-enable spellings, persistence across reopen,
and a NULL-config default (a config-store failure must never silently
disable the watcher).
- docs: document watcher_enabled, and add the previously-undocumented auto_watch
row to CONFIGURATION.md so the watcher knobs are described together; README
example.
Closes#335.
Signed-off-by: soren <nguiasoren@gmail.com>
The daemon/CLI rendezvous directory is created under %LOCALAPPDATA% (Windows) or
/tmp -- /private/tmp on macOS -- and every ancestor of it must pass the
private-directory walk. That ancestry is not always acceptable, and when it is
not, EVERY invocation fails, `config list` included, so the settings surface
cannot be reached either:
codebase-memory-mcp: secure daemon endpoint could not be created
#1623 narrowed the Windows side of this by admitting AppContainer package and
capability SIDs on ancestors, and named the remainder explicitly: a live local
group, Authenticated Users inherited from a secondary volume root, and orphaned
unresolvable SIDs still refuse, and "those need CBM_RUNTIME_DIR or a separate
change". #1621 is the POSIX shape of the same dead end -- /private/tmp/cbm-daemon-<uid>
refused with no way to move it.
There was no way to move it in a shipped build. The only relocation hook,
CBM_TEST_DAEMON_RUNTIME_PARENT, is compiled out unless CBM_ENABLE_TEST_SEAMS is
defined, so a test build started while the shipped build did not; CBM_CACHE_DIR
is no help either, because it moves the cache and never the rendezvous.
CBM_RUNTIME_DIR names the parent directory the rendezvous is created under. It
does NOT relax the check: the directory it names goes through exactly the same
validation as the default -- ancestors owned by you or root, not world-writable,
no allow-ACL; the rendezvous directory itself still forced to owner-only -- and a
value that fails is refused rather than silently replaced by the default. The
operator only chooses an ancestry that passes. cbm_safe_getenv never truncates,
so no half of an over-long value can become a runtime parent.
The override is resolved in cbm_daemon_bootstrap_endpoint_new(), the one function
every product endpoint goes through: the daemon, the MCP client, the local CLI,
the index worker, and the install/update/uninstall activation path in cli.c. No
call site can silently keep the default, and the detached daemon inherits the
value with the rest of its environment. An explicit parent still wins, so the
compile-time test seam and the lifecycle guards' isolated namespace behave
exactly as before.
Approach and variable name from #1576 by Leonardo trindade miranda, resolved one
layer lower so the activation path is covered too.
Refs #1574
Refs #1621
Co-Authored-By: Leonardo trindade miranda <tmonestudio@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Routes the MCP index_repository handler and the graph UI's POST /api/index
through a single decision function. The UI route previously checked only that
root_path was a directory, so an operator's configured boundary held on one
entry point and not the other; it now canonicalizes first and applies the same
policy, answering 403 with the reason.
The decision is two-tier, because a bare default-deny would refuse every
first run and a bare opt-in leaves the default open:
- Breadth is always enforced, with nothing configured. Filesystem, drive and
share roots, top-level system trees, the home directory itself and
credential directories are refused as indexing roots out of the box.
- Containment in a declared root applies once CBM_ALLOWED_ROOT is set or a
grant exists, and is evaluated first so a path outside a configured root is
reported as exactly that.
Three things the tests caught, each a real defect rather than a test fix:
- On macOS /etc, /tmp and /var are firmlinked under /private, so
canonicalizing "/etc" yields "/private/etc" and counted two deep — sailing
past a minimum of two, missing the very path being guarded. Depth now
discounts a leading "private" component.
- An earlier draft refused any root containing the cache directory. That was
over-claimed: the indexer only parses recognised source files and a graph
database is binary SQLite it would never extract. Refusing a whole root is
also the wrong remedy where the concern does hold — not walking the cache
is. Removed, with the reasoning recorded at the site.
- Rewording the refusal to "outside every allowed root" broke an assertion
matching "outside the allowed root", and that test's early return skipped
its CBM_ALLOWED_ROOT cleanup, leaking the variable into every later test in
the suite. The original wording is kept and guidance appended instead.
Worth remembering: these contracts match strings, not properties.
Docs updated in the same change, since both env-var tables said "unset
imposes no restriction" and that is no longer true: CONFIGURATION.md and
README.md describe the two tiers, and CONFIGURATION.md lists the always-
refused roots along with the two limits that matter — this constrains scope
rather than sensitivity, and the credential list is a denylist that raises the
cost of a mistake rather than closing the class. SECURITY.md's supported-
versions table was still on 0.8.x and now reads 0.9.x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.
Daemon lifecycle:
- daemon start/stop/status subcommands. `daemon start` launches a
PERMANENT daemon (spawn shape is byte-exact argv; survives idle
periods and session ends) and reports an already-active daemon
instead of failing. Permanence is honored at every stop latch:
last-committed-client disconnect, host initial-client window,
coordinator release, and application final-session close — a
permanent daemon also keeps admitting new sessions after its last
one closes.
- daemon stop refuses while sessions are active and lists the blocking
peers (pid/role) that must finish first; an idle daemon drains
through the activation-shutdown machinery with the ACK ordered after
connection interrupts. A second stop is idempotent. The wire ops are
no-cohort first-frame requests with peer fingerprint authentication,
so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
keep their local supervised path). A cold CLI run that had to spawn a
temporary daemon prints a hint that `daemon start` removes the
per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
emits a visible, rate-limited notice (Claude-dialect systemMessage
plus stderr for other dialects) and always exits 0 — augmentation is
never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
older-build permanent daemon (strict semantic-version triples only;
dev builds never auto-drain) and the build-conflict message names
`cbm daemon stop` as the manual escape hatch.
Windows IPC/runtime (real-VM verified):
- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
path used to destroy its listening pipe instance on every 20 ms poll
timeout; a client attaching in the teardown window was severed or left
on an orphaned pipe object whose HELLO no server handle could ever
read, absorbing the connect until the client's own timeout expired.
The pending connect now survives poll timeouts and nothing is
destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
server handle can discard a just-sent response before the peer reads
it (POSIX stream sockets never lose buffered data on close). A bounded
cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
immediate on interrupted connections) now precedes close in
runtime_worker_finish and runtime_reject_inline, so hello-conflict,
capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
transport half-close, so close_begin now announces departure with an
explicit frame (ordered after APPLICATION_CANCEL, before the local
interrupt); the server releases the client's admission on receipt
instead of waiting for the handle to close. Admission-drop timing is
now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
request publication under the send lock, best-effort sends the active
token's APPLICATION_CANCEL frame, then interrupts local I/O; the
server cancels MCP/subprocess work promptly. Contract tests accept
both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
the requester's license to act on "snapshotted and draining", so every
connection interrupt is now initiated before the ACK is sent; a
session could previously get one more request serviced after the
requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
timer granularity) burned ~1.6 s against permanently obstructed paths,
stalling hello rejections past the client's timeout. The retry budget
is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
nonblocking send() on Windows is absorbed wholesale into AFD kernel
buffering regardless of SO_SNDBUF, so send deadlines and interrupts
could never engage against a slow peer (and the full payload was
pinned in nonpaged pool). Slicing restores a deterministic
backpressure point; a test hook pins SO_SNDBUF for the deadline and
interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
deadline, explicit connection states, refusal to free a server while
a listener-owned connection is active.
Windows long-path support:
- Central path-aware wide conversion (canonicalize via GetFullPathNameW
and prepend the extended-length prefix for absolute paths >=240) at
the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
store opens, and the daemon build-fingerprint/log paths. Deep managed
installs (a 64-hex generation directory routinely exceeds MAX_PATH)
now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
ancestry validators now operate in the extended-length namespace;
the launcher path is canonicalized (and prefixed when deep) once at
entry so every downstream exact-string comparison stays
form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
GetFileAttributesW) so non-ASCII install paths survive argv[0]
resolution.
Windows launcher install/uninstall transaction:
- FileRenameInfoEx names are NUL-terminated in an over-allocated
buffer. FileNameLength governs per the contract, but filter drivers
read FileName as NUL-terminated and appended adjacent heap bytes to
created names — a flaky, garbage-suffixed rename target. Both the CLI
and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
.cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
chars so the bare rename target stays under the FileRenameInfoEx
NT-conversion ceiling at guard depths; 64 bits still uniquely
identify the generation.
- When the running launcher's mapped generation backings pin .cbm
against rename, the backings are relocated to activation-<pid>-N
.retired tombstones beside the install (a mapped image may be renamed,
never deleted; the launcher's liveness-guarded sweep reclaims stale
tombstones). Every relocation is recorded, and a FAILED uninstall
reverses the moves after restoring .cbm — via MoveFileExW with
extended-length paths on both arguments, since the deep generation
target is beyond the handle-based rename's bare-path reach — so a
restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
out so the tree is shallow enough for the detached cleanup's rd, and
the cleanup's working directory strips the extended-length prefix
(CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
(CopyFileW destinations, CREATE_NEW tombstones, probe directories)
are explicitly owner-stamped so the exact-owner validators hold on
runner images; guard fixtures stamp hand-built trees the same way.
Diagnostics, tests and infra:
- diagnostics: discovery is now an always-delivered JSON control record
(new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
paths containing spaces; placement honors $TMPDIR with /tmp fallback
via a diagnostics-local helper; the soak parser reads the JSON record;
documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
log levels, TMPDIR-with-spaces, and native Windows output-contract
assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
parents with production-shaped ancestry (LocalAppData on Windows, via
th_secure_runtime_parent_new) — the runtime ancestry validation
correctly refuses temp roots whose ancestors grant mutation rights to
Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
drive the documented startup-owner publication flow before reading
generation-bound endpoint addresses. This turns the 26 Windows
failures previously visible in CI's full-test job green without
weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
contract including failed-uninstall restore and immediate reinstall
after uninstall; new daemon lifecycle and reworked hook-augment
guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
(it was set by CI but never read, so perf suites ran everywhere);
four throughput/bench suites are classified as perf, the heavy
store_arch suite moved to the slow-timeout tier, and two
wall-clock-sensitive assertions were rewritten as invariant checks
with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
contract, soak daemon-recovery contract, path-safety helper, the
Windows VM worktree-sync contract wired into scripts/test.sh, and
vm/win.sh guards building its clean embedded-UI product in an
isolated BUILD_DIR so it cannot clobber the incremental test build.
provision-windows.sh now installs Node.js for the guards UI build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add CBM_ALLOWED_ROOT to the environment-variable tables in README.md and
docs/CONFIGURATION.md: when set, index_repository refuses a repo_path that
resolves outside it; unset imposes no restriction.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel
wargloom
刘冲
Nohj
soren
Andrew Barnes