Merge origin/main into feat/index-resource-worker

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

docs/CONFIGURATION.md

@@ -87,6 +87,7 @@ Current keys:
 |---|---|---|
 | `auto_index` | `false` | Automatically index new projects when an MCP session starts. |
 | `auto_index_limit` | `50000` | Maximum file count allowed for automatic indexing of a new project. |
+| `ignore_worktrees` | `false` | Skip linked git worktrees (`git worktree add`) when indexing automatically. |
 | `auto_watch` | `true` | Global switch for MCP session watch registration. On the next refresh or registration request, `true` allows session roots and successful `index_repository` targets to be watched; `false` removes all watches. The background watcher can still run with no registered projects. |
 | `doc_candidates` | `auto` | Score documentation sections against code to store doc↔code candidates (the possible targets of a section, and the sections that may describe a function or file, each with a judged probability). Unset (`auto`), the index mode decides: on for `full` indexes (the default mode), off for `moderate` and `fast`. Set `true` or `false` to choose for every mode; `config reset doc_candidates` returns to `auto`. The scoring grows with sections × functions: under a second up to ~100k functions, ~15 s at ~400k, ~3 minutes on the Linux kernel. Off stores no candidates and leaves the rest of the graph unchanged; on in `fast` mode stores candidates without adding `fast`'s skipped `SIMILAR_TO` / `SEMANTICALLY_RELATED` edges. `index_repository` also takes `doc_candidates` for a single index, which wins over this setting. Applies from the next index. |
 | `watcher_enabled` | `true` | Master switch for the background watcher subsystem. Set `false` to stop the watcher from starting at all — no poll thread and no project registration. Reindex manually with `index_repository` when disabled. Active MCP sessions can register their root project and successful explicit index targets; under `daemon start` those watches outlive the session until the daemon stops, the project is deleted, its directory disappears, or `auto_watch` is set to `false`. `index_status` reports the current state in its `watch` object. |
@@ -146,6 +147,44 @@ duration is independent of the existing 15-minute no-log-progress timeout. See
 [Index resource limits](INDEX_RESOURCE_LIMITS.md) for counting, validation, and
 error-response details.
 
+### `ignore_worktrees`
+
+Every indexed project is registered under its own absolute root path, so each
+linked worktree becomes a separate permanent index. On machines that create many
+short-lived worktrees, the automatic paths (`auto_index`, and the session hook's
+"index this project first" guidance) turn every throwaway checkout into another
+stored index of what is largely the same repository.
+
+Enable the key to keep those checkouts out of the index:
+
+```bash
+codebase-memory-mcp config set ignore_worktrees true
+```
+
+With it enabled:
+
+- automatic indexing skips a session whose root is a linked worktree;
+- the `hook-augment` context says the worktree is unindexed on purpose instead
+  of telling the agent to run `index_repository`;
+- an explicit `index_repository` call on a linked worktree is refused, and names
+  both ways forward — pass `index_worktree=true` for that one call, or turn the
+  key back off.
+
+The main checkout of the same repository is unaffected, as are ordinary clones
+and submodules. Detection is git plumbing only: a linked worktree's `.git` is a
+file pointing at a gitdir that contains a `commondir` entry.
+
+A subdirectory counts as the worktree it sits in. Detection walks up from the
+given path to the nearest `.git` entry, so `index_repository` on any
+subdirectory of a linked worktree is refused the same way, and a session root or
+hook working directory inside one is treated as that worktree. The walk stops at
+the first `.git` it finds; a main checkout or a submodule ends it as "not a
+worktree". A path outside any git checkout is checked one directory at a time up
+to the filesystem root and is never treated as a worktree. None of this runs
+while the key is off.
+
+The default is `false`, so indexing behavior is unchanged unless you opt in.
+
 ## 3. UI Settings
 
 The optional built-in graph UI stores its settings in:

docs/EVALUATION_PLAN.md

@@ -274,8 +274,7 @@ then its index is **deleted** before the next language — so **only one project
 every index is genuinely cold. The skeleton:
 
 ```bash
-# Start clean: no leftover DBs, fresh results tree
-rm -f ~/.cache/codebase-memory-mcp/*.db
+# Start clean: fresh results tree
 mkdir -p /tmp/eval-results
 
 for lang in $ALL_LANGS; do                      # ALL_LANGS = full 159-name list (§2 / §8)
@@ -283,8 +282,13 @@ for lang in $ALL_LANGS; do                      # ALL_LANGS = full 159-name list
 
   # --- step 2: cold index in the main channel, TIMED (key metric) ---
   t0=$(now_ms)
+  CBM_BENCH_KEEP_RUNTIME=1 \
   scripts/benchmark-index.sh ~/.local/bin/codebase-memory-mcp "$lang" /tmp/bench/"$lang" /tmp/eval-results
   index_ms=$(( $(now_ms) - t0 ))                 # clone+index wall-clock → manifest + report (§5)
+  # The harness indexes into a run-private root (#1696). CBM_BENCH_KEEP_RUNTIME leaves that
+  # root behind on success and records its paths; the graph session in steps 4-7 must be
+  # started with these CBM_RUNTIME_DIR / CBM_CACHE_DIR, and step 8 removes the root.
+  set -a; . /tmp/eval-results/"$lang"/runtime-root.txt; set +a
 
   # --- step 3: record per-type histograms (zeros back-filled) ---
   #   node-types.json, edge-types.json  (every label + all 32 edge types, zeros kept, §7 below)
@@ -293,7 +297,14 @@ for lang in $ALL_LANGS; do                      # ALL_LANGS = full 159-name list
   #               + per-language report + (deferred, blind) judge ---
 
   # --- step 8: delete THIS language's index so the next is cold, then mark done ---
-  rm -f ~/.cache/codebase-memory-mcp/*.db
+  # The graph session (steps 4-7) may still have a daemon running under the kept
+  # CBM_RUNTIME_DIR (exported above); stop it before its root goes away. A refused
+  # stop (a session still attached) means the root is still in use: keep it.
+  if ~/.local/bin/codebase-memory-mcp daemon stop; then
+    rm -rf -- "$CBM_BENCH_RUNTIME_ROOT"
+  else
+    echo "$lang: daemon under $CBM_RUNTIME_DIR did not stop; keeping $CBM_BENCH_RUNTIME_ROOT" >&2
+  fi
   manifest_mark_done "$lang" "$index_ms"
 done
 ```
@@ -860,10 +871,13 @@ Deep-Dive section.
 # 1. Clone all 159 repos (shallow; skip existing)
 scripts/clone-bench-repos.sh /tmp/bench
 
-# 2. Cold index all 159 (LSP cohort in full mode)
-rm -f ~/.cache/codebase-memory-mcp/*.db
+# 2. Cold index all 159 (LSP cohort in full mode). Each language is indexed into a run-private
+#    runtime and cache (#1696); CBM_BENCH_KEEP_RUNTIME=1 keeps that root after a successful
+#    index and records its paths in /tmp/eval-results/<lang>/runtime-root.txt. Source it
+#    before the graph phase, then stop the daemon and remove the root, as in §7 step 8.
 mkdir -p /tmp/eval-results
 for lang in $ALL_LANGS; do
+  CBM_BENCH_KEEP_RUNTIME=1 \
   scripts/benchmark-index.sh ~/.local/bin/codebase-memory-mcp "$lang" /tmp/bench/"$lang" /tmp/eval-results
 done