feat(index): the documentation pass follows the index mode by default

Doc-section -> code candidates (doc_link_candidates, #2574/#2610) are the
documentation-in-the-graph feature of this release. They now run by
default for full indexes and not for moderate or fast ones; an explicit
`doc_candidates` argument, or a `doc_candidates` setting the user set,
overrides the default either way in every mode.

- pipeline: the switch is unset/on/off; unset means on only when the
  effective mode is full.
- MCP: forwards only an explicit choice (argument, then a user-set config
  value) to daemon jobs and workers; daemon auto-index and watcher/UI jobs
  now carry a user-set value too (their workers have no config, so
  `doc_candidates false` was ignored there).
- fast + doc_candidates on runs the semantic pass's corpus phases and the
  doc phase only: candidates, and still no SIMILAR_TO or
  SEMANTICALLY_RELATED edges.
- config: `doc_candidates` joins the config key table (default `auto`);
  `config set doc_candidates ...` was refused as an unknown key before.
  README, docs/CONFIGURATION.md and the MCP schema describe the default.

Proof (macOS arm64, django fb11376):
  full 11,304 rows · moderate 0 · fast 0 · full+off 0 · moderate+on 1 ·
  fast+on 1 (moderate/fast discovery skips docs/) -- nodes and edges are
  identical between on and off in every mode; fast+on adds 0 semantic
  edges.
- New tests (RED on the old code): pipeline
  pipeline_doc_candidates_default_follows_mode,
  pipeline_doc_candidates_explicit_wins_over_mode,
  pipeline_doc_candidates_fast_on_legacy_incremental; mcp
  tool_index_repository_doc_candidates_follow_mode,
  tool_index_repository_doc_candidates_forwarded_only_when_explicit;
  daemon_application
  daemon_application_programmatic_index_forwards_set_doc_candidates.
- mcp 374 passed (4 Windows-only skips), daemon_application 73 passed,
  pipeline/store_arch/cli/runtime_settings/index_policy/pipeline_frozen
  green on the release tree.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel committed 2026-10-13 18:33:40 +02:00
1 parent bb636d1ff7
commit 859294561a
15 files changed
+608 -75

No files matched your search

+1 -1
View File
@@ -88,7 +88,7 @@ Current keys:
| `auto_index` | `false` | Automatically index new projects when an MCP session starts. |
| `auto_index_limit` | `50000` | Maximum file count allowed for automatic indexing of a new project. |
| `auto_watch` | `true` | Register the session's project with the background git watcher on connect. Set `false` to keep a session from registering its project (the watcher still runs for other projects). |
| `doc_candidates` | `true` | Score documentation sections against code to store doc↔code candidates (the possible targets of a section, and the sections that may describe a function or file, each with a judged probability). The scoring grows with sections × functions: under a second up to ~100k functions, ~15 s at ~400k, ~3 minutes on the Linux kernel. Set `false` to skip it; no candidates are stored and the rest of the graph is unchanged. `index_repository` also takes `doc_candidates` for a single index, which wins over this setting. Applies from the next index. |
| `doc_candidates` | `auto` | Score documentation sections against code to store doc↔code candidates (the possible targets of a section, and the sections that may describe a function or file, each with a judged probability). Unset (`auto`), the index mode decides: on for `full` indexes (the default mode), off for `moderate` and `fast`. Set `true` or `false` to choose for every mode; `config reset doc_candidates` returns to `auto`. The scoring grows with sections × functions: under a second up to ~100k functions, ~15 s at ~400k, ~3 minutes on the Linux kernel. Off stores no candidates and leaves the rest of the graph unchanged; on in `fast` mode stores candidates without adding `fast`'s skipped `SIMILAR_TO` / `SEMANTICALLY_RELATED` edges. `index_repository` also takes `doc_candidates` for a single index, which wins over this setting. Applies from the next index. |
| `watcher_enabled` | `true` | Master switch for the background watcher subsystem. Set `false` to stop the watcher from starting at all — no poll thread and no project registration. Reindex manually with `index_repository` when disabled. Only an open MCP session's own project is ever watched; `index_status` reports the current state in its `watch` object. |
| `watch_non_git` | `false` | Also poll project roots that are **not git repositories**. By default the watcher only follows git projects, so a project indexed from a plain directory is never refreshed after its first index — reindex it manually with `index_repository`. Set `true` to poll such roots on the same adaptive cadence with a file-tree scan: the indexer's own discovery walk (same skip lists, `.gitignore` and `.cbmignore` rules) hashed over each file's path, size and mtime. Any change reindexes once; paths the indexer skips (including cbm's own `.codebase-memory/` output) never trigger. The first poll after the daemon starts reindexes each such project once, since nothing records which tree state the index holds. The scan walks the whole tree every poll, so it costs more than git polling on very large trees. Read once when the daemon starts, like `watcher_enabled`. |
| `index_max_files` | `off` | Optional maximum number of accepted source files in one discovery run. |