diff --git a/README.md b/README.md index 5135ca27f..0351d28e1 100644 --- a/README.md +++ b/README.md @@ -470,9 +470,9 @@ Published backend benchmarks on Apple M3 Pro. These are separate from the CBM Le **RAM-first pipeline**: All indexing runs in memory (LZ4 HC compressed read, in-memory SQLite, single dump at end). Memory is released back to the OS after indexing completes. -**Token efficiency, within the reported workloads**: The project's [original README report](https://github.com/DeusData/codebase-memory-mcp/blob/4ff6feeab639c4668da0f491930326ed5ec63093/README.md#performance) lists ~3,400 tokens for five structural queries via codebase-memory-mcp versus ~412,000 via file-by-file exploration: approximately **120× fewer tokens** for that comparison. The separate [31-repository preprint](https://arxiv.org/html/2603.27277v1#S4.SS1) reports **10× lower token use** and **2.1× fewer tool calls**, with answer-quality scores of **83% for CBM versus 92% for file-by-file exploration**. These are scoped backend measurements, not a promised saving for every task or a benchmark of CBM Lens. Actual savings depend on the question, repository, agent, and required verification. The study's [system-performance measurements](https://arxiv.org/html/2603.27277v1#S4.SS3) include the Linux indexing and Cypher timings above. +**More focused context, fewer tokens**: The project's [31-repository preprint](https://arxiv.org/html/2603.27277v1#S4.SS1) reports **10× lower token use** and **2.1× fewer tool calls** compared with file-by-file exploration. These March 2026 backend measurements cover the study's workloads and agent setup; the paper documents the full evaluation. An earlier [five-query comparison](https://github.com/DeusData/codebase-memory-mcp/blob/4ff6feeab639c4668da0f491930326ed5ec63093/README.md#performance) reported ~3,400 versus ~412,000 tokens for structural queries. The study's [system-performance measurements](https://arxiv.org/html/2603.27277v1#S4.SS3) include the Linux indexing and Cypher timings above. -To measure comparable quality, latency, and agent-efficiency metrics on your own workload, see [Measuring quality, latency, and agent savings](docs/MEASURING_SAVINGS.md). Exact reproduction of the figures above requires the original inputs and raw artifacts. +Explore the [measurement guide](docs/MEASURING_SAVINGS.md) to evaluate quality, latency and token use on your own workload. ## Troubleshooting & Diagnostics diff --git a/docs/index.html b/docs/index.html index 5511ba9a5..46ec36e4f 100644 --- a/docs/index.html +++ b/docs/index.html @@ -344,7 +344,7 @@
CBM LENS BETAcodebase-memory-mcp / architectureLOCAL WORKSPACE
- CBM Lens architecture map showing the source folders of codebase-memory-mcp + CBM Lens architecture walkthrough: explore the source folders, open src/pipeline, select artifact.c and read its source
CBM Lens, exploring this repository. Real product captures from October 2026. Capture details ↗
@@ -399,7 +399,7 @@
@@ -464,13 +464,13 @@
-

Performance and research

More useful context.
Less repeated work.

Reuse the graph to find relevant symbols, follow relationships and retrieve focused source snippets. Compact, paged results help keep repeated repository exploration out of your agent’s context window.

Our March 2026 preprint reports 10× lower token use and 2.1× fewer tool calls across 31 repositories compared with file-by-file exploration. Reported answer-quality scores were 83% for Codebase Memory and 92% for the file-exploration agent.

These historical backend results predate CBM Lens. Savings vary with the task, repository, agent and source verification needed.

+

Performance and research

More useful context.
Less repeated work.

Reuse the graph to find relevant symbols, follow relationships and retrieve focused source snippets. Compact, paged results help keep repeated repository exploration out of your agent’s context window.

Our March 2026 preprint reports 10× lower token use and 2.1× fewer tool calls across 31 repositories compared with file-by-file exploration.

These measurements cover the study’s backend workloads from March 2026. See the paper for the evaluation setup and full results.

Common questions

Is Codebase Memory
right for your workflow?

When should I use Codebase Memory?

Choose CBM when you want your coding agent and your own source-navigation workspace to use the same local graph of code, docs and infrastructure. It brings structural search, source evidence, CBM Lens, agent integrations and experimental test selection into one open-source package. Native releases run on macOS, Linux and Windows without Docker or a language runtime. Explore the capabilities and Smart CI’s current scope.

-
How does Codebase Memory reduce AI token use?

Your agent can query indexed relationships and retrieve the relevant symbols or source snippets instead of repeatedly exploring whole files. Compact results and pagination control how much context each query returns. The benefit depends on the task; read the measured token savings and quality tradeoff.

+
How does Codebase Memory reduce AI token use?

Your agent can query indexed relationships and retrieve the relevant symbols or source snippets instead of repeatedly exploring whole files. Compact results and pagination control how much context each query returns. The benefit depends on the task; explore the published token-use measurements.

Does Codebase Memory send my source code anywhere?

The backend indexes source and stores and queries its graph locally. Your coding agent has its own data-handling settings. Lens can optionally download a browser model after you enable it, with inference on-device. Installers and update checks use the network to obtain releases and assets.

How does the index stay current?

Incremental indexing updates changed files. With auto-watch enabled, an eligible active MCP session can watch an already indexed project. Automatic first indexing is a separate opt-in setting. The local UI daemon is not a promise of a permanent background watcher.

How much language coverage is available?

Bundled tree-sitter grammars cover a broad range of languages and formats. Supported language families also have Hybrid LSP semantic resolution. Extraction depth, type resolution and framework relationships vary; the language support matrix describes the details.

@@ -487,7 +487,7 @@ View all releases on GitHub →
- +