nprasad2077/codebase-memory-mcp

0 Fork 0

mirror of https://github.com/DeusData/codebase-memory-mcp.git synced 2026-11-04 10:48:20 +00:00

Files

T

刘冲 de1615264a feat(index): add worker resource watchdogs

Discovery limits bound what indexing accepts, not what it then costs. A
repository well inside those bounds can still exhaust the host through parser
memory, or simply never finish, and a supervised worker that hangs leaves the
parent waiting with nothing to report.

Add index_max_rss_mb and index_max_duration_seconds, enforced by the parent
against the worker process tree rather than the worker process alone, so a
runaway child cannot hide behind a small parent. Resident memory is sampled
through the platform interface on macOS, Linux and Windows. Crossing a limit
terminates the tree and yields one trusted, structured terminal result that
attributes the failure to the resource that caused it.

Both limits default to off. A measurement that cannot be taken fails the
attempt instead of passing it: a watchdog that quietly stops watching is
worse than no watchdog at all.

The shell fixture that stands in for the supervisor names the two new keys.
The worker accepts only a policy that spells out every key it knows, which is
what keeps a stale supervisor from starting a worker it cannot bound.

Signed-off-by: 刘冲 <mail@liuchong.dev>

2026-10-19 17:18:00 +08:00

4.3 KiB

Raw Blame History

Index resource limits

Index resource limits are optional operator controls for repositories whose discovery breadth or worker runtime is not known in advance. They are disabled by default so existing large-repository workloads retain their current behavior.

Discovery settings

Key	Default	Accepted value	Protects
`index_max_files`	`off`	`off` or `1..10000000`	Accepted source-file count
`index_max_source_mb`	`off`	`off` or `1..1048576`	Accepted source-file bytes

Set or reset them with the normal configuration command:

codebase-memory-mcp config set index_max_files 250000
codebase-memory-mcp config set index_max_source_mb 16384
codebase-memory-mcp config reset index_max_files

Values use base-10 integers. MiB means 1,048,576 bytes. Empty values, zero, negative values, suffixes, trailing characters, and values outside the stated ranges are rejected without changing the stored value.

Worker settings

Key	Default	Accepted value	Protects
`index_max_rss_mb`	`off`	`off` or `64..1048576`	Current RSS of the complete worker process tree
`index_max_duration_seconds`	`off`	`off` or `1..86400`	Total worker wall-clock duration

codebase-memory-mcp config set index_max_rss_mb 8192
codebase-memory-mcp config set index_max_duration_seconds 3600

RSS is the current resident memory of the contained worker and every descendant, not the worker's allocation budget and not peak memory. This hard watchdog is separate from the internal CBM_MEM_BUDGET_MB soft budget. The supervisor samples RSS at most once every 250 milliseconds so the watchdog does not turn full process-table enumeration into a busy loop.

Duration uses a monotonic clock from successful spawn. It is independent of the existing 15-minute quiet timeout: continuous log progress does not reset total duration, while the quiet timeout continues to identify a worker that stops making progress.

Equality is allowed. The first RSS or elapsed-duration observation above its limit starts the existing graceful-to-force process-tree shutdown. CBM reports terminal only after the tree is quiescent or a bounded containment failure is explicitly surfaced. Resource termination is not retried and does not quarantine a source file.

If RSS is enabled and three consecutive probes cannot obtain any trustworthy tree measurement while the root worker is still running, CBM fails closed with code=resource_probe_failed.

Counting and failure semantics

index_max_files counts a file only after it passes directory pruning, ignore rules, filename and suffix filters, language detection, and the existing per-file size rule. index_max_source_mb sums the filesystem sizes of that same accepted set.

Equality is allowed. The first file or byte that makes an observed value greater than its limit stops discovery. CBM discards the partial file list and does not publish a partial graph as a complete index.

For an explicit MCP request the error payload contains:

{
  "status": "error",
  "code": "resource_limit_exceeded",
  "stage": "discovery",
  "resource": "files",
  "observed": 250001,
  "limit": 250000,
  "unit": "files",
  "retryable": true,
  "serving_index_preserved": true,
  "message": "Index discovery exceeded index_max_files"
}

The previous database remains available because publication occurs only after a complete discovery and successful staged build. If no previous database exists, serving_index_preserved is false.

Worker limit failures use the same shape with stage=worker, resource=rss_bytes and unit=bytes, or resource=duration_ms and unit=milliseconds. RSS measurement failures use code=resource_probe_failed and omit observed, limit, and unit because no trustworthy observation was available.

Trust and compatibility

Limits are read from the CLI-managed _config.db; they are not MCP request arguments. A supervised parent replaces any caller-supplied internal policy before spawning its worker, and the worker rejects a missing or incomplete parent policy.

These settings do not replace or increase auto_index_limit, change the 512 MiB single-file cap, alter workspace-root authorization, or affect cross-repo-intelligence. With all settings off, discovery follows the existing path and the supervisor performs no periodic RSS probe or total-duration termination.

4.3 KiB Raw Blame History