mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-04 10:48:20 +00:00
feat(index): add worker resource watchdogs
Discovery limits bound what indexing accepts, not what it then costs. A repository well inside those bounds can still exhaust the host through parser memory, or simply never finish, and a supervised worker that hangs leaves the parent waiting with nothing to report. Add index_max_rss_mb and index_max_duration_seconds, enforced by the parent against the worker process tree rather than the worker process alone, so a runaway child cannot hide behind a small parent. Resident memory is sampled through the platform interface on macOS, Linux and Windows. Crossing a limit terminates the tree and yields one trusted, structured terminal result that attributes the failure to the resource that caused it. Both limits default to off. A measurement that cannot be taken fails the attempt instead of passing it: a watchdog that quietly stops watching is worse than no watchdog at all. The shell fixture that stands in for the supervisor names the two new keys. The worker accepts only a policy that spells out every key it knows, which is what keeps a stale supervisor from starting a worker it cannot bound. Signed-off-by: 刘冲 <mail@liuchong.dev>
This commit is contained in:
21 files changed
+1472
-111
No files matched your search
@@ -1,8 +1,8 @@
|
||||
# Index resource limits
|
||||
|
||||
Index resource limits are optional operator controls for repositories whose
|
||||
discovery breadth is not known in advance. They are disabled by default so
|
||||
existing large-repository workloads retain their current behavior.
|
||||
discovery breadth or worker runtime is not known in advance. They are disabled
|
||||
by default so existing large-repository workloads retain their current behavior.
|
||||
|
||||
## Discovery settings
|
||||
|
||||
@@ -23,6 +23,39 @@ Values use base-10 integers. MiB means 1,048,576 bytes. Empty values, zero,
|
||||
negative values, suffixes, trailing characters, and values outside the stated
|
||||
ranges are rejected without changing the stored value.
|
||||
|
||||
## Worker settings
|
||||
|
||||
| Key | Default | Accepted value | Protects |
|
||||
|---|---:|---:|---|
|
||||
| `index_max_rss_mb` | `off` | `off` or `64..1048576` | Current RSS of the complete worker process tree |
|
||||
| `index_max_duration_seconds` | `off` | `off` or `1..86400` | Total worker wall-clock duration |
|
||||
|
||||
```bash
|
||||
codebase-memory-mcp config set index_max_rss_mb 8192
|
||||
codebase-memory-mcp config set index_max_duration_seconds 3600
|
||||
```
|
||||
|
||||
RSS is the current resident memory of the contained worker and every descendant,
|
||||
not the worker's allocation budget and not peak memory. This hard watchdog is
|
||||
separate from the internal `CBM_MEM_BUDGET_MB` soft budget. The supervisor
|
||||
samples RSS at most once every 250 milliseconds so the watchdog does not turn
|
||||
full process-table enumeration into a busy loop.
|
||||
|
||||
Duration uses a monotonic clock from successful spawn. It is independent of the
|
||||
existing 15-minute quiet timeout: continuous log progress does not reset total
|
||||
duration, while the quiet timeout continues to identify a worker that stops
|
||||
making progress.
|
||||
|
||||
Equality is allowed. The first RSS or elapsed-duration observation above its
|
||||
limit starts the existing graceful-to-force process-tree shutdown. CBM reports
|
||||
terminal only after the tree is quiescent or a bounded containment failure is
|
||||
explicitly surfaced. Resource termination is not retried and does not
|
||||
quarantine a source file.
|
||||
|
||||
If RSS is enabled and three consecutive probes cannot obtain any trustworthy
|
||||
tree measurement while the root worker is still running, CBM fails closed with
|
||||
`code=resource_probe_failed`.
|
||||
|
||||
## Counting and failure semantics
|
||||
|
||||
`index_max_files` counts a file only after it passes directory pruning, ignore
|
||||
@@ -55,6 +88,12 @@ The previous database remains available because publication occurs only after a
|
||||
complete discovery and successful staged build. If no previous database exists,
|
||||
`serving_index_preserved` is false.
|
||||
|
||||
Worker limit failures use the same shape with `stage=worker`,
|
||||
`resource=rss_bytes` and `unit=bytes`, or `resource=duration_ms` and
|
||||
`unit=milliseconds`. RSS measurement failures use `code=resource_probe_failed`
|
||||
and omit `observed`, `limit`, and `unit` because no trustworthy observation was
|
||||
available.
|
||||
|
||||
## Trust and compatibility
|
||||
|
||||
Limits are read from the CLI-managed `_config.db`; they are not MCP request
|
||||
@@ -64,5 +103,6 @@ parent policy.
|
||||
|
||||
These settings do not replace or increase `auto_index_limit`, change the 512 MiB
|
||||
single-file cap, alter workspace-root authorization, or affect
|
||||
`cross-repo-intelligence`. With both settings `off`, discovery follows the
|
||||
existing unbounded path.
|
||||
`cross-repo-intelligence`. With all settings `off`, discovery follows the
|
||||
existing path and the supervisor performs no periodic RSS probe or total-duration
|
||||
termination.
|
||||
Reference in new issue
Block a user