nprasad2077/codebase-memory-mcp
1
0 Fork 0
mirror of https://github.com/DeusData/codebase-memory-mcp.git synced 2026-11-04 18:58:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

perf(mem): recommit purged memory on macOS so phys_footprint drops

Browse Source 
81264814 built the production allocator as mimalloc's release build
(-DMI_DEBUG=0). That bought a large speed-up, but it raised what macOS
charges an index worker, its phys_footprint:

| peak phys_footprint | v0.11.0 | main |
|---|---:|---:|
| TypeScript | 1.25 GB | 1.94 GB |
| dotnet/runtime | 4.5 GB | 12.7 GB |

The 2026-10-09 bench missed most of this, because it sampled RSS.

The difference is one compile-time branch in mimalloc's unix decommit.
With MI_DEBUG or MI_SECURE>2, a purged range is mprotect(PROT_NONE)'d
and flagged needs_recommit. A release build leaves it "committed".
When the arena hands such a range out again, it is re-accounted as a
whole with MADV_FREE_REUSE (arena.c, _mi_os_reuse). macOS then charges
every page of it to the footprint, touched or not.

On macOS, take the recommit path in every build. Linux and Windows keep
upstream's behaviour; there MADV_DONTNEED already drops RSS at once.

Measured on Apple arm64 (macOS 26.6), main ec4fbd3a vs this change, cold
full indexes interleaved with main (C#: 2 runs each, in the same session
as main):

| corpus | footprint before | footprint after | change | wall time |
|---|---:|---:|---:|---|
| TypeScript | 1.98 GB | 1.09 GB | -45 % | unchanged |
| kubernetes | 2.92 GB | 1.88 GB | -35 % | unchanged |
| dotnet/runtime | 12.6 GB | 4.9 GB | -61 % | unchanged |
| linux kernel | 16.0 GB | 13.5 GB | -16 % | 574 -> 559 s |

The graph is byte-identical: TypeScript, kubernetes and django give
the same nodes, edges and CALLS-set hash.

dotnet/runtime is now close to v0.11.0's 4.5 GB.

Measured but not taken:
- the whole debug build, which costs ~5 % wall time on C#;
- MI_STAT=2, MI_PADDING=1 or MI_ENCODE_FREELIST=1 alone, none of which
  moves the footprint;
- MIMALLOC_PURGE_DELAY=0, which is already set.

Vendored file changed in place (precedent 1f674c80); its hash in
scripts/vendored-checksums.txt is updated and security-vendored.sh passes.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel committed 2026-10-10 16:38:53 +02:00
1 parent ec4fbd3a50
commit 63fb7b3db5
2 files changed
+13 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/vendored-checksums.txt
+1 -1
View File
@@ -1024,7 +1024,7 @@ ce14eb6dfd55f241095b55f70983931a2d92f7a6a622a9b360cdd70ab3b53208 vendored/mimal
247a9952465eb105be03a9962e922b085ce5d52034775551a61cd8594275be73 vendored/mimalloc/src/prim/osx/prim.c
cc771788e5ba591efb681829a8724f8662a1f50f3db39cf5ebe7738b8ed57e44 vendored/mimalloc/src/prim/prim.c
792531421c247fca0766ff7630057ec491173578ca4d1919ae34f05eb289ccf4 vendored/mimalloc/src/prim/readme.md
c1ae32eaac7ccf18c10058cca99723892f8bd027318590151f5011c929c7e3be vendored/mimalloc/src/prim/unix/prim.c
20594d38ef4fc0c641f20a8664d17ee03ce195ca21b44e036f43cdb7bcd0bd0d vendored/mimalloc/src/prim/unix/prim.c
bee9c655a17e1e93bf4908e0a902fd3b566628447b1446fcbfa92bca2e76ca15 vendored/mimalloc/src/prim/wasi/prim.c
f70e61a1f2fe63da2f51c581895cf4a0918a9bffb0cf8600171f297141446962 vendored/mimalloc/src/prim/windows/etw-mimalloc.wprp
4f3110ef2054c95cb275be96cb279224d3d46a728c5117bd1435425f382de778 vendored/mimalloc/src/prim/windows/etw.h