nprasad2077/codebase-memory-mcp

0 Fork 0

mirror of https://github.com/DeusData/codebase-memory-mcp.git synced 2026-11-04 10:48:20 +00:00

Files

T

Martin VogelandFelix Rath f9f44b5869 fix(mem): vendor mimalloc 3.5.3 so free(NULL) before init is safe

With mimalloc 3.3.2's release build, free(NULL) before the allocator's
constructor dereferenced a page map that does not exist yet. glibc 2.44's
newlocale() does exactly that during libstdc++ start-up, so every Linux
binary died with SIGSEGV before main (reported with a reproduction in
#2562). Upstream fixed it in b2226c19 (mimalloc #1341), first released in
v3.4.4; v3.5.3, upstream's latest published release, contains it.

Vendored from microsoft/mimalloc tag v3.5.3 (tag object deac3dcf, commit
d4881d338125e1cb7c47ba4cfb398d6f7c0c8d45): LICENSE, include/ and src/,
written blob by blob from a bare clone. Before the local patches were
reapplied, the staged LICENSE blob and include/ and src/ trees had the
tag's exact object ids (50 files; src/arena-meta.c is gone upstream).
Afterwards the only differences are two documented hunks:

- src/prim/unix/prim.c: macOS always recommits on decommit, carried from
  63fb7b3d (the phys_footprint fix).
- src/prim/osx/alloc-override-zone.c: one comment reworded, because
  scripts/security-vendored.sh rejects the upstream "fork()" spelling
  even inside a comment. No code change.

The reproducible-banner patch (no __DATE__/__TIME__) is dropped: upstream
removed the build clock in v3.4.5 (mimalloc #1355). The two Makefile
comments that pointed at that patch now say so.

The production object is built with -DMI_PROFILE=0: mimalloc 3.5.2+
compiles its allocation-sampling profiler in by default, with a sample
countdown on the allocation path, and the product never uses it.

vendored/mimalloc/PATCHES.md records the pin, a re-verification recipe,
the check that pins each patch and the refresh procedure; THIRD_PARTY.md
points at it, the SBOM version is 3.5.3 and scripts/vendored-checksums.txt
is regenerated (mimalloc lines only; security-vendored.sh passes).

Measured on Apple M5 Pro (macOS 26.6), full indexes, 2 interleaved reps,
main fceb25d8 (3.3.2) -> this change: TypeScript peak phys_footprint
1.07/1.01 -> 1.04/1.04 GB, peak RSS 1.87/1.84 -> 1.85/1.86 GB, wall
30.6/29.1 -> 29.3/30.0 s; django footprint 0.41/0.41 -> 0.38/0.39 GB,
RSS 0.59/0.59 -> 0.58/0.60 GB, wall 6.5/6.2 -> 6.5/6.5 s. The graphs are
identical: full node and edge hashes, node and edge census, no dangling
edges (django 68022 nodes / 322464 edges, TypeScript 306239 / 835984).
The same tree without the recommit patch peaks at 2.09/2.04 GB footprint
on TypeScript, so 63fb7b3d still does its job. Linux arm64 (glibc 2.39,
gcc 13.3): both trees build without warnings; django peak RSS 0.67/0.68
GB on main, 0.69/0.68 GB with this change.

Co-authored-by: Felix Rath <felixm.rath@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

2026-10-21 00:33:51 +02:00

4.3 KiB

Raw Blame History

Vendored mimalloc: source pin and local patches

Source pin

Upstream: https://github.com/microsoft/mimalloc
Tag: v3.5.3 (annotated tag object deac3dcf5f0614884013d411a5c07e16bec96c9f)
Commit: d4881d338125e1cb7c47ba4cfb398d6f7c0c8d45 (tree fbc4e8f72287d127c079ada5ede870b7d63e55f5)
Vendored: LICENSE, every file under include/ and every file under src/. Nothing else (no build files, tests or docs).

v3.5.3 is upstream's latest published release at the time of the pin (GitHub release of 2026-09-17, on the main3 branch).

v3.5.3 contains upstream commit b2226c195fe884223f6d6f68ea52df476e57e5c2 (mimalloc #1341, first released in v3.4.4): free(NULL) must work before the allocator is initialized. With 3.3.2's release build, glibc 2.44's newlocale() freed NULL during libstdc++ start-up, before mimalloc's constructor, and every Linux binary died with SIGSEGV (reported with a reproduction by Felix Rath in #2562). The preinit fixture in tests/test_prod_allocator_contract.sh pins it.

Verifying the vendored bytes

Every vendored file is byte-identical to the tag except the two hunks below. Re-check from a fresh bare clone (nothing in it is built or run):

git clone --bare https://github.com/microsoft/mimalloc.git /tmp/mimalloc.git
git --git-dir=/tmp/mimalloc.git rev-parse 'v3.5.3^{commit}'   # d4881d33...
for f in $(git --git-dir=/tmp/mimalloc.git ls-tree -r --name-only v3.5.3 -- LICENSE include src); do
  [ "$(git --git-dir=/tmp/mimalloc.git rev-parse "v3.5.3:$f")" = \
    "$(git hash-object "vendored/mimalloc/$f")" ] || echo "differs: $f"
done

The expected output is exactly the two files named below, and vendored/mimalloc/include plus vendored/mimalloc/src must hold no file the tag lacks.

Local patches (reapply on every refresh)

Each patch is pinned by a check that fails when the patch is dropped.

1. macOS decommit always recommits: `src/prim/unix/prim.c`

In _mi_prim_decommit, macOS takes the recommit path that upstream reserves for MI_DEBUG / MI_SECURE>2 builds, so a purged range is not re-accounted as a whole by MADV_FREE_REUSE when the arena reuses it. This cut the indexer's peak phys_footprint substantially (measurements in the inline comment and in commit 63fb7b3d; re-measured on v3.5.3 when this pin was taken). Linux and Windows keep upstream's behaviour.

Pinned by: the decommit fixture in tests/test_prod_allocator_contract.sh (needs_recommit must be true on macOS and upstream's false elsewhere).

2. Comment wording: `src/prim/osx/alloc-override-zone.c`

Comment only, no code change: "crash in fork()." reads "crash while forking.". scripts/security-vendored.sh rejects call spellings such as fork( anywhere in vendored C, comments included, so the upstream sentence fails that gate. (Wording first proposed in #2562.)

Pinned by: scripts/security-vendored.sh itself, which reports the pristine upstream line as a blocked subprocess call.

Build configuration

Makefile.cbm compiles the production object with -DMI_PROFILE=0, which leaves out mimalloc's allocation-sampling profiler (on by default since 3.5.2) and its sample countdown on the allocation path. Pinned by: the banner fixture in tests/test_prod_allocator_contract.sh (a profiling build prints a profiling line). The test build keeps upstream's default.

Dropped patches

Reproducible version banner (src/options.c, carried from 1f674c80 up to 3.4.4): upstream removed __DATE__/__TIME__ from the banner in v3.4.5 (mimalloc #1355). The banner fixture in tests/test_prod_allocator_contract.sh still asserts that the first line is exactly vX.Y.Z with no build clock, so a future refresh cannot bring it back unnoticed.

Refresh procedure

Replace LICENSE, include/ and src/ with the new tag's files, deleting files the tag no longer has, and verify the staged tree against the tag before patching.
Reapply the patches above, or drop the ones upstream has absorbed (and move them to "Dropped patches" with the upstream reference).
Update the pin in this file and versionInfo in scripts/ci/generate-sbom.py, then run scripts/security-vendored.sh --update.
After make -f Makefile.cbm cbm, run bash tests/test_prod_allocator_contract.sh build/c on Linux and macOS (scripts/test.sh runs it on every CI leg), and scripts/security-vendored.sh.

4.3 KiB Raw Blame History