fix(build): stop blob regeneration from reverting the ELF stack hardening

vendored/nomic/code_vectors_blob.S is a GENERATED file that was hand-edited.
1f674c80 added the ELF .note.GNU-stack section to it after finding that every
Linux binary we had ever shipped requested an executable stack: an object
carrying no such note tells GNU ld nothing about its stack requirement, and ld
then assumes the worst for the entire link. That fix went into the artifact
only. scripts/extract_nomic_vectors.py still emitted the Mach-O branch alone,
so the next regeneration would have overwritten the tracked .S, dropped both
the ELF and COFF branches, and put GNU_STACK RWE back into every release --
silently, because nothing compared the generator against its own output.

write_blob_s() now emits the tracked wrapper verbatim, and the new contract
test fails if the two ever drift again. The template is compared rather than
executed so the test does not need the torch/transformers import the extraction
script pulls in. The write is pinned to UTF-8 because the WHY comment carries
an em dash and the default encoding is locale-dependent.

check-binary-composition.sh already fails a release whose binary has an
executable stack; this closes the same hole at edit time, where the diff is
still small enough to read.

Reported and originally fixed by Anand Aiyer in #1151, which also carried the
COFF branch. That PR sat unreviewed for a month while the .S half was
rediscovered and landed independently without credit.

Co-authored-by: Anand Aiyer <anand@aiyer.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

scripts/extract_nomic_vectors.py

@@ -332,9 +332,18 @@ static inline const int8_t *pretrained_vec_at(int i) {{
 
 
 def write_blob_s(path: str, incbin_path: str):
-    """Write assembler .incbin directive."""
-    with open(path, "w") as f:
-        f.write(f"""/* nomic-embed-code vector blob embedded via assembler. */
+    """Write the assembler .incbin wrapper for every target object format.
+
+    Must stay byte-identical to the tracked vendored/nomic/code_vectors_blob.S:
+    tests/test_nomic_blob_generator_contract.sh fails if the two drift. The ELF
+    branch's .note.GNU-stack is load-bearing -- regenerating without it puts an
+    executable stack back into every Linux release binary.
+    """
+    with open(path, "w", encoding="utf-8") as f:
+        f.write(f"""/* nomic-embed-code vector blob embedded via assembler.
+ * Cross-platform: macOS (Mach-O) vs Linux (ELF) vs Windows (COFF). */
+
+#if defined(__APPLE__)
     .section __DATA,__const
     .globl _PRETRAINED_VECTOR_BLOB
     .globl _PRETRAINED_VECTOR_BLOB_LEN
@@ -347,6 +356,46 @@ _PRETRAINED_VECTOR_BLOB_END:
     .p2align 2
 _PRETRAINED_VECTOR_BLOB_LEN:
     .long _PRETRAINED_VECTOR_BLOB_END - _PRETRAINED_VECTOR_BLOB
+
+#elif defined(_WIN32) || defined(__CYGWIN__) || defined(__MINGW32__)
+    .section .rdata,"dr"
+    .globl PRETRAINED_VECTOR_BLOB
+    .globl PRETRAINED_VECTOR_BLOB_LEN
+    .p2align 4
+PRETRAINED_VECTOR_BLOB:
+    .incbin "{incbin_path}"
+PRETRAINED_VECTOR_BLOB_END:
+
+    .section .rdata,"dr"
+    .p2align 2
+PRETRAINED_VECTOR_BLOB_LEN:
+    .long PRETRAINED_VECTOR_BLOB_END - PRETRAINED_VECTOR_BLOB
+
+#else
+    /* WHY: an ELF object that carries no .note.GNU-stack tells the linker
+     * nothing about its stack requirement, and GNU ld then assumes the WORST
+     * for the whole link — every Linux release binary shipped GNU_STACK RWE
+     * because of this one omission. This is the only assembly source in the
+     * build, so it alone decided that property. The note must stay even though
+     * a blob of constant data obviously never executes: absence is the signal,
+     * not the contents. -Wl,-z,noexecstack in the link flags enforces the
+     * outcome, and scripts/ci/check-binary-composition.sh fails the release if
+     * an executable stack ever comes back. */
+    .section .note.GNU-stack,"",@progbits
+
+    .section .rodata,"a",@progbits
+    .globl PRETRAINED_VECTOR_BLOB
+    .globl PRETRAINED_VECTOR_BLOB_LEN
+    .p2align 4
+PRETRAINED_VECTOR_BLOB:
+    .incbin "{incbin_path}"
+PRETRAINED_VECTOR_BLOB_END:
+
+    .section .rodata,"a",@progbits
+    .p2align 2
+PRETRAINED_VECTOR_BLOB_LEN:
+    .long PRETRAINED_VECTOR_BLOB_END - PRETRAINED_VECTOR_BLOB
+#endif
 """)
     print(f"  {path}: written")
 

scripts/test.sh

@@ -218,6 +218,9 @@ bash "$ROOT/tests/test_windows_bundle_contract.sh"
 echo "=== Step 0f: tree-sitter runtime Makefile dependencies ==="
 bash "$ROOT/tests/test_makefile_ts_runtime_dependencies.sh"
 
+echo "=== Step 0f2: nomic blob generator contract ==="
+bash "$ROOT/tests/test_nomic_blob_generator_contract.sh"
+
 echo "=== Step 0g: security fuzz harness self-test ==="
 bash "$ROOT/tests/test_security_fuzz_harness.sh"
 

tests/test_nomic_blob_generator_contract.sh

@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Contract: scripts/extract_nomic_vectors.py must still emit the tracked
+# vendored/nomic/code_vectors_blob.S byte for byte.
+#
+# WHY this exists: the .S is a GENERATED file that was hand-edited to add the
+# ELF .note.GNU-stack section. Without that note GNU ld assumes the whole link
+# needs an executable stack, and every Linux release binary shipped GNU_STACK
+# RWE because of the omission. The generator kept emitting only the Mach-O
+# branch, so the next regeneration would have silently reverted the hardening
+# and reintroduced the defect. check-binary-composition.sh catches that at
+# release time; this catches it at edit time.
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+GEN="$ROOT/scripts/extract_nomic_vectors.py"
+TRACKED="$ROOT/vendored/nomic/code_vectors_blob.S"
+
+for f in "$GEN" "$TRACKED"; do
+    if [ ! -f "$f" ]; then
+        echo "FAIL: missing $f" >&2
+        exit 1
+    fi
+done
+
+# The generator holds the .S as an f-string whose only placeholder is the
+# incbin path, so the tracked file with that path re-parameterized must appear
+# in it verbatim. Comparing the template rather than running the generator
+# keeps this test free of the torch/transformers import the script needs.
+python3 - "$GEN" "$TRACKED" <<'PYEOF'
+import sys
+
+gen_path, tracked_path = sys.argv[1], sys.argv[2]
+
+with open(gen_path, encoding="utf-8") as fh:
+    generator = fh.read()
+with open(tracked_path, encoding="utf-8") as fh:
+    tracked = fh.read()
+
+INCBIN = "vendored/nomic/code_vectors.bin"
+if INCBIN not in tracked:
+    sys.exit("FAIL: %s no longer references %s" % (tracked_path, INCBIN))
+if "{" in tracked or "}" in tracked:
+    sys.exit("FAIL: %s gained a brace; the generator f-string would need "
+             "escaping and this contract can no longer compare them directly"
+             % tracked_path)
+
+template = tracked.replace(INCBIN, "{incbin_path}")
+
+if template not in generator:
+    sys.exit(
+        "FAIL: scripts/extract_nomic_vectors.py no longer emits the tracked\n"
+        "      vendored/nomic/code_vectors_blob.S.\n"
+        "      Regenerating the blob wrapper would overwrite the tracked file\n"
+        "      with different content. Update write_blob_s() so its f-string\n"
+        "      matches the tracked .S with the incbin path parameterized.")
+
+# The note is the whole point; assert it explicitly so a future edit that keeps
+# the files in sync while dropping the note still fails loudly.
+NOTE = '.section .note.GNU-stack,"",@progbits'
+for label, blob in (("tracked .S", tracked), ("generator template", generator)):
+    if NOTE not in blob:
+        sys.exit("FAIL: %s lost the ELF %s section -- Linux binaries would ship "
+                 "an executable stack again" % (label, NOTE))
+
+print("ok: generator template is byte-identical to the tracked blob wrapper")
+PYEOF