ci(release): double-verify ML antivirus false positives instead of re-rolling builds

Release run 30464288732 was blocked by the VirusTotal gate: three linux-amd64
binaries flagged 1/62 by Microsoft's Wacatac.B!ml -- fully stripped binaries
(0 symbols, verified on the exact artifacts), the state that scanned clean in
the two previous cycles. Meanwhile a real Defender endpoint (engine
1.1.26060.3008, signatures 1.455.410.0 updated the same day, RTP on) scans
the identical bytes clean. Four cycles of evidence now say the same thing:
this verdict is an unstable ML decision boundary, not a property of the code,
and no build-side lever moves it durably -- stripping, downloader removal and
metadata changes each "worked" only until a later build flipped it back.

So stop treating the flag as buildable-away and verify it honestly instead:

  check-virustotal.sh may downgrade BLOCKED to TOLERATED only when ALL hold:
    - pre-release version (-rc./-pre/-alpha/-beta); stable releases never
    - every failing file flagged by exactly ONE engine
    - that engine is Microsoft and the verdict ends in "!ml" (never a
      signature name)
    - hash-pinned Defender ENDPOINT evidence is attached to the draft release
      (defender-endpoint-verification.txt) proving Microsoft's shipping
      product, signature-updated at scan time, reports the exact bytes clean

  av-endpoint-verify.sh (new) produces that evidence: downloads the draft
  assets, scans them on the local Windows VM endpoint, refuses to attest if
  RTP is off or Defender itself detects, uploads the hash-pinned result.
  The gate prints the exact command when evidence is missing; re-running the
  failed verify job does not rebuild, so the bytes stay fixed.

  append-vt-notes.sh (new, extracted from inline YAML per venue-parity) then
  renders the release-notes table honestly: a tolerated file reads "1/62 ML
  false positive, endpoint-verified clean", never "0 detections".

tests/test_vt_gate_tolerance_contract.sh pins all nine decision directions
against a stubbed VT API and release store -- clean pass, stable-never,
missing/stale/DETECTED evidence, signature-named verdict, non-Microsoft
engine, multi-engine -- so the tolerance provably fails closed.

Also: release.yml gains skip_tests for re-releases of an already test-green
tree (build/smoke/soak/verify always run; lint failures still gate via
!cancelled() && !failure()).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

.github/workflows/release.yml

@@ -31,6 +31,11 @@ on:
         required: false
         type: boolean
         default: false
+      skip_tests:
+        description: "Skip the test phase entirely (re-release of an already test-green tree; build/smoke/soak/verify still run)"
+        required: false
+        type: boolean
+        default: false
 
 permissions:
   contents: read
@@ -46,7 +51,12 @@ jobs:
     uses: ./.github/workflows/_lint.yml
 
   # ── 2. Tests (all platforms, full suite for release) ────────────
+  # skip_tests exists for RE-releases of a tree whose tests are already green:
+  # when only packaging/gating changed and the previous run proved the suites,
+  # re-running ~2h of tests adds no information. It skips ONLY this phase —
+  # build, smoke, soak and verify always run against the fresh artifacts.
   test:
+    if: ${{ !inputs.skip_tests }}
     needs: [lint]
     uses: ./.github/workflows/_test.yml
     with:
@@ -55,7 +65,10 @@ jobs:
       shard_suites: true
 
   # ── 3. Build all platforms ──────────────────────────────────────
+  # !cancelled() && !failure(): run when `test` is deliberately skipped, but
+  # never when lint or test actually failed.
   build:
+    if: ${{ !cancelled() && !failure() }}
     needs: [test]
     permissions:
       contents: read
@@ -220,44 +233,27 @@ jobs:
           vt_api_key: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
           files: binaries/*
 
+      # VERSION + GH_TOKEN enable the pre-release ML false-positive tolerance:
+      # the gate may downgrade a single-engine Microsoft "!ml" verdict to
+      # TOLERATED, but only for -rc./-pre versions and only when hash-pinned
+      # Defender ENDPOINT evidence is attached to the draft release
+      # (defender-endpoint-verification.txt, from scripts/ci/av-endpoint-verify.sh).
       - name: Wait for VirusTotal results
         env:
           VT_API_KEY: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
           VT_ANALYSIS: ${{ steps.virustotal.outputs.analysis }}
+          VERSION: ${{ inputs.version }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: scripts/ci/check-virustotal.sh
 
+      # Renders per-binary sha256 + VT links into the notes; when the gate
+      # tolerated an ML false positive it states that honestly instead of
+      # claiming "0 detections".
       - name: Append VirusTotal scan links to release notes
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           VERSION: ${{ inputs.version }}
-        run: |
-          # Hash the extracted binaries (not the archives — VT indexes binary hashes)
-          TABLE="\n\n## Security Verification\n\n"
-          TABLE+="All release binaries scanned with 70+ antivirus engines — **0 detections**.\n\n"
-          TABLE+="| Binary | SHA-256 | VirusTotal |\n"
-          TABLE+="|--------|---------|------------|\n"
-
-          for bin in binaries/codebase-memory-mcp-*; do
-            [ -f "$bin" ] || continue
-            name=$(basename "$bin")
-            # Skip UI variants and non-binary files
-            echo "$name" | grep -qE \
-              '^codebase-memory-mcp-(linux|darwin|windows)-(amd64|arm64)(\.exe)?$' || continue
-            sha256=$(sha256sum "$bin" 2>/dev/null | awk '{print $1}' \
-              || shasum -a 256 "$bin" | awk '{print $1}')
-            label=$(echo "$name" \
-              | sed 's/^codebase-memory-mcp-//' \
-              | sed 's/\.exe$//')
-            short="${sha256:0:20}..."
-            vt_url="https://www.virustotal.com/gui/file/${sha256}/detection"
-            TABLE+="| \`${label}\` | \`${short}\` | [0/72 ✅](${vt_url}) |\n"
-          done
-
-          CURRENT=$(gh release view "$VERSION" \
-            --json body --jq '.body // ""' --repo "$GITHUB_REPOSITORY")
-          printf '%s%b' "$CURRENT" "$TABLE" > /tmp/release_notes.md
-          gh release edit "$VERSION" \
-            --notes-file /tmp/release_notes.md --repo "$GITHUB_REPOSITORY"
+        run: scripts/ci/append-vt-notes.sh
 
   # ── 8. Publish package wrappers (npm + PyPI) ──────────────────
   # Wrappers in pkg/npm and pkg/pypi download the released binary at

scripts/ci/append-vt-notes.sh

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+# Append the Security Verification section to the release notes: per-binary
+# sha256 + VirusTotal links, and — when the pre-release ML false-positive
+# tolerance fired (scripts/ci/check-virustotal.sh wrote /tmp/vt_tolerated.tsv)
+# — an honest annotation instead of a blanket "0 detections" claim.
+# Expects: GH_TOKEN, VERSION; run from the verify job workspace (binaries/).
+set -euo pipefail
+
+TOL=/tmp/vt_tolerated.tsv
+
+TABLE="\n\n## Security Verification\n\n"
+if [ -s "$TOL" ]; then
+  N=$(wc -l < "$TOL" | tr -d ' ')
+  TABLE+="All release binaries were scanned with 70+ antivirus engines. "
+  TABLE+="**${N} of them currently show a single machine-learning heuristic detection** "
+  TABLE+="(Microsoft \`Wacatac!ml\`) that we have verified to be a false positive: "
+  TABLE+="an up-to-date Microsoft Defender endpoint — signatures updated at scan time, "
+  TABLE+="real-time protection on — scans the identical bytes clean. The hash-pinned "
+  TABLE+="evidence is attached to this release as \`defender-endpoint-verification.txt\`, "
+  TABLE+="and a false-positive report for these exact hashes has been submitted to "
+  TABLE+="Microsoft. See **Antivirus false positives on release binaries** in the notes "
+  TABLE+="above for the full story and what we are doing about it.\n\n"
+else
+  TABLE+="All release binaries scanned with 70+ antivirus engines — **0 detections**.\n\n"
+fi
+TABLE+="| Binary | SHA-256 | VirusTotal |\n"
+TABLE+="|--------|---------|------------|\n"
+
+for bin in binaries/codebase-memory-mcp-*; do
+  [ -f "$bin" ] || continue
+  name=$(basename "$bin")
+  sha256=$(sha256sum "$bin" 2>/dev/null | awk '{print $1}' \
+    || shasum -a 256 "$bin" | awk '{print $1}')
+  label=$(echo "$name" | sed 's/^codebase-memory-mcp-//' | sed 's/\.exe$//')
+  short="${sha256:0:20}..."
+  vt_url="https://www.virustotal.com/gui/file/${sha256}/detection"
+  if [ -s "$TOL" ] && grep -q "^${name}	" "$TOL"; then
+    completed=$(grep "^${name}	" "$TOL" | cut -f6)
+    cell="[1/${completed} ⚠️ ML false positive, endpoint-verified clean](${vt_url})"
+  else
+    cell="[0 detections ✅](${vt_url})"
+  fi
+  TABLE+="| \`${label}\` | \`${short}\` | ${cell} |\n"
+done
+
+CURRENT=$(gh release view "$VERSION" \
+  --json body --jq '.body // ""' --repo "$GITHUB_REPOSITORY")
+printf '%s%b' "$CURRENT" "$TABLE" > /tmp/release_notes.md
+gh release edit "$VERSION" \
+  --notes-file /tmp/release_notes.md --repo "$GITHUB_REPOSITORY"

scripts/ci/av-endpoint-verify.sh

@@ -0,0 +1,153 @@
+#!/usr/bin/env bash
+# av-endpoint-verify.sh — verify release binaries on a REAL Microsoft Defender
+# endpoint (the local Windows test VM) and produce the hash-pinned evidence
+# file that scripts/ci/check-virustotal.sh accepts for pre-release ML false
+# positives.
+#
+# VirusTotal's "Microsoft" engine and the shipping Defender product can
+# disagree: VT runs a differently-configured engine instance, and for our
+# large binaries it sits at an unstable ML decision boundary (Wacatac!ml came
+# and went across four release cycles with no code-side cause). This script
+# asks the authoritative source — Defender itself, signatures updated minutes
+# before the scan, real-time protection on — about the exact bytes VT flagged,
+# and records the answer:
+#
+#   defender-endpoint-verification.txt
+#     # release/engine/signature header lines
+#     <sha256>  <asset-name>  CLEAN|DETECTED
+#
+# The release gate downgrades a pre-release BLOCK to TOLERATED only when every
+# flagged binary's sha256 appears here as CLEAN.
+#
+# Usage: scripts/ci/av-endpoint-verify.sh <version> <asset-name...> [--upload]
+#   version     release tag holding the draft assets (e.g. v0.9.1-rc.1)
+#   asset-name  archive basename without extension, as printed by the gate
+#               (e.g. codebase-memory-mcp-ui-linux-amd64)
+#   --upload    attach/replace the evidence file on the (draft) release
+#
+# Runs on the dev machine: needs gh (authenticated) and the local Windows VM
+# (test-infrastructure/vm/win.sh), plus ~300 MB VM disk per binary.
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+cd "$ROOT"
+WIN=test-infrastructure/vm/win.sh
+[ -x "$WIN" ] || { echo "av-endpoint-verify: $WIN not found — this runs on the dev machine" >&2; exit 2; }
+
+VERSION="${1:-}"
+[ -n "$VERSION" ] || { echo "usage: scripts/ci/av-endpoint-verify.sh <version> <asset-name...> [--upload]" >&2; exit 2; }
+shift
+UPLOAD=false
+NAMES=()
+for arg in "$@"; do
+  case "$arg" in
+  --upload) UPLOAD=true ;;
+  *) NAMES+=("$arg") ;;
+  esac
+done
+[ "${#NAMES[@]}" -ge 1 ] || { echo "av-endpoint-verify: no asset names given" >&2; exit 2; }
+
+TMP="$(mktemp -d "${TMPDIR:-/tmp}/av-endpoint.XXXXXX")"
+trap 'rm -rf "$TMP"' EXIT
+
+# ── Fetch + extract the exact release bytes ─────────────────────────────────
+declare -a SHAS
+for name in "${NAMES[@]}"; do
+  if gh release download "$VERSION" --pattern "$name.tar.gz" --dir "$TMP" 2>/dev/null; then
+    mkdir -p "$TMP/x-$name"
+    tar -xzf "$TMP/$name.tar.gz" -C "$TMP/x-$name"
+    BIN="$TMP/x-$name/codebase-memory-mcp"
+  elif gh release download "$VERSION" --pattern "$name.zip" --dir "$TMP" 2>/dev/null; then
+    mkdir -p "$TMP/x-$name"
+    unzip -oq "$TMP/$name.zip" -d "$TMP/x-$name"
+    BIN="$TMP/x-$name/codebase-memory-mcp.exe"
+  else
+    echo "av-endpoint-verify: no $name.tar.gz or $name.zip on release $VERSION" >&2
+    exit 2
+  fi
+  [ -f "$BIN" ] || { echo "av-endpoint-verify: archive $name held no binary" >&2; exit 2; }
+  mv "$BIN" "$TMP/probe-$name"
+  SHAS+=("$(shasum -a 256 "$TMP/probe-$name" | awk '{print $1}')")
+  echo "=== fetched $name (${SHAS[${#SHAS[@]}-1]}) ==="
+done
+
+# ── Build the scan driver (pure ASCII: PowerShell 5.1 decodes BOM-less .ps1
+# as ANSI, and a stray non-ASCII byte corrupts the parse) ────────────────────
+PS1="$TMP/av-verify.ps1"
+{
+  echo '$mp = Join-Path $env:ProgramFiles "Windows Defender\MpCmdRun.exe"'
+  echo '& $mp -SignatureUpdate | Out-Null'
+  echo '$s = Get-MpComputerStatus'
+  echo 'Write-Output ("ENGINE=" + $s.AMEngineVersion)'
+  echo 'Write-Output ("SIGNATURES=" + $s.AntivirusSignatureVersion)'
+  echo 'Write-Output ("SIG_UPDATED=" + $s.AntivirusSignatureLastUpdated.ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ssZ"))'
+  echo 'Write-Output ("RTP=" + $s.RealTimeProtectionEnabled)'
+  for name in "${NAMES[@]}"; do
+    echo '$f = "C:\Users\test\av-verify-'"$name"'"'
+    echo 'if (-not (Test-Path $f)) { Write-Output ("RESULT '"$name"' MISSING") } else {'
+    echo '  & $mp -Scan -ScanType 3 -File $f | Out-Null'
+    echo '  $code = $LASTEXITCODE'
+    # RTP quarantining the file between push and scan is itself a detection.
+    echo '  if ($code -eq 0 -and (Test-Path $f)) { Write-Output ("RESULT '"$name"' CLEAN") }'
+    echo '  else { Write-Output ("RESULT '"$name"' DETECTED code=" + $code) }'
+    echo '  Remove-Item $f -Force -ErrorAction SilentlyContinue'
+    echo '}'
+  done
+} > "$PS1"
+LC_ALL=C grep -q '[^ -~]' "$PS1" && { echo "av-endpoint-verify: driver is not pure ASCII" >&2; exit 2; }
+
+# ── Push and scan ───────────────────────────────────────────────────────────
+for name in "${NAMES[@]}"; do
+  echo "=== pushing $name to the VM ==="
+  "$WIN" push-file "$TMP/probe-$name" "C:/Users/test/av-verify-$name"
+done
+"$WIN" push-file "$PS1" "C:/Users/test/av-verify.ps1"
+echo "=== updating signatures + scanning on the endpoint ==="
+SCAN_OUT="$TMP/scan-out.txt"
+"$WIN" sh "powershell -NoProfile -ExecutionPolicy Bypass -File C:/Users/test/av-verify.ps1" | tee "$SCAN_OUT"
+
+ENGINE=$(sed -n 's/^ENGINE=//p' "$SCAN_OUT" | tr -d '\r' | tail -1)
+SIGS=$(sed -n 's/^SIGNATURES=//p' "$SCAN_OUT" | tr -d '\r' | tail -1)
+SIG_UPDATED=$(sed -n 's/^SIG_UPDATED=//p' "$SCAN_OUT" | tr -d '\r' | tail -1)
+RTP=$(sed -n 's/^RTP=//p' "$SCAN_OUT" | tr -d '\r' | tail -1)
+[ -n "$ENGINE" ] && [ -n "$SIGS" ] || { echo "av-endpoint-verify: endpoint did not report engine/signature versions" >&2; exit 1; }
+[ "$RTP" = "True" ] || { echo "av-endpoint-verify: real-time protection is OFF on the VM — evidence would be meaningless" >&2; exit 1; }
+
+# ── Write the evidence file ─────────────────────────────────────────────────
+OUT="defender-endpoint-verification.txt"
+{
+  echo "# defender-endpoint-verification"
+  echo "# release: $VERSION"
+  echo "# generated: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
+  echo "# venue: local Windows 11 ARM64 VM, Microsoft Defender endpoint, real-time protection on"
+  echo "# engine: $ENGINE"
+  echo "# signatures: $SIGS (updated $SIG_UPDATED)"
+} > "$OUT"
+
+FAILED=false
+for i in "${!NAMES[@]}"; do
+  name="${NAMES[$i]}"
+  verdict=$(sed -n "s/^RESULT $name //p" "$SCAN_OUT" | tr -d '\r' | tail -1)
+  case "$verdict" in
+  CLEAN) printf '%s  %s  CLEAN\n' "${SHAS[$i]}" "$name" >> "$OUT" ;;
+  *)
+    printf '%s  %s  DETECTED\n' "${SHAS[$i]}" "$name" >> "$OUT"
+    echo "av-endpoint-verify: $name verdict '$verdict' — NOT clean" >&2
+    FAILED=true
+    ;;
+  esac
+done
+
+echo "=== evidence written: $OUT ==="
+cat "$OUT"
+
+if [ "$FAILED" = "true" ]; then
+  echo "av-endpoint-verify: Defender itself reports a detection — this is NOT" >&2
+  echo "the false-positive pattern; do not release these binaries." >&2
+  exit 1
+fi
+
+if [ "$UPLOAD" = "true" ]; then
+  gh release upload "$VERSION" "$OUT" --clobber
+  echo "=== uploaded $OUT to release $VERSION ==="
+fi

scripts/ci/check-virustotal.sh

@@ -1,10 +1,37 @@
 #!/usr/bin/env bash
 # Wait for VirusTotal scans to complete and check results.
 # Expects: VT_API_KEY, VT_ANALYSIS (comma-separated "file=URL" pairs)
+# Optional: VERSION (release tag) + GH_TOKEN — these enable the PRE-RELEASE
+#           single-ML-detection tolerance described below.
 set -euo pipefail
 
 MIN_ENGINES=60
-rm -f /tmp/vt_gate_fail
+rm -f /tmp/vt_gate_fail /tmp/vt_suspects.tsv /tmp/vt_tolerated.tsv
+
+# ── Pre-release ML false-positive tolerance ──────────────────────────────────
+# A verdict may be downgraded from BLOCKED to TOLERATED only when ALL of:
+#   1. the version is a pre-release (-rc. / -pre / -alpha / -beta),
+#   2. every failing file was flagged by exactly ONE engine,
+#   3. that engine is Microsoft and the verdict is a machine-learning
+#      heuristic (result string ends in "!ml") — never a signature name,
+#   4. an up-to-date Microsoft Defender ENDPOINT has scanned the identical
+#      bytes clean, proven by a hash-pinned evidence file attached to the
+#      draft release (defender-endpoint-verification.txt, produced by
+#      scripts/ci/av-endpoint-verify.sh).
+#
+# Why this is sound: VT's "Microsoft" engine is a differently-configured
+# instance of the same engine that ships in Windows, and for our large
+# binaries it sits at an unstable ML decision boundary — across four release
+# cycles the Wacatac!ml flag appeared and disappeared with no code-side cause
+# (stripping, downloader removal and metadata changes each "fixed" it only
+# until the next build). The shipping product's verdict on identical bytes,
+# with signatures updated minutes before the scan, is the stronger statement
+# of Microsoft's own estimate. Multi-engine hits, signature-named verdicts,
+# non-Microsoft engines, timeouts and stable releases ALWAYS block.
+TOLERANCE_ELIGIBLE=false
+case "${VERSION:-}" in
+*-rc.* | *-pre* | *-alpha* | *-beta*) TOLERANCE_ELIGIBLE=true ;;
+esac
 
 echo "=== Waiting for VirusTotal scans to fully complete ==="
 
@@ -68,6 +95,22 @@ print(f'{status},{malicious},{suspicious},{completed},{total}')
         echo "BLOCKED: $BASENAME flagged ($MALICIOUS malicious, $SUSPICIOUS suspicious / $COMPLETED engines)"
         echo "  $URL"
         echo "FAIL" >> /tmp/vt_gate_fail
+        # Record WHO flagged it so the tolerance can classify the failure.
+        FLAG_INFO=$(echo "$RESULT" | python3 -c "
+import json, sys
+d = json.loads(sys.stdin.read())
+res = d.get('data', {}).get('attributes', {}).get('results', {})
+flagged = [(e, (r.get('result') or '?')) for e, r in res.items()
+           if r.get('category') in ('malicious', 'suspicious')]
+engines = ';'.join(e for e, _ in flagged)
+verdicts = ';'.join(v for _, v in flagged)
+print(f'{len(flagged)}\t{engines}\t{verdicts}')
+" 2>/dev/null || printf 'parse-error\tparse-error\tparse-error\n')
+        SHA256=$( (sha256sum "$FILE" 2>/dev/null || shasum -a 256 "$FILE" 2>/dev/null) \
+          | awk 'NR==1{print $1}')
+        [ -n "$SHA256" ] || SHA256="unknown"
+        printf '%s\t%s\t%s\t%s\n' "$BASENAME" "$SHA256" "$FLAG_INFO" "$COMPLETED" \
+          >> /tmp/vt_suspects.tsv
       else
         echo "OK: $BASENAME clean ($COMPLETED engines, 0 detections)"
       fi
@@ -84,8 +127,76 @@ print(f'{status},{malicious},{suspicious},{completed},{total}')
   fi
 done
 
-if [ -f /tmp/vt_gate_fail ]; then
+if [ ! -f /tmp/vt_gate_fail ]; then
+  echo "=== All VirusTotal scans passed ==="
+  exit 0
+fi
+
+# ── Classify the failures against the tolerance conditions ──────────────────
+FAILS=$(wc -l < /tmp/vt_gate_fail | tr -d ' ')
+SUSPECTS=0
+[ -f /tmp/vt_suspects.tsv ] && SUSPECTS=$(wc -l < /tmp/vt_suspects.tsv | tr -d ' ')
+
+TOLERABLE="$TOLERANCE_ELIGIBLE"
+# Timeouts and unparseable URLs produce FAIL lines with no suspect record —
+# those are never tolerable, and the counts diverging catches them.
+[ "$FAILS" -eq "$SUSPECTS" ] || TOLERABLE=false
+if [ "$TOLERABLE" = "true" ]; then
+  while IFS=$'\t' read -r _name _sha nflagged engines verdicts _completed; do
+    [ "$nflagged" = "1" ] || TOLERABLE=false
+    [ "$engines" = "Microsoft" ] || TOLERABLE=false
+    case "$verdicts" in
+    *'!ml') ;;
+    *) TOLERABLE=false ;;
+    esac
+  done < /tmp/vt_suspects.tsv
+fi
+
+if [ "$TOLERABLE" != "true" ]; then
   echo "BLOCKED: One or more VirusTotal checks failed"
   exit 1
 fi
-echo "=== All VirusTotal scans passed ==="
+
+# ── Every failure matches the ML-FP pattern: require endpoint evidence ──────
+EVIDENCE=/tmp/defender-endpoint-verification.txt
+rm -f "$EVIDENCE"
+if ! gh release download "${VERSION:?}" \
+  --pattern defender-endpoint-verification.txt \
+  --output "$EVIDENCE" --repo "${GITHUB_REPOSITORY:-DeusData/codebase-memory-mcp}" \
+  2>/dev/null; then
+  echo "BLOCKED: all detections match the pre-release ML false-positive pattern"
+  echo "  (single engine, Microsoft, '!ml' verdict) but the release carries no"
+  echo "  Defender endpoint evidence. To verify Microsoft's shipping product"
+  echo "  against these exact bytes and attach the proof, run locally:"
+  echo ""
+  NAMES=$(cut -f1 /tmp/vt_suspects.tsv | sed 's/\.exe$//' | tr '\n' ' ')
+  echo "    scripts/ci/av-endpoint-verify.sh $VERSION $NAMES--upload"
+  echo ""
+  echo "  then RE-RUN THIS JOB (Re-run failed jobs — it does not rebuild, so"
+  echo "  the bytes and hashes stay identical)."
+  exit 1
+fi
+
+MISSING=false
+while IFS=$'\t' read -r name sha _rest; do
+  if grep -qE "^${sha}[[:space:]]+[^[:space:]]+[[:space:]]+CLEAN[[:space:]]*$" "$EVIDENCE"; then
+    echo "ENDPOINT-VERIFIED: $name ($sha) clean on a current Defender endpoint"
+  else
+    echo "BLOCKED: $name ($sha) has no CLEAN endpoint-evidence line"
+    MISSING=true
+  fi
+done < /tmp/vt_suspects.tsv
+if [ "$MISSING" = "true" ]; then
+  echo "  Evidence file present but stale or incomplete — regenerate it with"
+  echo "  scripts/ci/av-endpoint-verify.sh (hashes must match this build)."
+  exit 1
+fi
+
+cp /tmp/vt_suspects.tsv /tmp/vt_tolerated.tsv
+echo "=== TOLERATED (pre-release ML false-positive policy) ==="
+sed -n 's/^# //p' "$EVIDENCE"
+echo "Every remaining engine reports clean; the detections above are Microsoft"
+echo "ML heuristics that Microsoft's own shipping Defender does not reproduce."
+echo "The release notes will state this transparently, with the evidence file"
+echo "attached as a release asset."
+exit 0

tests/test_vt_gate_tolerance_contract.sh

@@ -0,0 +1,133 @@
+#!/usr/bin/env bash
+# Contract: the VirusTotal gate's pre-release ML false-positive tolerance
+# (scripts/ci/check-virustotal.sh) fails CLOSED in every direction.
+#
+# The tolerance may downgrade a BLOCK to TOLERATED only for the exact pattern
+#   pre-release version AND single engine AND Microsoft AND "!ml" verdict
+#   AND hash-pinned Defender endpoint evidence attached to the release.
+# Anything else — stable version, multi-engine, signature-named verdict,
+# non-Microsoft engine, missing or stale evidence — must still block. A bug in
+# the tolerating direction ships a flagged binary, so every branch is pinned
+# here with a stubbed VT API (curl) and release store (gh).
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+GATE="$ROOT/scripts/ci/check-virustotal.sh"
+FIX="$(mktemp -d "${TMPDIR:-/tmp}/vt-gate-contract.XXXXXX")"
+trap 'rm -rf "$FIX"' EXIT
+
+fail() { echo "FAIL: $*" >&2; exit 1; }
+
+# ── Stub venue ──────────────────────────────────────────────────────────────
+mkdir -p "$FIX/bin" "$FIX/work/binaries" "$FIX/responses"
+
+# curl: serve the canned VT analysis JSON selected by the analysis id in the URL.
+cat > "$FIX/bin/curl" <<'EOF'
+#!/usr/bin/env bash
+url="${@: -1}"
+id="${url##*/analyses/}"
+resp="$STUB_RESPONSES/$id.json"
+[ -f "$resp" ] || exit 22
+cat "$resp"
+EOF
+
+# gh: "release download" delivers the evidence fixture if the scenario has one.
+cat > "$FIX/bin/gh" <<'EOF'
+#!/usr/bin/env bash
+out=""
+prev=""
+for a in "$@"; do
+  [ "$prev" = "--output" ] && out="$a"
+  prev="$a"
+done
+[ -n "$STUB_EVIDENCE" ] && [ -f "$STUB_EVIDENCE" ] || exit 1
+cp "$STUB_EVIDENCE" "$out"
+EOF
+chmod +x "$FIX/bin/curl" "$FIX/bin/gh"
+
+mk_response() { # id malicious results-json
+  cat > "$FIX/responses/$1.json" <<EOF
+{"data":{"attributes":{"status":"completed",
+  "stats":{"malicious":$2,"suspicious":0,"undetected":60,"harmless":0},
+  "results":{$3}}}}
+EOF
+}
+MS_ML='"Microsoft":{"category":"malicious","result":"Trojan:Script/Wacatac.B!ml"}'
+MS_SIG='"Microsoft":{"category":"malicious","result":"Trojan:Win32/Emotet.A"}'
+BKAV_ML='"Bkav":{"category":"malicious","result":"W64.AIDetectMalware!ml"}'
+TWO="$MS_ML,$BKAV_ML"
+
+mk_response clean 0 ''
+mk_response msml 1 "$MS_ML"
+mk_response mssig 1 "$MS_SIG"
+mk_response bkav 1 "$BKAV_ML"
+mk_response two 2 "$TWO"
+
+echo "flagged-bytes" > "$FIX/work/binaries/probe"
+PROBE_SHA=$( (sha256sum "$FIX/work/binaries/probe" 2>/dev/null \
+  || shasum -a 256 "$FIX/work/binaries/probe") | awk 'NR==1{print $1}')
+
+good_evidence() {
+  printf '# defender-endpoint-verification\n# engine: 1.1.1\n# signatures: 1.2.3\n%s  probe  CLEAN\n' "$PROBE_SHA"
+}
+
+run_gate() { # version analysis-id evidence-file(optional) -> echoes exit code
+  rm -f /tmp/vt_gate_fail /tmp/vt_suspects.tsv /tmp/vt_tolerated.tsv
+  local rc=0
+  (cd "$FIX/work" &&
+    PATH="$FIX/bin:$PATH" \
+      STUB_RESPONSES="$FIX/responses" \
+      STUB_EVIDENCE="${3:-}" \
+      GITHUB_REPOSITORY="stub/repo" \
+      VT_API_KEY="stub" \
+      VERSION="$1" \
+      VT_ANALYSIS="binaries/probe=https://www.virustotal.com/gui/file-analysis/$2/detection" \
+      bash "$GATE" >"$FIX/last.log" 2>&1) || rc=$?
+  echo "$rc"
+}
+
+# ── 1. Clean scan passes, no tolerance involved ─────────────────────────────
+[ "$(run_gate v0.9.1-rc.1 clean)" = "0" ] || fail "clean scan must pass"
+[ ! -f /tmp/vt_tolerated.tsv ] || fail "clean scan must not write a tolerance record"
+
+# ── 2. Stable version: the exact ML-FP pattern with evidence still blocks ───
+good_evidence > "$FIX/evidence.txt"
+[ "$(run_gate v0.9.1 msml "$FIX/evidence.txt")" != "0" ] || \
+  fail "stable release must never tolerate"
+
+# ── 3. Pre-release + pattern + NO evidence blocks, with operator instructions ─
+[ "$(run_gate v0.9.1-rc.1 msml)" != "0" ] || fail "missing evidence must block"
+grep -q "av-endpoint-verify.sh" "$FIX/last.log" || \
+  fail "missing-evidence block must print the endpoint-verify instructions"
+
+# ── 4. Pre-release + pattern + matching evidence tolerates ──────────────────
+[ "$(run_gate v0.9.1-rc.1 msml "$FIX/evidence.txt")" = "0" ] || \
+  fail "pattern + evidence must tolerate (got a block)"
+[ -s /tmp/vt_tolerated.tsv ] || fail "tolerance must record what it tolerated"
+grep -q "TOLERATED" "$FIX/last.log" || fail "tolerance must announce itself"
+
+# ── 5. Stale evidence (wrong hash) blocks ───────────────────────────────────
+printf '0000000000000000000000000000000000000000000000000000000000000000  probe  CLEAN\n' \
+  > "$FIX/stale.txt"
+[ "$(run_gate v0.9.1-rc.1 msml "$FIX/stale.txt")" != "0" ] || \
+  fail "evidence for different bytes must block"
+
+# ── 6. Evidence that says DETECTED blocks ───────────────────────────────────
+printf '%s  probe  DETECTED\n' "$PROBE_SHA" > "$FIX/detected.txt"
+[ "$(run_gate v0.9.1-rc.1 msml "$FIX/detected.txt")" != "0" ] || \
+  fail "endpoint DETECTED must block"
+
+# ── 7. Signature-named Microsoft verdict blocks even with evidence ──────────
+[ "$(run_gate v0.9.1-rc.1 mssig "$FIX/evidence.txt")" != "0" ] || \
+  fail "non-!ml (signature) verdict must block"
+
+# ── 8. Non-Microsoft single-engine !ml blocks ───────────────────────────────
+[ "$(run_gate v0.9.1-rc.1 bkav "$FIX/evidence.txt")" != "0" ] || \
+  fail "non-Microsoft engine must block"
+
+# ── 9. Multi-engine blocks ──────────────────────────────────────────────────
+[ "$(run_gate v0.9.1-rc.1 two "$FIX/evidence.txt")" != "0" ] || \
+  fail "multi-engine detection must block"
+
+rm -f /tmp/vt_gate_fail /tmp/vt_suspects.tsv /tmp/vt_tolerated.tsv
+echo "PASS: VT gate tolerance fails closed in all nine pinned directions"