fix(ci): finish the three-candidate wiring and restore full-surface VT scanning

Two things, both found by the dry-run and by sweeping what it exposed.

1. The three-candidate change was incomplete. The dry-run failed at
   stage-release-candidates.py, which carries its OWN copy of the transform
   validation that the previous commit only fixed in the selector:

     stage-release-candidates: candidate linux-amd64/debug-stripped has
     invalid transform: 'strip-debug'

   Sweeping for that assumption found it in six places, not two:
   stage-release-candidates.py, select-release-candidates.py,
   verify-release-selection.py, append-vt-notes.sh and two contract-test
   fixtures — as hardcoded 16s, `len(TARGETS) * 2`, two-entry VARIANTS tuples
   and two-key truth tables. All are now derived from len(VARIANTS).

   Field prefixes needed care: the variant NAME keeps its hyphen because it is
   the on-disk directory, while the evidence columns use underscores, so
   `debug-stripped` reads `debug_stripped_sha256`. Every lookup now goes through
   an explicit FIELD_KEY map instead of interpolating the variant directly.

   The selection contract test now covers the truth table EXHAUSTIVELY: three
   variants x two tolerated classifications is exactly eight combinations, and
   there are exactly eight targets, so every case is exercised once.

2. Full-surface VirusTotal scanning is restored. This PR had moved scanning
   upstream to the candidates and deleted the post-package pass, which silently
   narrowed coverage from everything we ship to executables only. The 42 runtime
   files across the 14 containers — install.sh, install.ps1, LICENSE,
   THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the unpacked UI assets —
   were still extracted, structurally verified and strings-audited, but no
   longer scanned at all. install.sh and install.ps1 are the highest-consequence
   non-executable bytes we publish; users pipe them straight into a shell.

   The verify job scans every extracted object again, under the same policy.
   Re-submitting the selected executables alongside them is close to free
   because VirusTotal is content-addressed and answers for identical bytes from
   its own record — the same property that made the analysis-id equality check
   untenable two commits ago.

   The gate-chain contract asserted the opposite ("duplicate post-package
   VirusTotal path remains"). That assertion is inverted: the pass is required,
   and it is not a duplicate, since it covers a strictly larger set. README and
   SECURITY.md updated from "archive containers are checksummed rather than
   redundantly rescanned" to state the full covered surface.

All five release/VT contract tests pass.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel committed 2026-08-14 00:21:31 +02:00
1 parent 98d7dbab01
commit 7b4533b67d
10 files changed
+199 -90

No files matched your search

+54
View File
@@ -298,6 +298,60 @@ jobs:
scripts/security-strings.sh "$f"
done
# Full-surface VirusTotal pass over the SHIPPED containers.
#
# The candidate scan upstream covers executables only. Everything else we
# actually ship — install.sh, install.ps1, LICENSE, THIRD_PARTY_NOTICES.md,
# the MCPB manifest.json and the unpacked UI assets, 42 runtime files
# across 14 containers — is extracted here and gets scanned here. Dropping
# it would have narrowed the promise made in README/SECURITY.md, and
# install.sh and install.ps1 are the highest-consequence non-executable
# bytes we publish: users pipe them straight into a shell.
#
# The selected executables are re-submitted too. That is deliberate and
# nearly free: VirusTotal is content-addressed, so identical bytes return
# the analysis it already holds instead of re-running 70+ engines — the
# same property that made the analysis-id equality check untenable.
- name: VirusTotal scan of every extracted release object
uses: crazy-max/ghaction-virustotal@936d8c5c00afe97d3d9a1af26d017cfdf26800a2 # v5.0.0
id: virustotal
with:
vt_api_key: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
files: binaries/objects/*
request_rate: 4
# Same policy as the candidate gate: a single Microsoft `!ml` is tolerated
# and disclosed, anything else blocks. Engine count is evidence, not a gate.
- name: Wait for VirusTotal results
env:
VT_API_KEY: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
VT_ANALYSIS: ${{ steps.virustotal.outputs.analysis }}
VT_EXPECTED_SCAN_SET: binaries/scan-set.tsv
VT_ASSOCIATIONS: binaries/associations.tsv
VT_RESULTS_PATH: binaries/vt-results.tsv
MIN_ENGINES: 50
run: scripts/ci/check-virustotal.sh
- name: Preserve extracted-object VirusTotal evidence
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: virustotal-evidence-release-${{ github.run_id }}
path: |
binaries/associations.tsv
binaries/scan-set.tsv
binaries/vt-results.tsv
if-no-files-found: warn
- name: Publish durable public VirusTotal evidence
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ inputs.version }}
VT_EXPECTED_SCAN_SET: binaries/scan-set.tsv
VT_ASSOCIATIONS: binaries/associations.tsv
VT_RESULTS_PATH: binaries/vt-results.tsv
run: bash scripts/ci/publish-vt-evidence.sh
# Candidate bytes were scanned before smoke/soak and the verifier above
# proved every packaged executable has the selected candidate SHA-256.
# Re-submitting the same executable bytes here would add wait time but no
+1 -1
View File
@@ -794,7 +794,7 @@ internal/cbm/ Vendored tree-sitter grammars (158 languages) + AST extrac
Every release is verified through a multi-layer pipeline before publication:
- **VirusTotal** — all 24 executable candidates (unstripped, debug-stripped and stripped) across the eight release products are scanned before smoke/soak (clean is preferred and only the single-Microsoft `!ml` tolerance documented in [SECURITY.md](SECURITY.md#our-release-policy) may pass; the number of engines that returned a decisive result is recorded as evidence but is VirusTotal fleet availability, not a pass condition). The selected executable is packaged without changing its SHA-256, and release notes link every candidate result and the selection evidence. Archive containers are checksummed, not redundantly resubmitted.
- **VirusTotal** — all 24 executable candidates (unstripped, debug-stripped and stripped) across the eight release products are scanned before smoke/soak (clean is preferred and only the single-Microsoft `!ml` tolerance documented in [SECURITY.md](SECURITY.md#our-release-policy) may pass; the number of engines that returned a decisive result is recorded as evidence but is VirusTotal fleet availability, not a pass condition). The selected executable is packaged without changing its SHA-256, and release notes link every candidate result and the selection evidence. Every distinct object extracted from the shipped containers — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — is then scanned as well, so the full published surface is covered, not just the executables.
- **SLSA Level 3** — cryptographic build provenance generated by the trusted GitHub Actions build workflow; verify with `gh attestation verify <file> --repo DeusData/codebase-memory-mcp --signer-workflow DeusData/codebase-memory-mcp/.github/workflows/_build.yml`
- **Sigstore cosign** — keyless signatures on all artifacts; bundles included in every release
- **SHA-256 checksums** — `checksums.txt` published with every release; verified by both install scripts before extraction
+1 -1
View File
@@ -126,7 +126,7 @@ Releases are created as **drafts** (invisible to users) and only published after
2. **Sigstore cosign signing** — keyless digital signatures verifiable by anyone
3. **SBOM** — Software Bill of Materials (SPDX) listing all vendored dependencies
4. **SHA-256 checksums** — published with every release
5. **VirusTotal scanning** — three behaviourally identical executable candidates — unstripped, debug-stripped and stripped — are derived from one linker output for every release product and scanned before smoke/soak. Zero malicious and zero suspicious is preferred; the only tolerated result is exactly one Microsoft malicious label ending in `!ml`, which is disclosed. The selected executable is packaged without changing its SHA-256; archive containers are checksummed rather than redundantly rescanned. Release notes link every candidate result and the selection evidence.
5. **VirusTotal scanning** — three behaviourally identical executable candidates — unstripped, debug-stripped and stripped — are derived from one linker output for every release product and scanned before smoke/soak. Zero malicious and zero suspicious is preferred; the only tolerated result is exactly one Microsoft malicious label ending in `!ml`, which is disclosed. The selected executable is packaged without changing its SHA-256. After packaging, every distinct object extracted from the 14 shipped containers is scanned under the same policy — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — so the covered surface is everything we publish, not only the executables. Release notes link every candidate result and the selection evidence.
6. **OpenSSF Scorecard** — repository security health score
Scope of the SLSA claim: this is a build provenance claim for release
+25 -17
View File
@@ -38,7 +38,8 @@ TARGETS = (
"windows-amd64",
"windows-arm64",
)
VARIANTS = ("unstripped", "stripped")
VARIANTS = ("unstripped", "debug-stripped", "stripped")
FIELD_KEY = {variant: variant.replace("-", "_") for variant in VARIANTS}
CANDIDATE_FIELDS = (
"target", "variant", "relative_path", "source_sha256", "pre_sign_sha256",
"sha256", "size", "format", "architecture", "linkage", "transform",
@@ -54,7 +55,11 @@ SELECTION_FIELDS = (
"target", "selected_variant", "selected_path", "selected_sha256",
"selected_size", "decision", "unstripped_sha256", "unstripped_scan_path",
"unstripped_classification", "unstripped_analysis_id",
"unstripped_virustotal_url", "stripped_sha256", "stripped_scan_path",
"unstripped_virustotal_url",
"debug_stripped_sha256", "debug_stripped_scan_path",
"debug_stripped_classification", "debug_stripped_analysis_id",
"debug_stripped_virustotal_url",
"stripped_sha256", "stripped_scan_path",
"stripped_classification", "stripped_analysis_id", "stripped_virustotal_url",
)
SHA256 = re.compile(r"[0-9a-f]{64}\Z")
@@ -110,20 +115,22 @@ selection_meta, selections = read_tsv(
selection_path, "cbm-release-selection-v1", SELECTION_FIELDS
)
expected_pairs = [(target, variant) for target in TARGETS for variant in VARIANTS]
if candidate_meta != {"targets": "8", "candidates": "16"}:
fail("candidate metadata does not bind the 8-target/16-candidate matrix")
if candidate_meta != {"targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * len(VARIANTS))}:
fail("candidate metadata does not bind the canonical target/variant matrix")
if [(row["target"], row["variant"]) for row in candidates] != expected_pairs:
fail("candidate rows are not the canonical target/variant matrix")
if result_meta.get("scan_objects") != "16" or result_meta.get("associations") != "16":
fail("VirusTotal results do not cover all 16 candidates")
if selection_meta != {"policy": "virustotal-v2", "targets": "8", "candidates": "16"}:
expected_scans = str(len(TARGETS) * len(VARIANTS))
if result_meta.get("scan_objects") != expected_scans or result_meta.get("associations") != expected_scans:
fail("VirusTotal results do not cover every candidate")
if selection_meta != {"policy": "virustotal-v2", "targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * len(VARIANTS))}:
fail("selection was not produced by the scanned release policy")
if [row["target"] for row in selections] != list(TARGETS):
fail("selection rows are not the canonical target matrix")
candidate_by_pair = {(row["target"], row["variant"]): row for row in candidates}
candidate_by_path = {row["scan_path"]: row for row in candidates}
if len(candidate_by_pair) != 16 or len(candidate_by_path) != 16:
expected_candidates = len(TARGETS) * len(VARIANTS)
if len(candidate_by_pair) != expected_candidates or len(candidate_by_path) != expected_candidates:
fail("candidate evidence contains duplicates")
results_by_path: dict[str, dict[str, str]] = {}
@@ -175,9 +182,10 @@ for selection in selections:
target = selection["target"]
pair = {variant: candidate_by_pair[target, variant] for variant in VARIANTS}
pair_results = {variant: results_by_path[pair[variant]["scan_path"]] for variant in VARIANTS}
stripped_class = pair_results["stripped"]["policy_classification"]
unstripped_class = pair_results["unstripped"]["policy_classification"]
expected_variant = "stripped" if stripped_class == "clean" or unstripped_class != "clean" else "unstripped"
classes = {v: pair_results[v]["policy_classification"] for v in VARIANTS}
order = ("stripped", "debug-stripped", "unstripped")
clean = [v for v in order if classes[v] == "clean"]
expected_variant = clean[0] if clean else "stripped"
selected = pair[expected_variant]
if (
selection["selected_variant"] != expected_variant
@@ -188,11 +196,11 @@ for selection in selections:
for variant in VARIANTS:
result = pair_results[variant]
if (
selection[f"{variant}_sha256"] != pair[variant]["sha256"]
or selection[f"{variant}_scan_path"] != pair[variant]["scan_path"]
or selection[f"{variant}_classification"] != result["policy_classification"]
or selection[f"{variant}_analysis_id"] != result["analysis_id"]
or selection[f"{variant}_virustotal_url"] != result["virustotal_url"]
selection[f"{FIELD_KEY[variant]}_sha256"] != pair[variant]["sha256"]
or selection[f"{FIELD_KEY[variant]}_scan_path"] != pair[variant]["scan_path"]
or selection[f"{FIELD_KEY[variant]}_classification"] != result["policy_classification"]
or selection[f"{FIELD_KEY[variant]}_analysis_id"] != result["analysis_id"]
or selection[f"{FIELD_KEY[variant]}_virustotal_url"] != result["virustotal_url"]
):
fail(f"selection evidence is not bound to both candidate verdicts: {target}")
selection_by_target[target] = selection
@@ -219,7 +227,7 @@ section = [
"## Security Verification",
"",
(
"Before smoke and soak testing, VirusTotal completed **16 executable scans**: "
f"Before smoke and soak testing, VirusTotal completed **{len(TARGETS) * len(VARIANTS)} executable scans**: "
"stripped and unstripped candidates for each of the **8 release products**. "
f"Every scan had at least 50 decisive engines (observed range: {engine_range})."
),
+1 -1
View File
@@ -253,7 +253,7 @@ def load_candidates(
marker="cbm-release-candidates-v1",
fields=CANDIDATE_FIELDS,
)
if metadata != {"targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * 2)}:
if metadata != {"targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * len(VARIANTS))}:
raise ContractError("candidate metadata does not bind the exact canonical matrix")
expected_pairs = [(target, variant) for target in TARGETS for variant in VARIANTS]
if [(row["target"], row["variant"]) for row in rows] != expected_pairs:
+5 -1
View File
@@ -194,7 +194,11 @@ def validate_row_shape(row: dict[str, str], *, target: str, variant: str) -> pat
"format": file_format,
"architecture": architecture,
"linkage": linkage,
"transform": "copy" if variant == "unstripped" else "strip",
"transform": {
"unstripped": "copy",
"debug-stripped": "strip-debug",
"stripped": "strip",
}[variant],
"signature": "adhoc-verified" if target.startswith("darwin-") else "not-applicable",
"pair_verification": "same-linker-output-v1",
}
+30 -18
View File
@@ -31,7 +31,8 @@ TARGETS = (
"windows-amd64",
"windows-arm64",
)
VARIANTS = ("unstripped", "stripped")
VARIANTS = ("unstripped", "debug-stripped", "stripped")
FIELD_KEY = {variant: variant.replace("-", "_") for variant in VARIANTS}
SELECTION_FIELDS = (
"target",
"selected_variant",
@@ -44,6 +45,11 @@ SELECTION_FIELDS = (
"unstripped_classification",
"unstripped_analysis_id",
"unstripped_virustotal_url",
"debug_stripped_sha256",
"debug_stripped_scan_path",
"debug_stripped_classification",
"debug_stripped_analysis_id",
"debug_stripped_virustotal_url",
"stripped_sha256",
"stripped_scan_path",
"stripped_classification",
@@ -163,7 +169,7 @@ def load_selection(
target = row["target"]
binary = "codebase-memory-mcp.exe" if target.startswith("windows-") else "codebase-memory-mcp"
classifications = {
variant: row[f"{variant}_classification"] for variant in VARIANTS
variant: row[f"{FIELD_KEY[variant]}_classification"] for variant in VARIANTS
}
if metadata["policy"] == "virustotal-v2":
if any(
@@ -172,27 +178,33 @@ def load_selection(
):
raise ContractError(f"release selection contains a hard classification: {target}")
for variant in VARIANTS:
sha256 = row[f"{variant}_sha256"]
analysis_id = row[f"{variant}_analysis_id"]
sha256 = row[f"{FIELD_KEY[variant]}_sha256"]
analysis_id = row[f"{FIELD_KEY[variant]}_analysis_id"]
expected_url = f"https://www.virustotal.com/gui/file/{sha256}/detection"
if ANALYSIS_ID_RE.fullmatch(analysis_id) is None:
raise ContractError(f"release selection lacks VT analysis evidence: {target}")
if analysis_id in analysis_ids:
raise ContractError("release selection reuses one VT analysis id")
analysis_ids.add(analysis_id)
if row[f"{variant}_virustotal_url"] != expected_url:
if row[f"{FIELD_KEY[variant]}_virustotal_url"] != expected_url:
raise ContractError(f"release selection has unbound VT URL: {target}")
stripped_class = classifications["stripped"]
unstripped_class = classifications["unstripped"]
if stripped_class == "clean":
expected_variant = "stripped"
expected_decision = "stripped-preferred"
elif unstripped_class == "clean":
expected_variant = "unstripped"
expected_decision = "unstripped-clean-after-stripped-microsoft-ml"
# Re-derived here independently of the selector: smallest artifact
# first, first CLEAN one wins, and only if every candidate drew the
# tolerated verdict do we ship the smallest flagged one.
order = ("stripped", "debug-stripped", "unstripped")
clean = [v for v in order if classifications[v] == "clean"]
if clean:
expected_variant = clean[0]
if expected_variant == "stripped":
expected_decision = "stripped-preferred"
else:
others = "-".join(
f"{v}:{classifications[v]}" for v in order if v != expected_variant
)
expected_decision = f"{expected_variant}-clean-after-{others}"
else:
expected_variant = "stripped"
expected_decision = "stripped-both-microsoft-ml"
expected_decision = "stripped-all-candidates-microsoft-ml"
else:
if any(
classification != "unscanned-dry-run"
@@ -200,7 +212,7 @@ def load_selection(
):
raise ContractError(f"dry-run selection disguises unscanned evidence: {target}")
if any(
row[f"{variant}_{suffix}"]
row[f"{FIELD_KEY[variant]}_{suffix}"]
for variant in VARIANTS
for suffix in ("analysis_id", "virustotal_url")
):
@@ -217,14 +229,14 @@ def load_selection(
raise ContractError(f"release selection has unsafe selected path: {target}")
if (
SHA256_RE.fullmatch(row["selected_sha256"]) is None
or row["selected_sha256"] != row[f"{selected_variant}_sha256"]
or row["selected_sha256"] != row[f"{FIELD_KEY[selected_variant]}_sha256"]
or not row["selected_size"].isdigit()
or not 0 < int(row["selected_size"]) <= MAX_MEMBER_BYTES
):
raise ContractError(f"release selection has invalid selected content: {target}")
for variant in VARIANTS:
sha256 = row[f"{variant}_sha256"]
scan_path = pathlib.PurePosixPath(row[f"{variant}_scan_path"])
sha256 = row[f"{FIELD_KEY[variant]}_sha256"]
scan_path = pathlib.PurePosixPath(row[f"{FIELD_KEY[variant]}_scan_path"])
if (
SHA256_RE.fullmatch(sha256) is None
or scan_path.as_posix() != f"objects/{sha256}"
+14 -4
View File
@@ -291,11 +291,21 @@ for token in ("--pattern 'release-selection.tsv'",
failures.append(
f"release verify: final draft-byte selection binding is missing token: {token}")
for forbidden in ("crazy-max/ghaction-virustotal", "scripts/ci/check-virustotal.sh",
"scripts/ci/publish-vt-evidence.sh"):
if forbidden in verify_body:
# The post-package scan is REQUIRED, not forbidden. It is not a duplicate of the
# candidate scan: that one covers executables only, while this one covers every
# distinct object extracted from the 14 shipped containers - install.sh,
# install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the
# unpacked UI assets. Those are the bytes users pipe into a shell, and dropping
# them would narrow the promise made in README.md and SECURITY.md.
#
# Re-submitting the selected executables alongside them is close to free:
# VirusTotal is content-addressed and returns the analysis it already holds for
# identical bytes.
for token in ("crazy-max/ghaction-virustotal", "scripts/ci/check-virustotal.sh",
"scripts/ci/publish-vt-evidence.sh", "files: binaries/objects/*"):
if token not in verify_body:
failures.append(
f"release verify: duplicate post-package VirusTotal path remains: {forbidden}")
f"release verify: full-surface VirusTotal pass is missing: {token}")
for token in ("release-candidates.tsv", "virustotal-candidate-results.tsv",
"release-selection.tsv", "scripts/ci/append-vt-notes.sh"):
if token not in verify_body:
+40 -26
View File
@@ -41,7 +41,8 @@ TARGETS = (
"linux-arm64-portable", "darwin-amd64", "darwin-arm64",
"windows-amd64", "windows-arm64",
)
VARIANTS = ("unstripped", "stripped")
VARIANTS = ("unstripped", "debug-stripped", "stripped")
ORDER = ("stripped", "debug-stripped", "unstripped")
FIELDS = (
"target", "variant", "relative_path", "source_sha256", "pre_sign_sha256",
"sha256", "size", "format", "architecture", "linkage", "transform",
@@ -106,7 +107,11 @@ def write_tsv(path: pathlib.Path, marker: str, metadata: dict[str, object],
def candidate_bytes(target: str, variant: str) -> bytes:
linked = f"codebase-memory-mcp:{target}:same-linker-output\n".encode()
suffix = b"debug-and-local-symbols\n" if variant == "unstripped" else b"stripped\n"
suffix = {
"unstripped": b"debug-and-local-symbols\n",
"debug-stripped": b"local-symbols-only\n",
"stripped": b"stripped\n",
}[variant]
return linked + suffix
@@ -137,7 +142,7 @@ def make_artifacts(directory: pathlib.Path, *, reverse: bool = False) -> dict[tu
"format": "pe" if target.startswith("windows-") else "macho" if target.startswith("darwin-") else "elf",
"architecture": target.split("-")[1],
"linkage": "portable" if target.endswith("-portable") else "dynamic" if target.startswith("linux-") else "native",
"transform": "copy" if variant == "unstripped" else "strip",
"transform": {"unstripped": "copy", "debug-stripped": "strip-debug", "stripped": "strip"}[variant],
"signature": "adhoc-verified" if target.startswith("darwin-") else "not-applicable",
"strip_tool": "contract-strip",
"strip_version": "1.0",
@@ -152,7 +157,7 @@ def make_artifacts(directory: pathlib.Path, *, reverse: bool = False) -> dict[tu
def invoke_stage(source: pathlib.Path, output: pathlib.Path) -> subprocess.CompletedProcess[str]:
return run(stage_tool, source, output, "--expect-targets", 8, "--expect-candidates", 16)
return run(stage_tool, source, output, "--expect-targets", 8, "--expect-candidates", 24)
artifacts = fix / "artifacts"
@@ -166,7 +171,7 @@ require(sorted(path.name for path in staged.iterdir()) ==
cmeta, candidates = manifest(staged / "candidates.tsv")
require(cmeta["marker"] == "# cbm-release-candidates-v1", "candidate marker changed")
require(cmeta.get("targets") == "8" and cmeta.get("candidates") == "16", "candidate counts not bound")
require(cmeta.get("targets") == "8" and cmeta.get("candidates") == "24", "candidate counts not bound")
require([(row["target"], row["variant"]) for row in candidates] ==
[(target, variant) for target in TARGETS for variant in VARIANTS],
"candidate rows are not in canonical target/variant order")
@@ -254,19 +259,33 @@ def result_rows(classifications: dict[tuple[str, str], str], *, shuffle: bool =
def write_results(path: pathlib.Path, rows: list[dict[str, object]]) -> None:
write_tsv(path, "cbm-virustotal-results-v2", {
"scan_objects": 16, "associations": 16, "min_engines_policy": 50,
"scan_objects": 24, "associations": 24, "min_engines_policy": 50,
"min_completed_engines": 71, "max_completed_engines": 71,
}, RESULT_FIELDS, rows)
# Keys are (stripped, unstripped), matching the policy's preferred-candidate
# order and making the asymmetric fallback case unambiguous.
TRUTH = {
("clean", "clean"): "stripped",
("clean", "microsoft-ml"): "stripped",
("microsoft-ml", "clean"): "unstripped",
("microsoft-ml", "microsoft-ml"): "stripped",
}
# Keys are (stripped, debug-stripped, unstripped) in the policy's preferred
# order. Three variants x two tolerated classifications is exactly eight
# combinations, so the eight targets below cover the table EXHAUSTIVELY.
TRUTH_KEYS = [
(a, b, c)
for a in ("clean", "microsoft-ml")
for b in ("clean", "microsoft-ml")
for c in ("clean", "microsoft-ml")
]
def expected_choice(key: tuple[str, str, str]) -> tuple[str, str]:
"""Return (selected_variant, decision) for a (stripped, debug, unstripped) key."""
by_variant = dict(zip(ORDER, key))
clean = [v for v in ORDER if by_variant[v] == "clean"]
if not clean:
return "stripped", "stripped-all-candidates-microsoft-ml"
chosen = clean[0]
if chosen == "stripped":
return "stripped", "stripped-preferred"
others = "-".join(f"{v}:{by_variant[v]}" for v in ORDER if v != chosen)
return chosen, f"{chosen}-clean-after-{others}"
def selection_rows(directory: pathlib.Path) -> tuple[dict[str, str], list[dict[str, str]]]:
@@ -277,11 +296,13 @@ def selection_rows(directory: pathlib.Path) -> tuple[dict[str, str], list[dict[s
# repeat once, proving exact decision behavior and both result bindings.
classes: dict[tuple[str, str], str] = {}
expected: dict[str, str] = {}
truth_keys = list(TRUTH)
expected_decisions: dict[str, str] = {}
require(len(TRUTH_KEYS) == len(TARGETS), "truth table must map one combination per target")
for index, target in enumerate(TARGETS):
pair = truth_keys[index % len(truth_keys)]
classes[target, "stripped"], classes[target, "unstripped"] = pair
expected[target] = TRUTH[pair]
key = TRUTH_KEYS[index]
for variant, classification in zip(ORDER, key):
classes[target, variant] = classification
expected[target], expected_decisions[target] = expected_choice(key)
results_file = fix / "vt-results.tsv"
write_results(results_file, result_rows(classes, shuffle=True))
selected = fix / "selected"
@@ -296,14 +317,7 @@ require([row["target"] for row in selections] == list(TARGETS), "selection order
for row in selections:
target = row["target"]
require(row["selected_variant"] == expected[target], f"truth-table decision wrong for {target}")
target_pair = (classes[target, "stripped"], classes[target, "unstripped"])
expected_decision = (
"unstripped-clean-after-stripped-microsoft-ml"
if expected[target] == "unstripped"
else "stripped-both-microsoft-ml"
if target_pair == ("microsoft-ml", "microsoft-ml")
else "stripped-preferred"
)
expected_decision = expected_decisions[target]
require(row["decision"] == expected_decision,
f"truth-table decision reason wrong for {target}: {row['decision']!r}")
chosen = next(c for c in candidates if c["target"] == target and c["variant"] == expected[target])
+28 -21
View File
@@ -21,7 +21,8 @@ targets = (
"linux-arm64-portable", "darwin-amd64", "darwin-arm64",
"windows-amd64", "windows-arm64",
)
variants = ("unstripped", "stripped")
variants = ("unstripped", "debug-stripped", "stripped")
field_key = {v: v.replace("-", "_") for v in variants}
candidate_fields = (
"target", "variant", "relative_path", "source_sha256", "pre_sign_sha256",
"sha256", "size", "format", "architecture", "linkage", "transform",
@@ -37,7 +38,11 @@ selection_fields = (
"target", "selected_variant", "selected_path", "selected_sha256",
"selected_size", "decision", "unstripped_sha256", "unstripped_scan_path",
"unstripped_classification", "unstripped_analysis_id",
"unstripped_virustotal_url", "stripped_sha256", "stripped_scan_path",
"unstripped_virustotal_url",
"debug_stripped_sha256", "debug_stripped_scan_path",
"debug_stripped_classification", "debug_stripped_analysis_id",
"debug_stripped_virustotal_url",
"stripped_sha256", "stripped_scan_path",
"stripped_classification", "stripped_analysis_id", "stripped_virustotal_url",
)
@@ -74,7 +79,7 @@ for target in targets:
"size": "1234", "format": "pe" if target.startswith("windows-") else "macho" if target.startswith("darwin-") else "elf",
"architecture": target.split("-")[1],
"linkage": "portable" if target.endswith("-portable") else "dynamic" if target.startswith("linux-") else "native",
"transform": "copy" if variant == "unstripped" else "strip",
"transform": {"unstripped": "copy", "debug-stripped": "strip-debug", "stripped": "strip"}[variant],
"signature": "adhoc-verified" if target.startswith("darwin-") else "not-applicable",
"strip_tool": "strip", "strip_version": "contract",
"pair_verification": "same-linker-output-v1", "scan_path": f"objects/{sha}",
@@ -98,15 +103,17 @@ for target in targets:
result_by_pair = {(row["target"], row["variant"]): result for row, result in zip(candidates, results)}
selections = []
for target in targets:
stripped_class = classes[target, "stripped"]
unstripped_class = classes[target, "unstripped"]
chosen = "stripped" if stripped_class == "clean" or unstripped_class != "clean" else "unstripped"
order = ("stripped", "debug-stripped", "unstripped")
clean = [v for v in order if classes[target, v] == "clean"]
chosen = clean[0] if clean else "stripped"
selected = by_pair[target, chosen]
decision = (
"unstripped-clean-after-stripped-microsoft-ml" if chosen == "unstripped"
else "stripped-both-microsoft-ml" if stripped_class == unstripped_class == "microsoft-ml"
else "stripped-preferred"
)
if not clean:
decision = "stripped-all-candidates-microsoft-ml"
elif chosen == "stripped":
decision = "stripped-preferred"
else:
others = "-".join(f"{v}:{classes[target, v]}" for v in order if v != chosen)
decision = f"{chosen}-clean-after-{others}"
row = {
"target": target, "selected_variant": chosen,
"selected_path": f"selected/{target}/" + ("codebase-memory-mcp.exe" if target.startswith("windows-") else "codebase-memory-mcp"),
@@ -117,21 +124,21 @@ for target in targets:
candidate = by_pair[target, variant]
result = result_by_pair[target, variant]
row.update({
f"{variant}_sha256": candidate["sha256"],
f"{variant}_scan_path": candidate["scan_path"],
f"{variant}_classification": result["policy_classification"],
f"{variant}_analysis_id": result["analysis_id"],
f"{variant}_virustotal_url": result["virustotal_url"],
f"{field_key[variant]}_sha256": candidate["sha256"],
f"{field_key[variant]}_scan_path": candidate["scan_path"],
f"{field_key[variant]}_classification": result["policy_classification"],
f"{field_key[variant]}_analysis_id": result["analysis_id"],
f"{field_key[variant]}_virustotal_url": result["virustotal_url"],
})
selections.append(row)
write(root / "release-candidates.tsv", "cbm-release-candidates-v1",
{"targets": 8, "candidates": 16}, candidate_fields, candidates)
{"targets": 8, "candidates": 24}, candidate_fields, candidates)
write(root / "virustotal-candidate-results.tsv", "cbm-virustotal-results-v2",
{"scan_objects": 16, "associations": 16, "min_engines_policy": 50,
{"scan_objects": 24, "associations": 24, "min_engines_policy": 50,
"min_completed_engines": 61, "max_completed_engines": 61}, result_fields, results)
write(root / "release-selection.tsv", "cbm-release-selection-v1",
{"policy": "virustotal-v2", "targets": 8, "candidates": 16},
{"policy": "virustotal-v2", "targets": 8, "candidates": 24},
selection_fields, selections)
PY
@@ -183,7 +190,7 @@ run_notes "$FIX/current.md" "$FIX/first.md"
grep -q 'Intro text.' "$FIX/first.md" || fail "content before section was lost"
grep -q 'Outro text.' "$FIX/first.md" || fail "content after section was lost"
! grep -q 'stale data' "$FIX/first.md" || fail "stale section was appended"
grep -q '16 executable scans' "$FIX/first.md" || fail "candidate scan scope missing"
grep -q '24 executable scans' "$FIX/first.md" || fail "candidate scan scope missing"
grep -q '8 release products' "$FIX/first.md" || fail "product scope missing"
grep -q '3 candidate(s).*Microsoft' "$FIX/first.md" || fail "tolerated findings not disclosed"
grep -q 'no other decisive engine reported malicious or suspicious' "$FIX/first.md" || fail "non-decisive engines are overstated as clean"
@@ -221,4 +228,4 @@ if run_notes "$FIX/reversed.md" "$FIX/reversed-capture.md"; then
fail "reversed verification markers must fail closed"
fi
echo 'PASS: release notes reuse all 16 candidate verdicts, disclose selection, and avoid a duplicate scan'
echo 'PASS: release notes reuse all 24 candidate verdicts, disclose selection, and avoid a duplicate scan'