Files
codebase-memory-mcp/.github/workflows/_build.yml
T
Martin Vogel e3a9fa34ed ci: let apt fail over between mirrors instead of hanging
The hosted Ubuntu x64 runners read the archive through
mirror+file:/etc/apt/apt-mirrors.txt. With Acquire::Retries above 0
(including the apt default of 3), a failover from a dead first entry
of that list (connection refused, DNS failure or a stalled host) to
the next entry goes idle: apt-get fetches a few InRelease files from
the second mirror and then waits until the step timeout. With
Acquire::Retries=0 the same failover completes. Seen on noble
(apt 2.8.3) and jammy (apt 2.4.14).

Every apt step this branch hardened now picks the retry count at
runtime, applied to both update and install: 0 when the runner has the
mirror list (moving to the next mirror is the retry), 3 otherwise
(arm64 runners and any host with plain sources keep their retries).
--error-on=any, the http and https timeouts and the step timeouts are
unchanged.

Proof: the step text extracted from the edited _build.yml (update plus
install of zlib1g-dev), run under bash -e in ubuntu:24.04 and
ubuntu:22.04 containers with runner-shaped mirror lists:
- first mirror refused: exit 0 in 8 s / 9 s, every index and deb
  fetched from the second mirror
- first mirror stalled (192.0.2.1): exit 0 in 68 s / 69 s, one 30 s
  timeout each for update and install, then the second mirror
- healthy first mirror: exit 0 in 7 s / 9 s
- no mirror list, plain sources: the prefix keeps Retries=3, exit 0
  in 8 s / 7 s
- every mirror dead: update exits 100 in under 1 s, install skipped
- revert check, the previous step text with a dead first mirror
  (refused or stalled): idle with no Packages index at the 180 s cap
  on both releases

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-11-01 12:32:45 +01:00

512 lines
21 KiB
YAML

# Reusable: build release binaries on all supported platforms
name: Build
on:
workflow_call:
inputs:
version:
description: 'Version string (e.g. v0.8.0)'
type: string
default: ''
attest:
description: 'Generate build provenance attestations for release artifacts'
type: boolean
default: false
scan_candidates:
description: 'VirusTotal-scan both candidates before tuple-local selection (release must keep true)'
type: boolean
default: true
secrets:
VIRUS_TOTAL_SCANNER_API_KEY:
description: 'VirusTotal API key used for candidate selection'
required: false
permissions:
contents: read
id-token: write
attestations: write
jobs:
build-unix:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
goos: linux
goarch: amd64
cc: gcc
cxx: g++
- os: ubuntu-24.04-arm
goos: linux
goarch: arm64
cc: gcc
cxx: g++
# The macOS minimum (14.0) is pinned in Makefile.cbm, not inherited
# from this runner's SDK, so both darwin binaries still start on macOS 14.
- os: macos-15
goos: darwin
goarch: arm64
cc: cc
cxx: c++
- os: macos-15-intel
goos: darwin
goarch: amd64
cc: cc
cxx: c++
runs-on: ${{ matrix.os }}
# Every leg is BLOCKING — no continue-on-error. The darwin-amd64 binary
# built on macos-15-intel must ship with every release; if that runner is
# unavailable the build fails loudly rather than silently publishing a
# release with no Intel macOS binary (a user-reported gap). macos-15-intel
# is GitHub's supported Intel image through Aug 2027 (the last x86_64 macOS
# runner); revisit the Intel leg before that retirement.
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install deps (Ubuntu)
if: startsWith(matrix.os, 'ubuntu')
timeout-minutes: 10
run: APT_RETRIES=3; [ -f /etc/apt/apt-mirrors.txt ] && APT_RETRIES=0; sudo apt-get -o Acquire::Retries=$APT_RETRIES -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 --error-on=any update && sudo apt-get -o Acquire::Retries=$APT_RETRIES -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 install -y zlib1g-dev
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI
env:
VERSION: ${{ inputs.version }}
CC: ${{ matrix.cc }}
CXX: ${{ matrix.cxx }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" "CC=$CC" "CXX=$CXX"
else
scripts/build.sh --with-ui "CC=$CC" "CXX=$CXX"
fi
- name: Frontend integrity scan
if: matrix.goos == 'linux' && matrix.goarch == 'amd64'
run: scripts/security-ui.sh
- name: Derive immutable stripped and unstripped release candidates
run: |
scripts/ci/prepare-release-candidates.sh \
"${{ matrix.goos }}" "${{ matrix.goarch }}" \
--binary build/c/codebase-memory-mcp \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
"release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/THIRD_PARTY_NOTICES.md"
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: |
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/candidate-provenance.tsv
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/stripped/*
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/unstripped/*
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-${{ matrix.goos }}-${{ matrix.goarch }}
path: release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}
if-no-files-found: error
build-windows:
runs-on: windows-latest
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: msys2/setup-msys2@66cd2cce69caa17b53920067426061ca1de3a884 # v2
with:
msystem: CLANG64
path-type: inherit
install: >-
mingw-w64-clang-x86_64-clang
mingw-w64-clang-x86_64-zlib
make
python
unzip
zip
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI
shell: msys2 {0}
env:
VERSION: ${{ inputs.version }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" CC=clang CXX=clang++
else
scripts/build.sh --with-ui CC=clang CXX=clang++
fi
- name: Derive immutable stripped and unstripped release candidates
shell: msys2 {0}
run: |
scripts/ci/prepare-release-candidates.sh windows amd64 \
--binary build/c/codebase-memory-mcp.exe \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
release-candidates/windows-amd64/THIRD_PARTY_NOTICES.md
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: |
release-candidates/windows-amd64/candidate-provenance.tsv
release-candidates/windows-amd64/stripped/*
release-candidates/windows-amd64/unstripped/*
release-candidates/windows-amd64/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-windows-amd64
path: release-candidates/windows-amd64
if-no-files-found: error
build-windows-arm64:
# Native ARM64 Windows binary via the CLANGARM64 toolchain on the
# windows-11-arm runner (the same toolchain the test suite already
# exercises). Without it, ARM-Windows users fall back to the x86_64
# binary under emulation, and npm/pip on native-ARM Node/Python 404.
runs-on: windows-11-arm
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: msys2/setup-msys2@66cd2cce69caa17b53920067426061ca1de3a884 # v2
with:
msystem: CLANGARM64
path-type: inherit
install: >-
mingw-w64-clang-aarch64-clang
mingw-w64-clang-aarch64-zlib
make
python
unzip
zip
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI
shell: msys2 {0}
env:
VERSION: ${{ inputs.version }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" CC=clang CXX=clang++
else
scripts/build.sh --with-ui CC=clang CXX=clang++
fi
- name: Derive immutable stripped and unstripped release candidates
shell: msys2 {0}
run: |
scripts/ci/prepare-release-candidates.sh windows arm64 \
--binary build/c/codebase-memory-mcp.exe \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
release-candidates/windows-arm64/THIRD_PARTY_NOTICES.md
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: |
release-candidates/windows-arm64/candidate-provenance.tsv
release-candidates/windows-arm64/stripped/*
release-candidates/windows-arm64/unstripped/*
release-candidates/windows-arm64/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-windows-arm64
path: release-candidates/windows-arm64
if-no-files-found: error
build-linux-portable:
# Fully static Linux binaries (gcc -static on Ubuntu).
# Runs on any Linux distro without shared library dependencies.
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
runner: ubuntu-latest
- arch: arm64
runner: ubuntu-24.04-arm
runs-on: ${{ matrix.runner }}
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install deps
timeout-minutes: 10
run: APT_RETRIES=3; [ -f /etc/apt/apt-mirrors.txt ] && APT_RETRIES=0; sudo apt-get -o Acquire::Retries=$APT_RETRIES -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 --error-on=any update && sudo apt-get -o Acquire::Retries=$APT_RETRIES -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 install -y zlib1g-dev
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI (static)
env:
VERSION: ${{ inputs.version }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" CC=gcc CXX=g++ STATIC=1
else
scripts/build.sh --with-ui CC=gcc CXX=g++ STATIC=1
fi
- name: Verify release static linking
run: |
file build/c/codebase-memory-mcp
ldd build/c/codebase-memory-mcp 2>&1 | grep -q "not a dynamic executable" || ldd build/c/codebase-memory-mcp 2>&1 | grep -q "statically linked"
- name: Derive immutable stripped and unstripped release candidates
run: |
scripts/ci/prepare-release-candidates.sh \
linux "${{ matrix.arch }}-portable" \
--binary build/c/codebase-memory-mcp \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
"release-candidates/linux-${{ matrix.arch }}-portable/THIRD_PARTY_NOTICES.md"
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: |
release-candidates/linux-${{ matrix.arch }}-portable/candidate-provenance.tsv
release-candidates/linux-${{ matrix.arch }}-portable/stripped/*
release-candidates/linux-${{ matrix.arch }}-portable/unstripped/*
release-candidates/linux-${{ matrix.arch }}-portable/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-linux-${{ matrix.arch }}-portable
path: release-candidates/linux-${{ matrix.arch }}-portable
if-no-files-found: error
# One blocking join owns the security-sensitive transition from candidates
# to public artifacts. Each target tuple contributes exactly three behavior-
# equivalent candidates (unstripped/debug-stripped/stripped); the selector may
# only choose within that tuple. A hard result on ANY sibling blocks
# the all-target release rather than selecting around a real detection.
select-package:
needs: [build-unix, build-windows, build-windows-arm64, build-linux-portable]
runs-on: ubuntu-latest
timeout-minutes: 330
permissions:
contents: read
id-token: write
attestations: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Download all native candidate pairs
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: release-candidates-*
path: ${{ runner.temp }}/release-candidate-artifacts
- name: Stage the exact candidate scan set
run: |
python3 scripts/ci/stage-release-candidates.py \
"$RUNNER_TEMP/release-candidate-artifacts" \
release-candidate-scan \
--expect-targets 8 \
--expect-candidates 24
# Staging copied and re-hashed every byte into the immutable object
# set. Drop the redundant artifact-download tree before the VT wait.
test -n "$RUNNER_TEMP"
rm -rf -- "$RUNNER_TEMP/release-candidate-artifacts"
test ! -e "$RUNNER_TEMP/release-candidate-artifacts"
- name: Submit all 24 candidates to VirusTotal
if: ${{ inputs.scan_candidates }}
uses: crazy-max/ghaction-virustotal@936d8c5c00afe97d3d9a1af26d017cfdf26800a2 # v5.0.0
id: virustotal-candidates
with:
vt_api_key: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
files: release-candidate-scan/objects/*
request_rate: 4
# The established gate tolerates only one Microsoft malicious label
# ending !ml and rejects every other malicious/suspicious combination.
# Every candidate still must complete with the minimum engine coverage.
- name: Wait for complete VirusTotal candidate results
if: ${{ inputs.scan_candidates }}
env:
VT_API_KEY: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
VT_ANALYSIS: ${{ steps.virustotal-candidates.outputs.analysis }}
VT_EXPECTED_SCAN_SET: release-candidate-scan/scan-set.tsv
VT_ASSOCIATIONS: release-candidate-scan/associations.tsv
VT_RESULTS_PATH: release-candidate-scan/vt-results.tsv
VT_POLL_TIMEOUT_SECONDS: 14400
MIN_ENGINES: 50
run: scripts/ci/check-virustotal.sh
- name: Select one scanned candidate inside every target tuple
if: ${{ inputs.scan_candidates }}
run: |
python3 scripts/ci/select-release-candidates.py \
--candidates release-candidate-scan/candidates.tsv \
--objects-dir release-candidate-scan/objects \
--results release-candidate-scan/vt-results.tsv \
--out-dir release-selection
# This is intentionally reachable only from dry-run.yml's explicit
# skip_virustotal input. release.yml pins scan_candidates=true.
- name: Mark unscanned dry run and default to stripped candidates
if: ${{ !inputs.scan_candidates }}
run: |
echo '::warning::VirusTotal candidate scan explicitly skipped; selecting stripped candidates as unscanned-dry-run.'
python3 scripts/ci/select-release-candidates.py \
--candidates release-candidate-scan/candidates.tsv \
--objects-dir release-candidate-scan/objects \
--default-stripped \
--out-dir release-selection
- name: Reclaim rejected and redundant candidate bytes
run: |
# release-selection owns fresh, hash-verified copies of the eight
# chosen executables. No later release step may consume the 24-object
# scan pool, so discard it before packaging to bound runner disk use.
rm -rf -- release-candidate-scan/objects
test ! -e release-candidate-scan/objects
- name: Package the selected exact bytes
env:
VERSION: ${{ inputs.version }}
run: |
for target in \
linux-amd64 linux-arm64 \
linux-amd64-portable linux-arm64-portable \
darwin-amd64 darwin-arm64 \
windows-amd64 windows-arm64; do
goos="${target%%-*}"
goarch="${target#*-}"
selected_name=codebase-memory-mcp
test "$goos" != windows || selected_name=codebase-memory-mcp.exe
selected="release-selection/selected/$target/$selected_name"
notices="release-candidate-scan/notices/$target/THIRD_PARTY_NOTICES.md"
test -f "$selected"
test -f "$notices"
# The expected hash comes from the selector's provenance manifest,
# not from the file being checked. package-release.sh independently
# re-hashes the file and therefore exercises manifest -> file ->
# archive/MCPB binding instead of comparing a file to itself.
selected_sha="$(awk -F '\t' -v wanted="$target" '!header { for (i = 1; i <= NF; i++) { if ($i == "target") target_col = i; if ($i == "selected_sha256") sha_col = i } if (target_col && sha_col) header = 1; next } $target_col == wanted { count++; selected = $sha_col } END { if (count != 1 || length(selected) != 64 || selected ~ /[^0-9a-f]/) exit 3; print selected }' release-selection/release-selection.tsv)"
scripts/package-release.sh "$goos" "$goarch" \
--selected-binary "$selected" \
--expected-sha256 "$selected_sha" \
--third-party-notices "$notices" \
--out-dir "release-containers/$target"
done
- name: Verify every packaged executable matches its selected hash
run: |
python3 scripts/ci/verify-release-selection.py \
--selection release-selection/release-selection.tsv \
--archive-dir release-containers
- name: Prepare durable candidate-selection evidence
if: ${{ always() }}
run: |
mkdir -p selection-evidence
test ! -f release-candidate-scan/candidates.tsv || \
cp release-candidate-scan/candidates.tsv selection-evidence/release-candidates.tsv
test ! -f release-candidate-scan/vt-results.tsv || \
cp release-candidate-scan/vt-results.tsv selection-evidence/virustotal-candidate-results.tsv
test ! -f release-selection/release-selection.tsv || \
cp release-selection/release-selection.tsv selection-evidence/release-selection.tsv
- name: Attest selected containers and decision evidence
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: |
release-containers/**/*.tar.gz
release-containers/**/*.zip
release-containers/**/*.mcpb
selection-evidence/*.tsv
- name: Preserve candidate selection evidence
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-selection-evidence
path: selection-evidence
if-no-files-found: error
# Keep the established artifact names so the reusable smoke workflow and
# all callers consume only the selected, byte-verified containers.
- name: Upload linux-amd64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-amd64
path: release-containers/linux-amd64/*
if-no-files-found: error
- name: Upload linux-arm64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-arm64
path: release-containers/linux-arm64/*
if-no-files-found: error
- name: Upload linux-amd64-portable containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-amd64-portable
path: release-containers/linux-amd64-portable/*
if-no-files-found: error
- name: Upload linux-arm64-portable containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-arm64-portable
path: release-containers/linux-arm64-portable/*
if-no-files-found: error
- name: Upload darwin-amd64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-darwin-amd64
path: release-containers/darwin-amd64/*
if-no-files-found: error
- name: Upload darwin-arm64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-darwin-arm64
path: release-containers/darwin-arm64/*
if-no-files-found: error
- name: Upload windows-amd64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-windows-amd64
path: release-containers/windows-amd64/*
if-no-files-found: error
- name: Upload windows-arm64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-windows-arm64
path: release-containers/windows-arm64/*
if-no-files-found: error