ci: fail fast when an apt mirror stalls

On 2026-11-01 two Linux jobs hung in `sudo apt-get update` right after
"Get:5 https://archive.ubuntu.com/ubuntu noble-security InRelease" until
their job limit; the build never started in either:

- PR 2641, test-tsan x86 (run 41251045506, job 124688326533): the
  "Install deps (Ubuntu)" step ran 2 h until the 120-minute job limit.
- PR 2395, test-unix gcc x86 (run 41256855177, job 124705168264): the
  same step ran 4 h until the 240-minute job limit.

Both logs show the azure mirror ignored after about 15 s, the fallback to
archive.ubuntu.com, and then no output until the cancel. The default apt
http timeout (120 s) did not end the stall.

Rule applied to every workflow step whose run block calls apt:

- A step-level timeout-minutes. 10 for steps that only provision packages
  (green runs on 2026-11-01 and the last green dry run: 5 to 76 s); 20 for
  the three pr.yml steps that also run scripts/build.sh (green: 246 to
  331 s). A stalled mirror now fails the step in minutes instead of
  holding the runner until the job limit.
- `-o Acquire::Retries=3 -o Acquire::http::Timeout=30
  -o Acquire::https::Timeout=30` on every apt-get update and install,
  with the -o options before the subcommand as in
  test-infrastructure/Dockerfile.msan.

Nothing else changes: no action, runner, package or step-order changes.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel committed 2026-11-01 09:23:09 +01:00
1 parent f159b4c21d
commit 339cd51660
15 files changed
+63 -34

No files matched your search

+2 -1
View File
@@ -95,7 +95,8 @@ jobs:
- name: Install build dependencies
if: ${{ matrix.language == 'c-cpp' }}
run: sudo apt-get update && sudo apt-get install -y zlib1g-dev
timeout-minutes: 10
run: sudo apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 update && sudo apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30 install -y zlib1g-dev
# PR c-cpp: compile the third-party objects (tree-sitter grammars,
# sqlite, mimalloc, lz4, zstd, the tree-sitter runtime: ~1.3 GB of