Jarek Potiuk
468ab0a583
* Spend fewer GitHub writes on fork CI job statuses and watch the limits The fork CI monitor's per-job commit statuses are its largest use of the GITHUB_TOKEN limits that every workflow of the repository shares (500 content-creating requests an hour, 80 a minute). During the pilot they already fill the monitor's per-run budget whenever a couple of full-matrix fork runs are active, which would leave little headroom for other workflows once fork CI is enabled for every contributor. A "running" status for each job doubled the writes while saying little: the monitor only runs every 15 minutes, so it lagged behind the fork's own run page, and the aggregate Fork CI status already shows the run as running and links it. Until now the only sign of hitting the limits was failures, so each run now logs the quota left and how many calls GitHub rate-limited, and Slack hears about rate limiting that persists. Generated-by: Claude Opus 5 * Report pull requests whose fork job statuses lag far behind Generated-by: Claude Opus 5
2.7 KiB
Table of Contents generated with DocToc
dev/ directory guidelines
Scripts
New scripts in dev/ must be standalone Python scripts (not bash). Each script must include
inline script metadata
placed after the Apache License header, so that uv run can execute it without any prior
installation:
#!/usr/bin/env python3
# Licensed to the Apache Software Foundation (ASF) ...
# http://www.apache.org/licenses/LICENSE-2.0
# ...
# /// script
# requires-python = ">=3.9"
# dependencies = [
# "some-package",
# ]
# ///
If the script only uses the standard library, omit the dependencies key but keep the
requires-python line.
Run scripts with:
uv run dev/my_script.py [args...]
Document uv run (not python) as the invocation method in READMEs and instructions.
Fork CI job statuses and GitHub's rate limits
The fork CI monitor (breeze ci fork-ci-reconcile, run by .github/workflows/fork-ci-monitor.yml)
reports every job of a gated pull request's fork run as its own commit status. Those writes are the
largest share of what the monitor does with the repository's GITHUB_TOKEN, whose rate limits every
workflow of apache/airflow shares (see "Write budget" in
dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md).
If the budget is exceeded, turn the job statuses off: set job_statuses = false in
.github/fork-ci-gate.toml. Signs that it is exceeded are the monitor's Slack note that job
statuses "have not fit in the budget" (posted only after three runs in a row deferred them)
staying for hours, pull requests listed because their fork run finished more than an hour ago with
job statuses still not posted, its Slack note that GitHub rate-limited it in two runs in a row, the
"Rate limits left" line of its run log approaching zero, or other workflows of the repository
failing on secondary rate limits. The aggregate Fork CI / Tests (AMD)
status and CI passed are posted either way, so turning job statuses off loses detail, not gating.
Lowering MAX_JOB_STATUSES_PER_RUN in fork_ci_reconcile.py is the softer alternative.