nprasad2077/airflow

0 Fork 0

mirror of https://github.com/apache/airflow.git synced 2026-11-04 07:12:48 +00:00

Files

T

Shahar EpsteinandJarek Potiuk 9cc55f2ad3 Update docs for the Python 3.11 baseline (#74159 )

* Update docs for the Python 3.11 baseline

Contributing docs, Breeze docs, Docker stack docs and the user-facing
installation pages used Python 3.10 as the example and default
version. Examples and defaults now use 3.11; historical changelog
entries and ADRs keep their original wording.

* Re-pad the image naming table after dropping Python 3.10

* Fix stale Python 3.10 examples left in the Python 3.11 docs update

Review found commands that still pin a 3.10.x interpreter, which can no
longer install Breeze or Airflow, shell prompts and pytest transcripts
that still show 3.10, and a note that tied the supported Python range to
Airflow 3.2.0 even though the 3.2 to 3.4 release lines still accept 3.10.
Two Breeze examples used options that do not exist. The python-client
release notes presented `breeze --python 3.11` as equivalent to
`UV_PYTHON=3.11`, but the reproducibility guard checks the interpreter
Breeze itself runs on, so only the `UV_PYTHON` form works. The Docker
stack build examples now refer to the default Python version instead of
a hard-coded one, so they stop drifting with every bump.

* Fix remaining Python 3.10 references in docs and image examples

Review found more shell prompts that still show a 3.10 Breeze image.
The air-gapped image example pinned Airflow 2.5.3, which publishes no
3.11 constraints, so its download step fetched a 404 page and failed;
it also used extras that Airflow 3 no longer defines. The contributor
quick start called 3.10 end-of-life before its October 2026 end-of-life
date.

* Update Python version in database setup guide

Updated Python version in setup instructions from 3.8.10 to 3.11.16.

* Fix stale Python 3.10 constraints examples in INSTALL

Generated-by: Claude Opus 5

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

2026-10-06 23:20:50 +02:00

6.0 KiB

Raw Permalink Blame History

Override: release-verify-rc — optional contributor check of your own changes

What this overrides

Adds an optional step to the release-verify-rc skill for Airflow provider waves: after (or instead of) the artefact checks, verify that every change the person running the skill authored is in the release candidate, and let them test those changes against the RC distributions running in Breeze. Run it when the user asks to "check my changes in the RC", "test my changes in the providers release", "tick my boxes", or similar. It is not part of the mechanical PASS / FAIL verdict and never replaces it.

Everything below is read-only on GitHub except the two steps marked confirm first, which follow the draft-then-confirm rule in airflow-pr-draft-summary.

Step A — Find the wave and the user's entries

Find the testing issue: gh issue list -R apache/airflow --search "Status of testing Providers in:title" --state all --limit 3. The newest open one is the wave; the previous one is needed in step 3.
From the issue body, take the checkbox lines that end in : @<handle> — those are the user's PRs. Lines under Linked issues: that mention the user are issues they reported on someone else's PR, not their boxes.
List the user's merged PRs touching providers/ since the previous wave (gh pr list --author <handle> --state merged --search "merged:<prev>..<this>"), ignoring [v3-*-test] backports — providers are released from main. Any PR not in the current issue must be either in the previous issue (already released) or in a provider of this wave that has no section in the issue — report the latter, since it has no box to tick.

Step B — Verify each change is in the RC

The wave's RC tags are providers-<name>/<version>rcN created on the preparation date. For every (PR, provider) pair confirm all three, and report any gap:

the PR's squash commit on upstream/main is an ancestor of the RC tag (git merge-base --is-ancestor);
providers/<path>/docs/changelog.rst at the tag mentions #<PR>;
the RC wheel from PyPI contains the changed code — download it with pip download --no-deps and look for a distinctive added line inside the wheel (zipfile). Docs-only and test-only PRs have nothing in the wheel; say so.

Git refuses shell loops that call git in some agent sandboxes; a small Python script calling git via subprocess works.

Step C — Install the whole wave from files

Make sure dist/ of the checkout Breeze will run from is empty, then download every RC distribution of the wave plus the latest released apache-airflow and apache-airflow-core into it (pip download --no-deps --only-binary=:all: --dest dist -r <requirements>).
Write one verification Dag to files/dags/ (gitignored; Breeze's Dags folder), with one task per changed behaviour. Each task exercises the changed code path through the RC distribution, replaces external services with unittest.mock (no real credentials), prints what it checked, and raises on the old behaviour. Skip — do not fail — a task whose optional extra is not installed. Run ruff format and ruff check on it.

Hand the user the command to run from that same checkout — breeze start-airflow needs Docker and an interactive terminal, so the agent does not run it:

breeze start-airflow --mount-sources remove --use-distributions-from-dist \
  --use-airflow-version wheel --python 3.11 --backend postgres --load-default-connections

Show the user a table of the tasks and what each log should contain, plus any change they can additionally check by hand in the UI.

Known pitfalls: No airflow package found means dist/ of the checkout Breeze runs from is empty (e.g. it was started from the main checkout instead of a worktree) or DISTRIBUTION_FORMAT is set to sdist. A Docker build failing with error getting credentials … "pass": executable file not found is the user's credsStore helper, not Airflow — point it out and let the user fix it.

Step D — Report back (confirm first)

Once the user says the Dag passed:

Tick the boxes (confirm first). Re-fetch the issue body immediately before editing, change only [ ] → [x] on lines ending in : @<handle> for PRs verified in step B, and abort if the number of matched lines differs from what was reported.
Draft the testing-issue comment (confirm first) listing what was tested per provider, including changes that are in the wave but missing from the issue. End it with the Drafted-by: footer.
When the user drafts their [VOTE] reply, mention that they used this check to test all their changes in the candidate.
Offer cleanup: delete the downloaded wheels and the verification Dag, and suggest breeze down.

Why

Contributors are asked to test their changes in every providers wave, but finding the right distributions, installing them all together, and building a quick reproduction for each change is tedious enough that it is often skipped. The agent can do the mechanical part — inclusion checks, download, test Dag — so the human spends their time on actually looking at the behaviour.

6.0 KiB Raw Permalink Blame History