Shahar EpsteinandJarek Potiuk
3e0ed979c1
* Require Python 3.11 in all distributions Every pyproject.toml that declares requires-python now says >= 3.11 and drops the Python 3.10 trove classifier, the tomli build requirement that only existed for 3.10 and the per-version dependency markers for 3.10. Lock files are regenerated. With 3.10 gone, the tomli fallbacks in scripts, dev tooling and tests go too, along with the tomli additional_dependencies of the prek hooks and the blacken-docs py310 target. The tomllib imports that only sat inside functions because of the fallback move to the top of their modules. The four dev/registry and scripts/ci/prek modules left out of the earlier UP017 pass also take their timezone.utc to datetime.UTC fixes here. The minimum host Python checked by check-min-python-version, the default Python of run_generate_constraints.py and the provider cut-off date parsing (fromisoformat, which reads a trailing Z from 3.11) follow the new baseline. ruff infers py311 from the new requires-python, which enables the PEP 646 star-index fix (UP044) applied here and makes two more rules reportable: UP042 (str, Enum to StrEnum) alters what str() and format() return for serialized enum values, and ASYNC109 means dropping the public timeout parameter from async APIs. Both need per-site judgement, so they go on the ignore list with that rationale. The datamodel-codegen target moves to 3.11 too, with use-specialized-enum turned off for the same reason: otherwise the generated Task SDK and airflowctl enums would switch from (str, Enum) to StrEnum. * Drop the Python 3.10 provider metadata shim With provider distributions now requiring Python 3.11, breeze can derive requires-python and classifiers from the default and allowed versions again. The patch-exclusion test and the matching example in MANAGING_PROVIDERS_LIFECYCLE.rst move from 3.10.0 to 3.11.0, the oldest patch release CI still covers. * Keep Breeze's locked dependency versions when dropping Python 3.10 Relocking Breeze for the Python 3.11 baseline ran while recently bumped versions were still inside the 4-day exclude-newer window, so it downgraded pyjwt (reverting #73969), prek, boto3/botocore and filelock. Relocking from the previous lock keeps them and only drops the packages Python 3.11 no longer needs. Generated-by: Claude Opus 5 --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Docker Image for Apache Airflow
For the ease of deployment in production, the community releases a production-ready reference container image.
The Apache Airflow community, releases Docker Images which are reference images for Apache Airflow.
Every time a new version of Airflow is released, the images are prepared in the
apache/airflow DockerHub
for all the supported Python versions.
You can find the following images there (Assuming Airflow version 3.4.0):
apache/airflow:latest- the latest released Airflow image with default Python version (3.12 currently)apache/airflow:latest-pythonX.Y- the latest released Airflow image with specific Python versionapache/airflow:3.4.0- the versioned Airflow image with default Python version (3.12 currently)apache/airflow:3.4.0-pythonX.Y- the versioned Airflow image with specific Python version
Those are "reference" regular images. They contain the most common set of extras, dependencies and providers that are often used by the users and they are good to "try-things-out" when you want to just take Airflow for a spin,
You can also use "slim" images that contain only core Airflow and are about half the size of the "regular" images but you need to add all the Reference for package extras and providers that you need separately via Building the image.
apache/airflow:slim-latest- the latest released Airflow image with default Python version (3.12 currently)apache/airflow:slim-latest-pythonX.Y- the latest released Airflow image with specific Python versionapache/airflow:slim-3.4.0- the versioned Airflow image with default Python version (3.12 currently)apache/airflow:slim-3.4.0-pythonX.Y- the versioned Airflow image with specific Python version
The Apache Airflow image provided as convenience package is optimized for size, and it provides just a bare minimal set of the extras and dependencies installed and in most cases you want to either extend or customize the image. You can see all possible extras in Reference for package extras. The set of extras used in Airflow Production image are available in the Dockerfile.
However, Airflow has more than 90 community-managed providers (installable via extras) and some of the default extras/providers installed are not used by everyone, sometimes others extras/providers are needed, sometimes (very often actually) you need to add your own custom dependencies, packages or even custom providers. You can learn how to do it in Building the image.
The production images are built in DockerHub from released version and release candidates. There are also images published from branches but they are used mainly for development and testing purpose. See Airflow Git Branching for details.
Usage
The AIRFLOW_HOME is set by default to /opt/airflow/ - this means that dags
are in default in the /opt/airflow/dags folder and logs are in the /opt/airflow/logs
The working directory is /opt/airflow by default.
If no AIRFLOW__DATABASE__SQL_ALCHEMY_CONN variable is set then SQLite database is created in
${AIRFLOW_HOME}/airflow.db.
For example commands that start Airflow see: Executing commands.
Airflow requires many components to function as it is a distributed application. You may therefore also be interested in launching Airflow in the Docker Compose environment, see: Running Airflow in Docker.
You can use this image in Helm Chart as well.