Files
Jarek Potiuk 359d361de9 Base CI and PROD images on hardened Python images (#73038)
* Base CI and PROD images on hardened Python images

Squashed from 19 commits (backup branch: backup/hardened-python-base-images-pre-rebase) before rebasing onto main.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Base images on trixie and keep a legacy bookworm PROD image flavor

The 3.4.x line keeps publishing the previous images - Python compiled from sources on bookworm - as a deprecated legacy flavor, so the PROD image has to be buildable both ways while the CI image only needs the hardened trixie one. A weekly canary builds and checks the legacy images so they do not rot between releases.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Support the trixie hardened images' Debian-packaged Python and test legacy images like regular ones

The Debian 13 hardened images install Python as Debian packages under /usr instead of /opt/python, so the image has to find the base Python's prefix and the system-Python guard must not mistake those packages for Debian's own Python. The legacy canary now runs the regular PROD image tests, not just a smoke check, so the legacy images are covered as well as the default ones.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Drop CI image packages that Debian trixie no longer has

The CI image is trixie-only now: libgcc-11-dev was a bookworm-era pin (build-essential brings the matching libgcc-14-dev) and software-properties-common was removed in Debian 13; nothing uses its add-apt-repository.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Trust Microsoft's newer signing key for the trixie MSSQL repository

Microsoft signs its Debian 13 package repository with its 2025 general signing key rather than the 2015 release key, so apt on trixie rejected the repository as unsigned.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Restore the PAM configs the trixie hardened images strip

The Debian 13 hardened images ship libpam-runtime and passwd already installed but without their /etc/pam.d files, so installing them restored nothing and adding the CI image's airflow user failed with pam_start() errors.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Document the anonymous Debian package repository of the hardened images

The trixie hardened images add Docker's own package repository, whose rebuilt packages take precedence over Debian's - users extending the image should know where their packages come from, and that no credentials are needed for it.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Keep a second hardened Python out of the trixie images

On trixie, libenchant-2-dev's build dependencies need python3, which Docker's package repository satisfies with its own python-3.13 - a second interpreter in the 3.12 image that broke building C extensions. pyenchant only needs the runtime library, which it finds through the ldconfig cache, and the guard now fails early on any hardened Python other than the image's own.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Install the Python build dependencies for the legacy image flavor

Dropping the packages that only existed to compile Python left the legacy flavor compiling a Python without sqlite3, lzma or readline - CPython builds those modules only when their headers are present, and the image could not even run airflow. The build now also fails when one of them is missing.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Refresh the shared library cache after apt runs in hardened images

The hardened images ship libc-bin without its dpkg trigger, so libraries installed by apt stayed invisible to ctypes.util.find_library() - the CI image's docs build failed with "The 'enchant' C library was not found". A dpkg post-invoke hook runs ldconfig after every apt run, also in images extending ours.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Warn about apt pulling a second Python into hardened images

Extending a hardened image with apt packages that depend on python3 silently installs another Python next to the image's own, and C extensions then build against the wrong one - users need to know to check before adding such packages.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Add trixie and sigstore to the docs spelling wordlist

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Count only host keys when waiting for sshd in the CI image

OpenSSH 10 in trixie prints a '# host:port' banner comment per connection to ssh-keyscan's stdout, so the line count never matched the three host keys and every test job waited until it timed out.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Invalidate the CI image mount cache for the trixie base

The uv cache restored into CI image builds held wheels compiled against bookworm's libraries - an xmlsec wheel built against bookworm's MD5-enabled libxmlsec1 failed to import on trixie, which dropped MD5. The cache key does not encode the base OS, so it has to be bumped when that changes.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Install Java that exists on trixie in the Java examples

Debian trixie ships no Java 17. The Java SDK e2e image keeps Java 17 for Spark 3.5 by taking it from the Temurin repository, and the docs example uses trixie's Java 21.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Let old C extensions build in the trixie CI image

GCC 14 turns incompatible pointer types, integer conversions and implicit declarations into errors, and the hardened Python passes -Werror=implicit-function-declaration on to extensions - the lowest-dependency tests' old sdists (xmlsec 1.3.14, ibm-db 3.2.0) only built because those were warnings.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Document the stricter C extension builds of the hardened images

Users extending the image with old source distributions hit the same GCC 14 errors as CI, and need to know how to relax them.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Export KinD logs when the lang-SDK K8S test setup fails

A failing lang-SDK helm upgrade raised past the log export that every other step of the complete K8S test run does, so the cluster was deleted with nothing saved and the job's KinD logs artifact stayed empty.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_014np1fVAxHmGaBFt3qebJ8x

* Keep apt from replacing the hardened images' Python

Debian 13's python3 packages conflict with the hardened images' python-3.X
packages, so installing NodeSource's nodejs (which depends on python3) into
the lang-SDK runtime image made apt remove the image's Python 3.11, and the
migration pods crash-looped. The image now holds its Python packages, so such
an install fails instead, and the runtime image copies the node binary from
the official node image rather than installing the deb.

Generated-by: Claude Opus 5

* Keep perl-base in step with the perl apt can install

The hardened apt repository publishes git, which depends on perl, but not perl
itself, while the hardened images ship their own build of perl-base. Debian's
perl depends on exactly its own perl-base version, so apt refused to install git
and the hardened CI image build failed. The image now moves perl-base to the
version of the installable perl before installing its packages; when the
hardened repository publishes a matching perl, the versions agree and nothing
changes.

Generated-by: Claude Opus 5

* Mirror the hardened base images only from main

The mirror is the default BASE_IMAGE of every Airflow build, and the job holds packages:write plus the Docker Hub credentials. Scheduled runs are always on main, but anyone with write access could dispatch the workflow from another branch and replace the mirrored images.

Generated-by: Claude Opus 5

* Check the Adoptium signing key fingerprint in the Java e2e image

The key is downloaded over HTTPS at build time; pinning its fingerprint means a replaced key on the server fails the build instead of being trusted for every package from that repository.

Generated-by: Claude Opus 5

* Allow mirroring the hardened base images from v3-*-test branches

The release branches build images from the same mirror, so maintainers need to refresh it from there too.

Generated-by: Claude Opus 5
2026-10-20 15:34:13 +02:00

470 lines
31 KiB
ReStructuredText

.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
Dockerfile Changelog
====================
The ``Dockerfile`` does not strictly follow the `SemVer <https://semver.org/>`_ approach of
Apache Airflow when it comes to features and backwards compatibility. While Airflow code strictly
follows it, the ``Dockerfile`` is really a way to give users a conveniently packaged Airflow
using standard container approach, so occasionally there are some changes in the building process
or in the entrypoint of the image that require slight adaptation of how it is used or built.
The Changelog below describes the changes introduced in each version of the docker images released by
the Airflow team.
:note: The Changelog below concerns only the convenience production images released at
`Airflow DockerHub <https://hub.docker.com/r/apache/airflow>`_ . The images that are released
there are usually built using the ``Dockerfile`` released together with Airflow. However, you are
free to take latest released ``Dockerfile`` from Airflow and use it to build an image for
any Airflow version from the ``Airflow 2`` line. There is no guarantee that it will work, but if it does,
then you can use latest features from that image to build images for previous Airflow versions.
Airflow 3.4.0
~~~~~~~~~~~~~
* The image does not support Python 3.10 anymore as Python 3.10 reaches end of life in October 2026.
Images are released for Python 3.11, 3.12, 3.13 and 3.14.
* The ``tdsodbc`` package was added to the image so that the FreeTDS ODBC driver
(``libtdsodbc.so``) is available for connecting to Sybase/TDS databases via ODBC.
* The ``libxmlsec1-openssl`` package was added to the image. ``libxmlsec1`` contains no crypto
engine of its own, so ``import xmlsec`` (used by ``python3-saml``) failed with
``libxmlsec1-openssl.so.1: cannot open shared object file``.
In Airflow 3.4.0 the base image changed again - from ``debian:bookworm-slim`` with a Python compiled in
the image, to the Debian 13 "trixie" `Docker Hardened Image <https://dhi.io>`_ for Python, which already
carries a Python built by Docker. This removes the Python compilation, and the source download and signature verification
that went with it, from the build entirely - the OS dependency layer of the production image went from
roughly 210s to 105s in a local cache-disabled build.
* The ``BASE_IMAGE`` arg now defaults to ``ghcr.io/apache/airflow/base/python:<version>-debian13-dev``,
which is Airflow's public mirror of the upstream ``dhi.io/python`` image. Pulling from ``dhi.io``
directly requires a ``docker login dhi.io``; pulling the mirror requires nothing. If you pass your own
``BASE_IMAGE``, it now has to be an image that already provides Python - a bare ``debian:bookworm-slim``
only works with ``AIRFLOW_IMAGE_FLAVOR="legacy"``.
* The image is based on Debian 13 "trixie" instead of Debian 12 "bookworm". Library packages renamed in
trixie's 64-bit ``time_t`` transition (for example ``libssl3t64``, ``libldap2``) have to be used in
custom ``RUNTIME_APT_DEPS``, and the MySQL-compatible client comes from MariaDB 11.8 (with
``mariadb-client-compat`` providing the ``mysql`` named commands).
* The new ``AIRFLOW_IMAGE_FLAVOR`` arg (``hardened`` by default) can be set to ``legacy`` to build the
image as before: Python compiled from sources on a plain Debian ``BASE_IMAGE`` such as
``debian:bookworm-slim``. Images of that flavor are published for the 3.4.x line, are deprecated, and
will be removed in Airflow 3.5.0. The flavor is recorded in the ``org.apache.airflow.image.flavor`` label.
* The ``PYTHON_LTO`` arg was removed. It only controlled Link-Time Optimization while compiling Python;
the legacy flavor always compiles with LTO. To build a FIPS-compliant image, point ``BASE_IMAGE`` at a FIPS
variant of the hardened image, for example ``dhi.io/python:3.13.16-debian13-fips-dev`` (those variants
require a paid Docker subscription).
* Python is installed under ``/usr`` by the base image, as Debian packages. ``/usr/python`` is a symlink to it, and the
``/usr/local/bin`` symlinks are unchanged, so paths that worked before keep working.
* The hardened images ship the standard library with no ``.pyc`` files, so the build compiles it into the
image. This keeps the behaviour introduced in Airflow 3.1.4 (see below): the standard library is owned
by root while the image runs as ``airflow``, so a missing bytecode cache could never be filled and every
import would leak a negative ``dentry``.
* Python in the hardened images is not the plain upstream build: Docker patches it (``pkgutil.get_data()``
rejects ``..`` resource paths), builds it without profile-guided optimization, and compiles it with a
1 MiB default thread stack. The image restores the previous 8 MiB thread stack. See
:ref:`image-hardened-base` for these and the other differences from the previous images.
As with any base image change, some ``apt`` packages that used to be present as a side effect are not
there any more. The hardened images are deliberately minimal - they ship no compiler, no ``curl``,
``wget``, ``git`` or ``gzip``, and a stripped ``/etc`` - so a custom image that relied on something being
present may need to install it explicitly. See :doc:`Building the image <build>`.
Airflow 3.1.4
~~~~~~~~~~~~~
In Airflow 3.1.4, the images are build without removing of .pyc and .pyo files when Python is built.
This increases the size of the image slightly (<0.5%), but improves performance of Python in the container
because Python does not need to recompile the files on the first run but more importantly, if you use
``exec`` to run Health Checks, removed .pyc files caused a small but ever growing memory leak in the Unix
kernel connected to negative ``dentries`` created when .pyc files were attempted to be compiled and failed.
This over time could lead to out-of-memory issues on the host running the container.
More information about ``dentries`` can be found in `this article <https://lwn.net/Articles/814535/>`_.
Airflow 3.1.0
~~~~~~~~~~~~~
In Airflow 3.1.0 we changed the base images used for building the Airflow images. Previously the images
were based on "official" Python images from DockerHub, however those images sometimes lag behind the
latest security patches and their maintainers made decisions about using older setuptools and pip versions,
however we want to be able to use the latest versions of those tools to build the images. Therefore
we decided to use ``bookworm-slim`` images from Debian as the base images for Airflow images and we compile
and install the latest Python version in the image based on the official packages from the Python Software
Foundation. This change should be transparent for those who extend the images or use them "as is", however
for those who want to build custom images, there are changes to the build-args passed to the ``docker build``
command:
* The ``PYTHON_BASE_IMAGE`` arg is no longer supported - the base image is always ``debian:bookworm-slim``
* Instead ``AIRFLOW_PYTHON_VERSION`` arg is introduced to select the Python version to be installed in the
image. It should be full Python version - for example ``3.12.7`` or two-digit version like ``3.12``
(the latest released Python patch version will be installed in that case).
* Python is now installed in ``/usr/python/`` directory not in ``/usr/local/`` as it was previously.
This should not affect regular uses because there are symlinks from ``/usr/local/bin/`` to
``/usr/python/bin/``, and ``/usr/python/bin`` is first on the PATH, however if the ``/usr/local/``
location has been hard-coded to - for example - detect ``site-packages`` dir, the folders moved to
the ``/usr/python`` directory.
There might be other subtle changes in the image due to the change of the base image, some packages (
especially dev libraries installed with ``apt`` might not be installed by default if they were installed
as a side-effect in the original image, however that should only affect those who want to customise the image.
They should be able to install in their custom images following the :doc:`Building the image <build>`
The Python 3.13 image for Airflow 3.1.0 (both slim and regular) does not contain ``fab`` provider because
at the time of release the ``fab`` provider did not support Python 3.13. It should be possible to install
future versions of ``fab`` provider (when they support Python 3.13) in the image using ``pip install``,
and until it is possible - if you need ``fab`` provider (and particularly FABAuthManager) in the
image, you should use Python 3.12 image. You can use experimental KeycloakAuthManager in Python 3.13 image
or develop your own AuthManager.
Airflow 3.0.2
~~~~~~~~~~~~~
* The ``git`` binary was added to the image by default which is needed for the git provider to work.
Airflow 3.0.1
~~~~~~~~~~~~~
* The ``ARM`` image is not experimental any more - we are running the ARM tests regularly in our CI. The
``MySQL`` support for `ARM` images however is still experimental.
Airflow 3.0
~~~~~~~~~~~
* The ``virtualenv`` package is no longer installed in the reference container. Airflow 3 and standard
provider relies on ``venv`` module from Python standard library.
* There is no ``pipx`` and ``mssql-cli`` installed in the image by default. We recommend to use
``uv tool`` instead of ``pipx`` and ``mssql-cli`` is not used in the image by default as we do not
have mssql metadata support any more.
* The ``INSTALL_PACKAGES_FROM_CONTEXT`` arg changed to ``INSTALL_DISTRIBUTIONS_FROM_CONTEXT``
* The parameter ``UPGRADE_INVALIDATION_STRING`` is renamed to ``UPGRADE_RANDOM_INDICATOR_STRING``
Airflow 2.11.1
~~~~~~~~~~~~~~
* The image does not support Python 3.9 anymore as Python 3.9 reached end of life on October 31, 2025 and
The Airflow 2.11.1 has been released on February 20, 2026 and Airflow 2.11.1 does not support Python 3.9.
Airflow 2.10
~~~~~~~~~~~~
* The image does not support Debian-Bullseye(11) anymore. The image is based on Debian-Bookworm (12).
Airflow 2.9
~~~~~~~~~~~
* The "latest" image (i.e. default Airflow image when ``apache/airflow`` is used or
``apache/airflow:slim-latest``) uses now the newest supported Python version. Previously it was using
the "default" Python version which was Python 3.8 as of Airflow 2.8. With Airflow reference images
released for Airflow 2.9.0, the images are going to use Python 3.12 as this is the latest supported
version for Airflow 2.9 line. Users can use Python 3.8 by using ``apache/airflow:2.9.0-python3.8`` and
``apache/airflow:slim-2.9.0-python-3.8`` images respectively so while the change is potentially
breaking, it is very easy to switch to the previous behaviour.
* The ``PIP_USER`` flag is removed and replaced by ``VIRTUAL_ENV`` pointing to ``~/.local`` where Airflow
is installed. This has the effect that the Airflow installation is treated as a regular virtual environment,
but unlike a regular virtualenv, the ``~/.local`` directory is seen as ``system level`` and when the
worker creates dynamically the virtualenv with ``--system-site-packages`` flag, the Airflow installation and all
packages there are also present in the new virtualenv. When you do not use the flag, they are not
copied there which is a backwards-compatible behaviour with having ``PIP_USER`` set.
* The image contains latest ``uv`` binary (latest at the moment of release) - which is a new faster
replacement for ``pip``. While the image is still using ``pip`` by default, you can use ``uv``
to install packages and - experimentally - you can also build custom images with
``--arg AIRFLOW_USE_UV=true`` which will us ``uv`` to perform the installation. This is an experimental
support, as ``uv`` is very fast but also a very new feature in the Python ecosystem.
* Constraints used to install the image are available in "${HOME}/constraints.txt" now - you can use them
to install additional packages in the image without having to find out which constraints you should use.
* The image adds ``libev`` library to the image as it is required by cassandra driver for Python 3.12, also
``libev`` will be used in other Python versions as a more robust and faster way for cassandra driver
to handle events.
Airflow 2.8
~~~~~~~~~~~
* 2.8.3
* The ``gosu`` binary was removed from the image. This is a potentially breaking change for users who relied on
``gosu`` to change the user in the container. The ``gosu`` binary was removed because it was a source of
security vulnerabilities as it was linked against older Go standard libraries.
* The ``smtp`` provider is now included in the list of providers installed by default in the image.
* 2.8.1
* Fixed a discrepancy in MySQL client libraries. In 2.8.0 if not specify ``INSTALL_MYSQL_CLIENT_TYPE`` build arg
during build custom X86 image by default packages would be compiled by using **MariaDB** libraries,
however **MySQL** libraries were installed in the final image.
* 2.8.0
* Add ``libxmlsec1`` and ``libxmlsec1-dev`` libraries to dev PROD image and ``libxmlsec1`` library to runtime PROD
image as it is required by ``python3-saml`` library.
* The image is based on ``Debian Bookworm`` in 2.8.0 rather than ``Debian Bullseye``. This might cause some
problems when building custom images. You are advised to make sure your system level dependencies are
working with ``Debian Bookworm``. While all reference images of Airflow 2.8.0 are built on ``Debian Bookworm``,
it is still possible to build deprecated custom ``Debian Bullseye`` based image in 2.8.0 following the
* By default the images now have "MariaDB" client installed. Previous images had "MySQL" client installed.
The MariaDB client is a drop-in replacement for "MySQL" one and is compatible with MySQL. This might
be a breaking change for users who used MySQL client in their images, however those should be very
specific cases and vast majority of users should not see any difference. Users can still use
MySQL client by setting ``INSTALL_MYSQL_CLIENT_TYPE=mysql`` build arg and build the custom X86 image.
The ARM image always uses MariaDB client, this argument is ignored. The "mysql" apt repository is
removed from the /etc/apt/sources.list.d/ and if you want to install anything from this repository when
extending the images, you need to manually add the right key and repository in your Dockerfile,
following the instructions in `A Quick Guide to Using the MySQL APT repository <https://dev.mysql.com/doc/mysql-apt-repo-quick-guide/en/>`_.
Airflow 2.7
~~~~~~~~~~~
* 2.7.3
* Add experimental feature for select type of MySQL Client libraries during the build custom image via ``INSTALL_MYSQL_CLIENT_TYPE``
build arg. ``mysql`` for install MySQL client libraries from `Oracle APT repository <https://dev.mysql.com/doc/mysql-apt-repo-quick-guide/en/>`_,
``mariadb`` for install MariaDB client libraries from `MariaDB repository <https://mariadb.com/kb/en/mariadb-package-repository-setup-and-usage/#mariadb-repository>`_.
The selection of MySQL Client libraries only available on AMD64 (x86_64) for ARM docker image it will always install
MariaDB client.
* Docker CLI version in the image is bumped to 24.0.6 version.
* PIP caching for local builds has been enabled to speed up local custom image building
* 2.7.0
* As of now, Python 3.7 is no longer supported by the Python community. Therefore, to use Airflow 2.7.0, you must ensure your Python version is
either 3.8, 3.9, 3.10, or 3.11.
Airflow 2.6
~~~~~~~~~~~~~
* 2.6.3
* Add ``libgeos-dev`` library to runtime PROD image as it is required by BigQuery library on ARM image
* 2.6.0
* Snowflake provider installed by default
* The ARM experimental image adds support for MySQL via MariaDB client libraries.
Airflow 2.5
~~~~~~~~~~~
* 2.5.1
* The ARM experimental image adds support for MSSQL
* 2.5.0
* The docker CLI binary is now added to the images by default (available on PATH). Version 20.10.9 is used.
Airflow 2.4
~~~~~~~~~~~
* 2.4.0
* You can specify additional ``pip install`` flags when you build the image via ``ADDITIONAL_PIP_INSTALL_FLAGS``
build arg.
* Support for ``Debian Buster`` was dropped, including the possibility of building customized images as
``Debian Buster`` reached end of life.
Airflow 2.3
~~~~~~~~~~~
* 2.3.0
* Airflow 2.3 ``Dockerfile`` is now better optimized for caching and "standalone" which means that you
can copy **just** the ``Dockerfile`` to any folder and start building custom images. This,
however, requires `Buildkit <https://docs.docker.com/develop/develop-images/build_enhancements/>`_
to build the image because we started using features that are only available in ``Buildkit``.
This can be done by setting ``DOCKER_BUILDKIT=1`` as an environment variable
or by installing `the buildx plugin <https://docs.docker.com/buildx/working-with-buildx/>`_
and running ``docker buildx build`` command.
* MySQL is experimentally supported on ARM through MariaDB client library
* Add Python 3.10 support
* Switch to ``Debian Bullseye`` based images. ``Debian Buster`` is deprecated and only available for
customized image building.
* Add Multi-Platform support (AMD64/ARM64) in order to accommodate MacOS M1 users
* Build parameters which control if packages and Airflow should be installed from context file were
unified
* The ``INSTALL_FROM_PYPI`` arg was removed - it is automatically detected now.
* The ``INSTALL_FROM_DOCKER_CONTEXT_FILES`` arg changed to ``INSTALL_PACKAGES_FROM_CONTEXT``
Airflow 2.2
~~~~~~~~~~~
* 2.2.4
* Add support for both ``.piprc`` and ``pip.conf`` customizations
* Add ArtifactHub labels for better discovery of the images
* Update default Python image to be 3.7
* Build images with ``Buildkit`` (optional)
* Fix building the image on Azure with ``text file busy`` error
* 2.2.3
* No changes
* 2.2.2
* No changes
* 2.2.1
* Workaround the problem with ``libstdcpp`` TLS error
* 2.2.0
* Remove AIRFLOW_GID (5000) from Airflow images (potentially breaking change for users using it)
* Added warnings for Quick-start docker compose
* Fix warm shutdown for celery worker (signal propagation)
* Add Oauth libraries to PROD images
* Add Python 3.9 support
Airflow 2.1
~~~~~~~~~~~
* MySQL changed the keys to sign their packages on 17 Feb 2022. This caused all released images
to fail when being extended. As result, on 18 Feb 2021 we re-released all
the ``2.2`` and ``2.1`` images with latest versions of ``Dockerfile``
containing the new signing key.
There were subtle changes in the behaviour of some 2.1 images due to that (more details below)
Detailed `issue here <https://github.com/apache/airflow/issues/20911>`_
:note: that the changes below were valid before image refreshing on 18 Feb 2022.
Since all the images were refreshed on 18 Feb with the same ``Dockerfile``
as 2.1.4, the changes 2.1.1 -> 2.1.3 are
effectively applied to all the images in 2.1.* line.
The images refreshed have also those fixes added:
* All 2.1.* image versions refreshed on 18 Feb 2022 have those fixes applied:
* Fix building the image on Azure with ``text file busy`` error
* Workaround the problem with ``libstdcpp`` TLS error
* Remove AIRFLOW_GID (5000) from Airflow images (potentially breaking change for users using it)
* Added warnings for Quick-start docker compose
* Add Oauth libraries to PROD images
Original image Changelog (before the refresh on 18 Feb 2022):
* 2.1.4
* Workaround the problem with ``libstdcpp`` TLS error
* fixed detection of port number in connection URL
* Improve warnings for quick-start-docker compose
* Fix warm shutdown for celery worker (signal propagation)
* 2.1.3
* fixed auto-creation of user to use non-deprecated ``create user`` rather than ``user_create``
* remove waiting for celery backend for ``worker`` and ``flower`` commands rather than ``scheduler`` and ``celery`` only
* remove deprecated ``airflow upgradedb`` command from Airflow 1.10 in case upgrade is requested
* Add Python 3.9 support
* 2.1.2
* No changes
* 2.1.1
* Fix failure of lack of default commands (failed when no commands were passed)
* Added ``_PIP_ADDITIONAL_REQUIREMENTS`` development feature
* 2.1.0
* Unset default ``PIP_USER`` variable - which caused PythonVirtualEnv to fail
Airflow 2.0
~~~~~~~~~~~
* MySQL changed the keys to sign their packages on 17 Feb 2022. This caused all released images
to fail when being extended. As result, on 18 Feb 2021 we re-released all
the ``2.2`` and ``2.1`` images with latest versions of ``Dockerfile``
containing the new signing key.
There were no changes in the behaviour of 2.0.2 image due to that
Detailed `issue here <https://github.com/apache/airflow/issues/20911>`_ .
Only 2.0.2 image was regenerated, as 2.0.1 and 2.0.0 versions are hardly used and it is unlikely someone
would like to extend those images. Extending 2.0.1 and 2.0.0 images will lead to failures of "missing key".
* 2.0.2
* Set correct PYTHONPATH for ``root`` user. Allows to run the image as root
* Warn if the deprecated 5000 group ID was used for airflow user when running the image
(should be 0 for the OpenShift compatibility). Fails if the group 5000 was used with any other user
(it would not work anyway but with cryptic errors)
* Set umask as 002 by default, so that you can actually change the user id used to run the image
(required for OpenShift compatibility)
* Skip checking the DB and celery backend if CONNECTION_CHECK_MAX_COUNT is equal to 0
* 2.0.1
* Avoid reverse IP lookup when checking DB availability. This helped to solve long delays on misconfigured
docker engines
* Add auto-detection of redis and amqp broker ports
* Fixed detection of all user/password combinations in URLs - helps in auto-detecting ports and testing
connectivity
* Add possibility to create Admin user automatically when entering the image
* Automatically create system user when different user than ``airflow`` is used. Needed for OpenShift
compatibility
* Allows to exec to ``bash`` or ``python`` if specified as parameters
* Remove ``airflow`` command if it is specified as first parameter of the ``run`` command
* 2.0.0
* Initial release of the image based on Debian Buster
Changes after publishing the images
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Occasionally our images need to be regenerated using newer ``Dockerfiles`` or constraints.
This happens when an issue is found or a breaking change is released by our dependencies
that invalidates the already released image, and regenerating the image makes it usable again.
While we cannot assure 100% backwards compatibility when it happens, we at least document it
here so that users affected can find the reason for the changes.
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| Date | Versions | Potentially breaking change | Reason | Link to Pull Request / Issue |
+==============+=============+===================================================================================+=======================================================+================================================+
| 19 July 2026 | 3.3.0 | * The ``apache-airflow-providers-databricks`` provider was upgraded from 6.13.0 | ``thrift`` 0.23.0 conflicted with | https://github.com/apache/airflow/issues/69603 |
| | | to 7.16.1 in the Python 3.11 and 3.12 images. | ``databricks-sql-connector`` 4.x (which requires | |
| | | * ``databricks-sql-connector`` upgraded from 2.0.2 to 4.2.5 and ``thrift`` from | ``thrift<0.21.0``), so the resolver kept the newer | |
| | | 0.23.0 to 0.16.0. | thrift and downgraded the connector, dragging the | |
| | | | databricks provider back to 6.13.0 (Oct 2024) with | |
| | | | import errors on 3.3.0. | |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 28 Feb 2026 | 2.11.1 | * The grpcio==1.78.0 is used instead of 1.78.1 in Python 3.12 images. | The 1.78.1 version was yanked - caused major outage. | https://github.com/grpc/grpc/issues/41725. |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 23 Feb 2026 | 2.11.1 | * Several dependencies upgraded with minor versions/patchlevels | Virtualenv project pulling out released 20.37.0 | https://github.com/pypa/virtualenv/issues/3061 |
| | | * See https://github.com/apache/airflow/commit/4a55543 for details | | |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 19 July 2025 | 3.0.3 | * The ``standard`` provider upgraded from 1.4.0 to 1.4.1 | Sensor skipping issue | https://github.com/apache/airflow/pull/53455 |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 24 Jun 2025 | 3.0.2 | * The ``fab`` provider upgraded from 2.2.0 to 2.2.1 | FAB provider user creation did not work | https://github.com/apache/airflow/issues/51854 |
| | | * ``common.messaging`` provider upgraded from 1.0.2 to 1.0.3 | importing SQS message failed with circular import | https://github.com/apache/airflow/issues/51770 |
| | | * git binary is added to the image | git bundle need it | https://github.com/apache/airflow/pull/51580 |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 02 Aug 2024 | 2.9.3 | * The ``apache-airflow-providers-fab`` upgraded from 1.2.1 to 1.2.2 | FAB provider logout did not work for 2.9.3 | https://github.com/apache/airflow/issues/40922 |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 12 Mar 2024 | 2.8.3 | * The image was refreshed with new dependencies (pandas < 2.2 and SMTP provider | Both dependencies caused breaking changes | https://github.com/apache/airflow/pull/37748 |
| | | 1.6.1) | | https://github.com/apache/airflow/pull/37701 |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 16 Dec 2023 | All 2.* | * The AIRFLOW_GID 500 was removed | MySQL repository is removed after the key expiry | https://github.com/apache/airflow/issues/36231 |
| | | * MySQL ``apt`` repository key changed. | fiasco | |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 17 June 2022 | 2.2.5 | * The ``Authlib`` library downgraded from 1.0.1 to 0.15.5 version | Flask App Builder not compatible with Authlib >= 1.0 | https://github.com/apache/airflow/pull/24516 |
| | 2.3.0-2.3.2 | | | |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+
| 18 Jan 2022 | All 2.2.*, | * The AIRFLOW_GID 500 was removed | MySQL changed keys to sign their packages on 17 Jan | https://github.com/apache/airflow/pull/20912 |
| | 2.1.* | * MySQL ``apt`` repository key changed. | 2022 | |
+--------------+-------------+-----------------------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------+