mirror of
https://github.com/apache/airflow.git
synced 2026-11-04 07:12:48 +00:00
Jarek PotiukandRahul Vats
48435351c4
* Refresh the hardened base image mirror every six hours Docker rebuilds the hardened images in place and moves the dhi.io apt repository on at the same time. With a weekly mirror, the base images Airflow builds on lag the repository by up to a week, and any apt install whose dependencies were rebuilt in between fails: build-essential currently cannot be installed because the mirrored base has an older libbz2 than the bzip2 the repository offers (docker-hardened-images/catalog#689). Mirroring copies manifests between registries without pulling layers, so running it several times a day is cheap. Generated-by: Claude Opus 5 * Pin the hardened base images to digests the CI has passed on Building on the mirror's moving tags made CI non-reproducible: any refresh of the mirror changed the image every build started from, without a change in the repository and without the full CI having run on it. A CI run could pass or fail depending only on when it ran, and rebuilding the same commit later gave a different image. The images now build on a digest, and only the upgrade check moves it, in a PR that must pass the full CI before the new base is used. The mirror refreshes every six hours and triggers that upgrade check when it moves a tag past its pin, so the pins follow Docker's rebuilds - and the dhi.io apt repository moving with them - within hours rather than drift behind it. Every pinned and mirrored digest keeps a tag of its own, so a pinned image never becomes untagged. The hardened images keep stock Debian's apt sources beside dhi.io, and apt took whichever offered the higher version, mixing Debian builds into an image whose dependencies were built by Docker. dhi.io is now preferred whenever it publishes the package. Generated-by: Claude Opus 5 * Let dhi.io's package versions win over the image's own Docker renumbered its builds ("1.0.8-6+dhi2" became "1.0.8-6+deb13u0+dhi2", which sorts lower) and its apt repository moved to the new numbers before its images did. Every package depending on a renumbered build - bzip2, libffi-dev, libgpg-error-dev among them - is then uninstallable, because apt will not replace the image's higher-numbered copy with the repository's lower-numbered one, and the CI image build fails (docker-hardened-images/catalog#689). Every package in the image comes from dhi.io, so letting its repository decide their versions is safe, and it keeps the build working through such a renumbering. Generated-by: Claude Opus 5 * Move the base images to Docker's current Python patch releases Docker stops rebuilding a Python patch release once it publishes the next, so the 3.11.17, 3.12.15 and 3.13.16 images still carry the old package numbering that the dhi.io apt repository has moved past, and installing packages on them fails (docker-hardened-images/catalog#689). Docker fixed it in the images of the current patch releases. Generated with the Python part of the upgrade check, which also pins the new images to the digests the mirror serves for them. Generated-by: Claude Opus 5 * Compare the registry host exactly in the mirror test Generated-by: Claude Opus 5 * Lower the dhi.io apt pin below 1000 so dependencies can resolve At priority 1001 the Package:* dhi.io preference forces dhi.io's build even as a downgrade, so apt keeps dhi.io's libpq5 17.11 and postgresql-client-18 (which needs libpq5 >= 18.6 from PGDG) cannot be installed. Below 1000 the preference still favours dhi.io for the packages it publishes but lets apt take a newer version from another repository to satisfy a dependency. The forced "downgrade" the >1000 priority handled no longer arises now the base image is pinned to a digest CI has passed on. * Rank the PostgreSQL apt repository above the dhi.io preference install_postgres adds the PGDG repository, but the hardened image's Package:* dhi.io preference outranks Debian's default, so apt kept dhi.io's libpq and postgresql-client-18 (which needs a newer libpq5 than dhi.io carries) failed with held broken packages. Pinning the PostgreSQL origin above the dhi.io preference lets apt take libpq from PGDG to satisfy the client. * Use US spelling favors to pass the docs spellcheck The docker-stack docs spellcheck rejects the British spelling favours in build.rst; use the US spelling there and in the matching install-script comments. --------- Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>