mirror of
https://github.com/apache/airflow.git
synced 2026-11-04 07:12:48 +00:00
Jarek Potiuk
e69c1881b3
* Refresh the image registry cache in a workflow that cannot be cancelled A branch built on push refreshed its registry cache from inside the CI run, which cancels in progress on the next push. With no cache the build runs cold, which widens the window the next push cancels in, which leaves the cache missing -- v3-3-test held only the default Python version's amd64 cache because that was the one matrix entry that finished in time. The refresh also has to cover every Python version: a version with no cache entry builds from scratch on every run, and selective checks narrows the list on a text-only push. * Refresh constraints from a workflow of its own, covering every flavour The manual constraints refresh shared a workflow with the automatic uv.lock push, so it was filtered by that workflow's paths and buried under its name in the Actions list. The documented procedures pointed at local builds -- refresh_images.sh for the image cache, breeze commands for the constraints -- which need a buildx/qemu setup, a committer login to ghcr.io, and enough bandwidth to push the layers. The workflows need none of that, and unlike a local run they always cover every Python version and every constraint flavour. * Exempt a manually started cache refresh from the push cancellation A push-triggered refresh should give way to the next one -- it is building the cache of a commit that is no longer the tip. A manual run should not: it is started because the branch has no cache and the push runs are the thing that keeps getting cancelled.
73 lines
3.1 KiB
YAML
73 lines
3.1 KiB
YAML
# Licensed to the Apache Software Foundation (ASF) under one
|
|
# or more contributor license agreements. See the NOTICE file
|
|
# distributed with this work for additional information
|
|
# regarding copyright ownership. The ASF licenses this file
|
|
# to you under the Apache License, Version 2.0 (the
|
|
# "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing,
|
|
# software distributed under the License is distributed on an
|
|
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
# KIND, either express or implied. See the License for the
|
|
# specific language governing permissions and limitations
|
|
# under the License.
|
|
#
|
|
---
|
|
name: Refresh constraints
|
|
# Refreshes the pinned constraint files on demand, when waiting for the automatic path is not an
|
|
# option -- a `main` / `vX-Y-test` build that cannot be made green, or newly released providers
|
|
# that a release candidate has to pick up before it is promoted.
|
|
#
|
|
# All three constraint flavours are refreshed, not just the PyPI one:
|
|
#
|
|
# * `constraints-source-providers-X.Y.txt` -- providers resolved from this checkout's sources,
|
|
# which is what a contributor's Breeze environment and CI install from;
|
|
# * `constraints-no-providers-X.Y.txt` -- core dependencies alone, used to install Airflow
|
|
# without any provider pinned;
|
|
# * `constraints-X.Y.txt` -- providers as published on PyPI, which is what users install with.
|
|
#
|
|
# Refreshing only the PyPI flavour would leave CI and Breeze on the older pins, so the three
|
|
# would disagree about the same dependency.
|
|
#
|
|
# This does the same work as the automatic `uv.lock` push (whose workflow it calls), against the
|
|
# ref you name rather than the branch the push landed on. Run it from `main`: the workflow
|
|
# definition and `breeze` come from `main`, while the sources come from `ref`, so refreshing a
|
|
# release line needs no cherry-pick to `vX-Y-test` / `vX-Y-stable` first. The `constraints-X-Y`
|
|
# branch written to is derived from that ref's `branch_defaults.py`.
|
|
#
|
|
# See dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md.
|
|
on: # yamllint disable-line rule:truthy
|
|
workflow_dispatch:
|
|
inputs:
|
|
ref:
|
|
description: "Repo reference to refresh constraints from (e.g. v3-3-stable). Empty = this branch."
|
|
required: false
|
|
default: ""
|
|
type: string
|
|
upgrade-to-newer-dependencies:
|
|
description: "Upgrade deps to newest from PyPI"
|
|
required: false
|
|
type: boolean
|
|
default: true
|
|
permissions:
|
|
contents: read
|
|
concurrency:
|
|
# Two refreshes of the same ref would race each other onto the same `constraints-X-Y` branch.
|
|
group: refresh-constraints-${{ inputs.ref || github.ref }}
|
|
cancel-in-progress: false
|
|
jobs:
|
|
refresh-constraints:
|
|
name: "Refresh constraints"
|
|
uses: ./.github/workflows/update-constraints-on-push.yml
|
|
permissions:
|
|
contents: write
|
|
packages: write
|
|
with:
|
|
ref: ${{ inputs.ref }}
|
|
upgrade-to-newer-dependencies: ${{ inputs.upgrade-to-newer-dependencies }}
|
|
secrets:
|
|
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
|