mirror of
https://github.com/apache/airflow.git
synced 2026-11-04 07:12:48 +00:00
* [main] CI: Upgrade important CI environment * Bump pypdf to 6.21.4 to fix high-severity advisories The CI environment upgrade resolved pypdf 6.21.3, which the dependency review gate rejects (GHSA-mfr5-77xh-9j7h, GHSA-cggf-277g-3q54, GHSA-5849-834g-hvw5). The fixed 6.21.4 is still inside the 4-day exclude-newer window, so add a temporary per-package cutoff for it. Generated-by: Claude Opus 5 * Support pydantic-ai 2.60+ and opensandbox 1.2 in common.ai provider pydantic-ai 2.60 records a durable streamed model request as the bare response rather than a response-plus-events wrapper, so replaying or tracking a streamed request in AirflowDurability failed to decode it. The same release answers a tool retry with the call's own ToolReturnPart (outcome "retried") instead of RetryPromptPart, which broke tests that told retries from results by part type. opensandbox 1.2 allows an egress rule with ports and no target. When the enforced policy did not match the requested one, sorting the allowed targets for the error message raised TypeError on that None, hiding the real reason the sandbox was refused. Both work with the older releases too, which the provider still supports. Generated-by: Claude Opus 5 * Fix provider tests and type checks after the dependency upgrade The tool-part predicates in the common.ai tests returned a plain bool, so mypy could not narrow the message part they matched and the providers type check failed. On Python 3.14 the upgrade moves snowflake-connector-python to 5.1 and snowflake-sqlalchemy to 2.1. connector 5.1's connect() no longer goes through SnowflakeConnection, so patching that class stopped intercepting the connection and the test tried a real login. snowflake-sqlalchemy 2.0.0 started percent-encoding "?" in the URL password, which the URI test pinned to the older, unencoded form. Older Python versions keep the earlier releases, so both tests have to accept either. Generated-by: Claude Opus 5 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
238 lines
10 KiB
Bash
Executable File
238 lines
10 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
# or more contributor license agreements. See the NOTICE file
|
|
# distributed with this work for additional information
|
|
# regarding copyright ownership. The ASF licenses this file
|
|
# to you under the Apache License, Version 2.0 (the
|
|
# "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing,
|
|
# software distributed under the License is distributed on an
|
|
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
# KIND, either express or implied. See the License for the
|
|
# specific language governing permissions and limitations
|
|
# under the License.
|
|
set -euo pipefail
|
|
|
|
MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
|
AIRFLOW_SOURCES=$(cd "${MY_DIR}/../.." && pwd)
|
|
|
|
COLOR_RED=$'\e[31m'
|
|
COLOR_GREEN=$'\e[32m'
|
|
COLOR_YELLOW=$'\e[33m'
|
|
COLOR_BLUE=$'\e[34m'
|
|
COLOR_RESET=$'\e[0m'
|
|
|
|
UV_VERSION="0.13.6"
|
|
|
|
SHIM_DIR="${HOME}/.local/bin"
|
|
SHIM_PATH="${SHIM_DIR}/breeze"
|
|
|
|
# Marker line embedded in the shim — used to tell our own shim apart from a
|
|
# foreign breeze binary (e.g. left over from `uv tool install`).
|
|
SHIM_MARKER="# Apache Airflow breeze shim — managed by scripts/tools/setup_breeze (ADR 0017)."
|
|
|
|
# Shim body version, stamped into the shim as a "# breeze-shim-version: N" line.
|
|
# Bump this whenever the shim body below changes. On startup breeze reads this
|
|
# value from the sources it operates on and compares it with the version baked
|
|
# into the installed shim; if the installed shim is older it warns the user to
|
|
# re-run this script. See warn_if_shim_outdated() in
|
|
# dev/breeze/src/airflow_breeze/utils/path_utils.py.
|
|
SHIM_VERSION="3"
|
|
|
|
# The shim itself. Runs breeze via 'uv run' against the dev/breeze folder of the
|
|
# *current* git worktree, so multiple checkouts / agentic worktrees never
|
|
# share a single global install. '--locked' pins every dependency to the
|
|
# committed dev/breeze/uv.lock, so a fresh third-party release cannot change
|
|
# what breeze runs with until that lock is upgraded. See ADR 0017.
|
|
#
|
|
# When invoked outside any Airflow worktree (e.g. from an SVN release checkout
|
|
# such as asf-dist during a provider release), the shim falls back to, in order:
|
|
# $AIRFLOW_REPO_ROOT (exported by the release docs) if it points at a worktree,
|
|
# then the worktree this shim was installed from (AIRFLOW_SOURCES, baked in below).
|
|
read -r -d '' BREEZE_SHIM_BODY <<BREEZE_SHIM || true
|
|
#!/usr/bin/env bash
|
|
${SHIM_MARKER}
|
|
# breeze-shim-version: ${SHIM_VERSION}
|
|
# Runs breeze from the dev/breeze folder of the current git worktree via 'uv run',
|
|
# so each worktree (e.g. parallel agentic runs) gets its own environment tied to
|
|
# that worktree's source, with dependencies resolved from dev/breeze/uv.lock.
|
|
#
|
|
# Resolution order for the Airflow sources breeze runs from:
|
|
# 1. the current git worktree (per-worktree isolation — see ADR 0017);
|
|
# 2. \$AIRFLOW_REPO_ROOT, if exported and pointing at an Airflow worktree — the
|
|
# release docs export this, so breeze resolves the same way across every
|
|
# release process regardless of where the shim was installed from;
|
|
# 3. the install-time fallback baked in below (the worktree setup_breeze ran from).
|
|
# Steps 2 and 3 apply only when the current directory is not an Airflow worktree,
|
|
# so the fallbacks never override a real worktree and isolation is preserved.
|
|
set -e
|
|
# Install-time fallback: the Airflow sources 'scripts/tools/setup_breeze' was run
|
|
# from. Used only when the current directory is not an Airflow worktree.
|
|
fallback_root="${AIRFLOW_SOURCES}"
|
|
repo_root=\$(git rev-parse --show-toplevel 2>/dev/null) || repo_root=""
|
|
if [ -n "\${repo_root}" ] && [ -d "\${repo_root}/dev/breeze" ]; then
|
|
breeze_root="\${repo_root}"
|
|
elif [ -n "\${AIRFLOW_REPO_ROOT:-}" ] && [ -d "\${AIRFLOW_REPO_ROOT}/dev/breeze" ]; then
|
|
breeze_root="\${AIRFLOW_REPO_ROOT}"
|
|
elif [ -d "\${fallback_root}/dev/breeze" ]; then
|
|
breeze_root="\${fallback_root}"
|
|
else
|
|
echo "breeze: not inside an Airflow worktree, AIRFLOW_REPO_ROOT is unset or not an Airflow worktree, and the install-time fallback '\${fallback_root}/dev/breeze' is missing — re-run scripts/tools/setup_breeze" >&2
|
|
exit 1
|
|
fi
|
|
# '--locked' already guarantees no rewrite; an inherited UV_FROZEN conflicts with it on older uv.
|
|
exec env -u UV_FROZEN AIRFLOW_ROOT_PATH="\${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \\
|
|
uv run --project "\${breeze_root}/dev/breeze" --locked --quiet breeze "\$@"
|
|
BREEZE_SHIM
|
|
|
|
function manual_instructions() {
|
|
echo
|
|
echo "${COLOR_BLUE}Please complete the setup manually:${COLOR_RESET}"
|
|
echo
|
|
echo " 1. Install uv (any reasonably recent version, >= ${UV_VERSION}):"
|
|
echo
|
|
echo " python -m pip install \"uv>=${UV_VERSION}\""
|
|
echo
|
|
echo " 2. Make sure ${SHIM_DIR} exists and is on your PATH."
|
|
echo
|
|
echo " 3. Write the following file to ${SHIM_PATH} and mark it executable (chmod +x):"
|
|
echo
|
|
echo "${BREEZE_SHIM_BODY}" | sed 's/^/ /'
|
|
echo
|
|
exit 1
|
|
}
|
|
|
|
function ensure_uv_installed() {
|
|
if ! command -v "uv" >/dev/null 2>/dev/null; then
|
|
echo
|
|
echo "${COLOR_RED}'uv' is not on PATH. It is required to run breeze via uvx.${COLOR_RESET}"
|
|
export TIMEOUT=0
|
|
if "${MY_DIR}/confirm" "Installing uv via pip"; then
|
|
python -m pip install "uv>=${UV_VERSION}" --upgrade
|
|
echo
|
|
echo "${COLOR_YELLOW}Please close and re-open the shell, then re-run this script.${COLOR_RESET}"
|
|
echo
|
|
exit
|
|
else
|
|
manual_instructions
|
|
fi
|
|
fi
|
|
}
|
|
|
|
function fail_on_legacy_global_install() {
|
|
# If a previous setup did `uv tool install -e ./dev/breeze`, both that install
|
|
# and our shim want to live at ~/.local/bin/breeze. Refuse to proceed until
|
|
# the user removes the legacy install — silent overwrite would corrupt uv's
|
|
# tool state and confuse later upgrades.
|
|
local legacy_uv=0 legacy_pipx=0 uv_tool_dir=""
|
|
if uv tool list 2>/dev/null | grep -q '^apache-airflow-breeze\b'; then
|
|
legacy_uv=1
|
|
else
|
|
# A tool whose environment is corrupted is dropped from `uv tool list` output entirely
|
|
# (uv only mentions it in a stderr warning), while its directory and the breeze entry
|
|
# point it owns both survive. Ask uv where its tools live so such an install is still
|
|
# reported here, instead of surfacing later as an unexplained refusal to install.
|
|
uv_tool_dir=$(uv tool dir 2>/dev/null) || uv_tool_dir=""
|
|
if [[ -n "${uv_tool_dir}" && -d "${uv_tool_dir}/apache-airflow-breeze" ]]; then
|
|
legacy_uv=1
|
|
fi
|
|
fi
|
|
if command -v pipx >/dev/null 2>&1 && pipx list --short 2>/dev/null | grep -q '^apache-airflow-breeze\b'; then
|
|
legacy_pipx=1
|
|
fi
|
|
if [[ ${legacy_uv} -eq 0 && ${legacy_pipx} -eq 0 ]]; then
|
|
return
|
|
fi
|
|
echo
|
|
echo "${COLOR_RED}A legacy global breeze install was detected.${COLOR_RESET}"
|
|
echo "${COLOR_YELLOW}It must be removed before installing the new shim, otherwise both${COLOR_RESET}"
|
|
echo "${COLOR_YELLOW}write to ${SHIM_PATH} and conflict. Run:${COLOR_RESET}"
|
|
echo
|
|
if [[ ${legacy_uv} -eq 1 ]]; then
|
|
echo " uv tool uninstall apache-airflow-breeze"
|
|
fi
|
|
if [[ ${legacy_pipx} -eq 1 ]]; then
|
|
echo " pipx uninstall apache-airflow-breeze"
|
|
fi
|
|
echo
|
|
echo "${COLOR_YELLOW}Then re-run this script.${COLOR_RESET}"
|
|
echo
|
|
exit 1
|
|
}
|
|
|
|
function check_shim_dir_on_path() {
|
|
case ":${PATH}:" in
|
|
*":${SHIM_DIR}:"*) return ;;
|
|
esac
|
|
echo
|
|
echo "${COLOR_YELLOW}Note: ${SHIM_DIR} is not on your PATH.${COLOR_RESET}"
|
|
echo "${COLOR_YELLOW}Add this to your shell rc to make 'breeze' available:${COLOR_RESET}"
|
|
echo
|
|
echo " export PATH=\"${SHIM_DIR}:\$PATH\""
|
|
echo
|
|
}
|
|
|
|
function install_breeze_shim() {
|
|
mkdir -p "${SHIM_DIR}"
|
|
|
|
# A dangling symlink is invisible to the checks below (-e and -f both follow symlinks) and
|
|
# would send our write to the link's target instead of ${SHIM_PATH}.
|
|
if [[ -L "${SHIM_PATH}" && ! -e "${SHIM_PATH}" ]]; then
|
|
local link_target uv_tool_dir=""
|
|
link_target=$(readlink "${SHIM_PATH}")
|
|
uv_tool_dir=$(uv tool dir 2>/dev/null) || uv_tool_dir=""
|
|
if [[ -n "${uv_tool_dir}" && "${link_target}" == "${uv_tool_dir}/"* ]]; then
|
|
# `uv tool uninstall` cannot retract an entry point once it has lost the tool
|
|
# metadata recording it, so a corrupted install leaves this behind for us to reclaim.
|
|
rm -f "${SHIM_PATH}"
|
|
echo "${COLOR_YELLOW}Removed the dangling ${SHIM_PATH} left behind by uv.${COLOR_RESET}"
|
|
else
|
|
echo
|
|
echo "${COLOR_RED}${SHIM_PATH} is a broken symlink to ${link_target}.${COLOR_RESET}"
|
|
echo "${COLOR_YELLOW}Inspect it; if it is safe to replace, remove it and re-run this script.${COLOR_RESET}"
|
|
echo
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
# If something exists at SHIM_PATH that we did not write, do not overwrite it
|
|
# silently — it could be the user's own script.
|
|
if [[ -e "${SHIM_PATH}" ]] && ! grep -qF "${SHIM_MARKER}" "${SHIM_PATH}"; then
|
|
echo
|
|
echo "${COLOR_RED}${SHIM_PATH} already exists and is not the breeze shim managed by this script.${COLOR_RESET}"
|
|
echo "${COLOR_YELLOW}Inspect it; if it is safe to replace, remove it and re-run this script.${COLOR_RESET}"
|
|
echo
|
|
exit 1
|
|
fi
|
|
|
|
if [[ -f "${SHIM_PATH}" ]] && grep -qF "${SHIM_MARKER}" "${SHIM_PATH}"; then
|
|
# Refresh in place — body might have changed across releases.
|
|
printf '%s\n' "${BREEZE_SHIM_BODY}" > "${SHIM_PATH}"
|
|
chmod +x "${SHIM_PATH}"
|
|
echo "${COLOR_GREEN}Refreshed breeze shim at ${SHIM_PATH}.${COLOR_RESET}"
|
|
return
|
|
fi
|
|
|
|
export TIMEOUT=0
|
|
if "${MY_DIR}/confirm" "Install breeze shim at ${SHIM_PATH}"; then
|
|
printf '%s\n' "${BREEZE_SHIM_BODY}" > "${SHIM_PATH}"
|
|
chmod +x "${SHIM_PATH}"
|
|
echo "${COLOR_GREEN}Installed breeze shim at ${SHIM_PATH}.${COLOR_RESET}"
|
|
else
|
|
manual_instructions
|
|
fi
|
|
}
|
|
|
|
ensure_uv_installed
|
|
fail_on_legacy_global_install
|
|
install_breeze_shim
|
|
check_shim_dir_on_path
|
|
|
|
echo
|
|
echo "${COLOR_GREEN}Breeze is configured. Run 'breeze' from any Airflow worktree.${COLOR_RESET}"
|
|
echo
|