Tell agents to stop when breeze cannot run, not use a host venv (#75599)

* Tell agents to stop when breeze cannot run instead of using a host venv

AGENTS.md said never to run pytest on the host but also listed
`uv run --project` commands, and had no guidance for when breeze is
unavailable. A sandboxed agent whose breeze calls were refused by the
container gateway built a host virtualenv with Airflow 2.11.1 instead of
reporting the failure.

Narrow the host exception to the listed `uv run` commands, tell agents
to stop and report when breeze cannot run, and document the breeze
command for testing providers against a released Airflow version
(generated from contributing-docs). Note in the magpie override that the
breeze allows do not make breeze work behind the container gateway.

* Record skill-eval run for the breeze-unavailable AGENTS.md guidance

.apache-magpie-overrides/setup-isolated-setup-install.md

@@ -74,6 +74,11 @@ plus read-only `docker ps` / `docker info`. No `docker rm`, no
 `docker network prune`, nothing that writes outside the breeze workflow —
 those stay prompt-gated.
 
+The allows only skip the permission prompt. They do not make breeze work
+behind the magpie container gateway, which refuses Docker calls breeze
+depends on (BuildKit sessions, untyped `docker inspect`) by design, so breeze
+cannot run in a session whose `DOCKER_HOST` points at the gateway.
+
 ## Scope notes
 
 - These land in the gitignored, per-machine `.claude/settings.local.json`,

AGENTS.md

@@ -23,7 +23,8 @@ Don't spell out **Directed Acyclic Graph** except for historical context.
 - Install prek: `uv tool install prek`
 - Enable commit hooks: `prek install`
 - Install breeze shim (one-time, per machine): `scripts/tools/setup_breeze` — installs `~/.local/bin/breeze` that runs breeze via `uv run --locked` from the current git worktree's `dev/breeze`, with dependencies pinned by `dev/breeze/uv.lock` (so each worktree, including ephemeral agent worktrees, gets its own breeze tied to its sources). See [ADR 0017](dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md).
-- **Never run pytest, python, or airflow commands directly on the host** — always use `breeze`.
+- **Never run pytest, python, or airflow commands directly on the host** — always use `breeze`. The only exception is the `uv run --project <PROJECT> ...` commands listed below, run against the workspace sources.
+- If a step needs `breeze` and it cannot run (Docker missing, or its Docker calls refused by a sandbox or socket proxy), stop and tell the user which call failed. Do not build a substitute host environment instead, such as a virtualenv with a released Airflow version.
 - Place temporary scripts in `dev/` (mounted as `/opt/airflow/dev/` inside Breeze).
 
 ## Commands
@@ -47,6 +48,7 @@ Don't spell out **Directed Acyclic Graph** except for historical context.
 - **Run other suites of tests** `breeze testing <test_group>` (test groups: `airflow-ctl-tests`, `docker-compose-tests`, `task-sdk-tests`)
 - **Run scripts tests:** `uv run --project scripts pytest scripts/tests/ -xvs`
 - **Run Airflow CLI:** `breeze run airflow dags list`
+- **Run provider tests against a released Airflow version:** `breeze shell --use-airflow-version <VERSION> --mount-sources providers-and-tests`, then `pytest providers/<provider>/tests/...` inside the shell
 - **Type-check (non-providers):** run the prek hook — `prek run mypy-<project> --all-files` (e.g. `mypy-airflow-core`, `mypy-task-sdk`, `mypy-shared-logging`; each `shared/<dist>` workspace member has its own `mypy-shared-<dist>` hook). The hook uses a dedicated virtualenv and mypy cache under `.build/mypy-venvs/<hook>/` and `.build/mypy-caches/<hook>/`; mypy itself is installed from `uv.lock` via the `mypy` dependency group (`uv sync --group mypy`), so it never mutates your project `.venv`. The hook prefers `uv` from the project's main `.venv/bin/uv` (installed by `uv sync` — `uv` is part of the `dev` dependency group via the `all` extras) for a project-pinned uv version; it falls back to `uv` on `$PATH` with a warning if that binary is missing. Clear with `breeze down --cleanup-mypy-cache`.
 - **Type-check (providers):** `breeze run mypy path/to/code`
 - **Lint with ruff only:** `prek run ruff --from-ref <target_branch>`

contributing-docs/testing/unit_tests.rst

@@ -1165,6 +1165,13 @@ directly to the container.
    breeze container, and you can re-run the tests inside the ``breeze`` container without restarting it
    (which makes it faster to iterate).
 
+.. AGENT-SKILL-START
+   type: agents-md-commands
+   order: 25
+   lines:
+     - "- **Run provider tests against a released Airflow version:** `breeze shell --use-airflow-version <VERSION> --mount-sources providers-and-tests`, then `pytest providers/<provider>/tests/...` inside the shell"
+.. AGENT-SKILL-END
+
 .. note::
 
    Since providers are installed from sources rather than from packages, plugins from providers are not

dev/skill-evals/last-eval-hash.txt

@@ -1,3 +1,3 @@
 # Generated by dev/skill-evals/eval.py — do not edit or resolve conflicts by hand.
 # Run `prek run run-skill-eval --hook-stage manual --all-files` to regenerate.
-6a82b68ab3d4179b148c7a545206c4e37e7611f85e9b830d89186a8713c8b272
+32e8c893747492d95d98e47b9244b693f9b23756c9d7502cd6aae94225e4ef27