mirror of
https://github.com/apache/airflow.git
synced 2026-11-04 07:12:48 +00:00
Check that the fork's GitHub Actions can run in breeze ci audit (#76136)
* Check the fork's GitHub Actions settings in breeze ci audit breeze ci audit checked only that the fork's ci-amd.yml workflow is active. A workflow can be active while GitHub Actions is disabled for the whole fork: then a push starts no run, and the audit still said ok. Actions can also allow only the owner's actions or a selected list: ci-amd.yml uses actions from other repositories, so its runs fail before any job starts. For a contributor who needs fork CI, the audit now reads the fork's Actions settings and fails when Actions is disabled or limited, with the command that enables all actions. Only the fork's owner can read these settings, so the fork CI monitor cannot check them. * Report fork Actions settings whatever the CI workflow's state Check 5 used the Actions settings only when Tests (AMD) was active. A fork whose workflow was inactive, or whose workflow could not be read, was never told that Actions was disabled or limited. Enabling the workflow alone still left it unable to run, and the Actions problem showed up only on a later audit, if at all. Generated-by: Claude Opus 5 * Detect a fork paused by GitHub and its idle workflows in breeze ci audit GitHub can pause GitHub Actions in a fork to reduce wasted compute. The fork's settings then still read as enabled and ci-amd.yml as active, so check 5 passed while no push started a run. Check 5 now dispatches ci-amd.yml to a ref that cannot exist: GitHub answers "No ref found" when Actions can run and "Actions has been disabled" when it is paused. Git does not allow ".." in a ref name, so the dispatch never starts a run. A fork that enables Actions also gets every cron of the repository, and each one starts a run there even when its jobs are skipped. The new check 8 lists the workflows that only a schedule or another workflow's run starts, and --fix disables the ones still active in the fork. Fork CI runs on push and does not need them. --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
This commit is contained in:
1 parent
4c110e645d
commit
1d5644d9de
6 files changed
+443
-20
No files matched your search
@@ -155,10 +155,21 @@ Set up your checkout and fork once:
|
||||
gh repo sync <you>/airflow --branch main
|
||||
gh workflow enable ci-amd.yml -R <you>/airflow
|
||||
|
||||
GitHub Actions must also be enabled in your fork and allow all actions: ``Tests (AMD)`` uses
|
||||
actions from other repositories.
|
||||
|
||||
Keep the workflow enabled for as long as you have open pull requests. If it gets disabled, from
|
||||
your fork's Actions tab or by GitHub after 60 days without activity in your fork, the fork CI
|
||||
monitor comments on your pull requests with the commands to turn it back on.
|
||||
|
||||
GitHub can also pause GitHub Actions in your fork to reduce wasted compute, and it sends no
|
||||
email when it does. Your pushes then start no run, and the fork CI monitor cannot tell this from
|
||||
a commit you have not pushed yet. ``breeze ci audit`` reports it. The banner on your fork's
|
||||
Actions page turns GitHub Actions back on, and a push made while it was paused must be pushed
|
||||
again. To make the pause less likely,
|
||||
``breeze ci audit --fix`` disables the workflows in your fork that only run on a schedule or
|
||||
after another workflow. Fork CI does not need them.
|
||||
|
||||
3. Check everything at once. ``breeze ci audit`` prints one row per check and the command that fixes
|
||||
anything missing; ``--fix`` runs those commands after asking:
|
||||
|
||||
|
||||
@@ -168,8 +168,10 @@ Pull requests from external contributors can run their CI in the contributor's f
|
||||
``breeze ci audit`` checks, read-only, whether your checkout and fork are set up for that: ``gh``
|
||||
installed and authenticated, the ``upstream`` remote pointing at ``apache/airflow`` and ``origin`` at your
|
||||
fork, whether you are a committer, collaborator, member of an exempted organisation, pilot participant or
|
||||
external contributor, whether the ``Tests (AMD)`` workflow in your fork is enabled (when your pull requests
|
||||
use fork CI), how far your fork's ``main`` is behind, and the branch you are on.
|
||||
external contributor, whether the ``Tests (AMD)`` workflow in your fork is enabled and GitHub Actions lets it
|
||||
run, including whether GitHub has paused GitHub Actions in your fork (when your pull requests use fork CI),
|
||||
how far your fork's ``main`` is behind, the branch you are on, and which workflows in your fork only start
|
||||
runs that fork CI does not need.
|
||||
Each failing check prints the command that fixes it; ``--fix`` runs those commands after asking.
|
||||
|
||||
These are all available flags of ``audit`` command:
|
||||
|
||||
@@ -169,7 +169,8 @@ Actions gets all of their crons too. Their jobs carry a job-level
|
||||
`if: github.repository == 'apache/airflow'` guard, combined with
|
||||
`github.event_name != 'schedule'` when the workflow has other triggers that should still run
|
||||
in a fork. A new scheduled workflow needs the same guard. The guard skips the jobs but not
|
||||
the run itself, so the frequent crons still leave an empty run in every enabled fork.
|
||||
the run itself, so the frequent crons still leave an empty run in every enabled fork. Check 8 of
|
||||
`breeze ci audit` offers to disable these workflows in the fork.
|
||||
|
||||
| Rule | Decision | Reason |
|
||||
|---|---|---|
|
||||
@@ -1102,9 +1103,10 @@ prints one row: status, finding, and the command that fixes it.
|
||||
| 2 | Remote naming | a remote named `upstream` points at `apache/airflow` and `origin` does not | `git remote rename`/`add` per `contributing-docs/10_working_with_git.rst` |
|
||||
| 3 | Fork exists | `origin` is a fork of `apache/airflow` owned by the authenticated user | `gh repo fork apache/airflow --remote=false` then add `origin`, unless check 2's fix already adds or repoints it |
|
||||
| 4 | Who you are | classifies the user in the gate's order: pilot participant (`[pilot]`, which gates whatever the role), committer (public `apache` membership, `GET /orgs/apache/public_members/<login>`, since the gate cannot see private membership), collaborator (`.asf.yaml`), exempted (`[allowlist]`), external; an external user whose `apache` membership is private is told to make it public or ask for `[allowlist]`; when the membership call fails for any reason but a 404, the role is reported as not checked and checks 2, 3 and 5 stay informational | informational |
|
||||
| 5 | Fork workflows | external or pilot participant: `ci-amd.yml` state on the fork is `active`; committer/collaborator/exempted: always OK, whatever the state | `gh workflow enable ci-amd.yml -R <fork>` |
|
||||
| 5 | Fork workflows | external or pilot participant: `ci-amd.yml` state on the fork is `active`, GitHub Actions is enabled in the fork and allows all actions (`GET /repos/<fork>/actions/permissions`, readable by the fork's owner), and GitHub has not paused GitHub Actions in the fork (see below); committer/collaborator/exempted: always OK, whatever the state | `gh workflow enable ci-amd.yml -R <fork>`; `gh api -X PUT repos/<fork>/actions/permissions -F enabled=true -f allowed_actions=all`; for a paused fork, the banner on its Actions page |
|
||||
| 6 | Fork `main` freshness | fork `main` is not behind `upstream/main` by more than 200 commits (`MAX_FORK_MAIN_BEHIND`) | `gh repo sync <fork> --branch main` |
|
||||
| 7 | Current branch | not `main`, has an upstream tracking `origin` | informational |
|
||||
| 8 | Idle fork workflows | external or pilot participant: no workflow in `.github/workflows` of the checkout that only `schedule` or `workflow_run` starts (`workflow_dispatch` aside) is `active` in the fork | `gh workflow disable <file> -R <fork>` for each |
|
||||
|
||||
The role is determined before checks 2 and 3, though printed as check 4. For a committer,
|
||||
collaborator or exempted contributor, whose pull requests run project CI, checks 2 and 3 are
|
||||
@@ -1114,6 +1116,26 @@ is enabled or not. The audit never suggests disabling it: a committer trying for
|
||||
`use fork ci` label needs it enabled, and a workflow disabled while a pull request waits for its
|
||||
fork CI leaves that pull request stuck until the workflow is enabled again.
|
||||
|
||||
A workflow can be `active` while nothing in the fork runs. GitHub Actions can be disabled for the
|
||||
whole fork: then a push starts no run. Or it can allow only the owner's actions or a selected list:
|
||||
`ci-amd.yml` uses actions from other repositories, so its runs fail before any job starts. Only the
|
||||
fork's owner can read these settings, which is why check 5 reads them and the reconciler cannot.
|
||||
|
||||
GitHub can also pause GitHub Actions in a fork on its own, to reduce wasted compute. The fork's
|
||||
Actions page then shows a banner, and GitHub sends no email. The settings still read as enabled
|
||||
and the workflow as `active`, so the reads above do not see it. Check 5 dispatches `ci-amd.yml`
|
||||
to the ref `breeze-ci-audit..probe` instead. Git does not allow `..` in a ref name, so the ref
|
||||
never exists and the dispatch never starts a run. GitHub answers "No ref found" when Actions
|
||||
can run, and "Actions has been disabled" when it is paused. The settings API cannot undo the
|
||||
pause, since it already reads as enabled; the banner on the fork's Actions page does. A push made
|
||||
while the fork was paused starts no run, also after that, so push again.
|
||||
|
||||
Check 8 makes the pause less likely. Fork CI runs on push, so a workflow that only a schedule or
|
||||
another workflow's run starts has nothing to do in a fork, and every run it starts there is
|
||||
empty. The check reads the triggers from `.github/workflows` of the checkout and offers to disable
|
||||
those workflows where they are still `active` in the fork. Committers, collaborators and
|
||||
exempted contributors do not get check 8, and check 5 does not dispatch for them.
|
||||
|
||||
A failed API call is reported as "could not check" rather than as a finding: a failed fork
|
||||
lookup is not "not a fork of apache/airflow", and a failed workflow lookup is not "missing".
|
||||
A 404 is a finding: the fork does not exist, or the workflow is missing. An `upstream` remote
|
||||
|
||||
@@ -1808,15 +1808,18 @@ def audit(fix: bool):
|
||||
import yaml
|
||||
from rich.table import Table
|
||||
|
||||
from airflow_breeze.utils.ci_audit import CheckStatus, run_audit, run_tool
|
||||
from airflow_breeze.utils.ci_audit import CheckStatus, find_idle_fork_workflows, run_audit, run_tool
|
||||
from airflow_breeze.utils.fork_ci_gate import load_config
|
||||
|
||||
asf_yaml = yaml.safe_load((AIRFLOW_ROOT_PATH / ".asf.yaml").read_text())
|
||||
collaborators = frozenset((asf_yaml.get("github") or {}).get("collaborators") or [])
|
||||
config = load_config(AIRFLOW_ROOT_PATH / ".github" / "fork-ci-gate.toml")
|
||||
idle_workflows = find_idle_fork_workflows(AIRFLOW_ROOT_PATH / ".github" / "workflows")
|
||||
|
||||
def report():
|
||||
results = run_audit(run=run_tool, config=config, collaborators=collaborators)
|
||||
results = run_audit(
|
||||
run=run_tool, config=config, collaborators=collaborators, idle_workflows=idle_workflows
|
||||
)
|
||||
table = Table(title="Fork CI setup")
|
||||
for column in ("#", "Check", "Status", "Finding", "Fix"):
|
||||
table.add_column(column)
|
||||
|
||||
@@ -29,8 +29,10 @@ import subprocess
|
||||
from collections.abc import Callable, Sequence
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import yaml
|
||||
from packaging.version import Version
|
||||
|
||||
from airflow_breeze.global_constants import APACHE_AIRFLOW_GITHUB_REPOSITORY, MIN_GH_VERSION
|
||||
@@ -41,6 +43,8 @@ if TYPE_CHECKING:
|
||||
|
||||
UPSTREAM_URL = f"https://github.com/{APACHE_AIRFLOW_GITHUB_REPOSITORY}.git"
|
||||
CI_WORKFLOW_FILE = "ci-amd.yml"
|
||||
# `..` is not allowed in a git ref name, so a dispatch to this ref can never start a run.
|
||||
PROBE_REF = "breeze-ci-audit..probe"
|
||||
# Commits a fork's main may lag behind upstream main before a sync is suggested.
|
||||
MAX_FORK_MAIN_BEHIND = 200
|
||||
|
||||
@@ -138,7 +142,65 @@ def fork_ci_required(role: Role, config: GateConfig) -> bool | None:
|
||||
return False
|
||||
|
||||
|
||||
def run_audit(*, run: Runner, config: GateConfig, collaborators: frozenset[str]) -> list[CheckResult]:
|
||||
def get_actions_problem(permissions_json: str, fork: str) -> str | None:
|
||||
"""Why the fork's GitHub Actions settings keep `Tests (AMD)` from running, or None."""
|
||||
permissions = json.loads(permissions_json)
|
||||
if not permissions.get("enabled"):
|
||||
return f"GitHub Actions is disabled in {fork}, so `Tests (AMD)` never runs"
|
||||
if permissions.get("allowed_actions") != "all":
|
||||
return (
|
||||
f"{fork} limits the actions workflows may use (`{permissions.get('allowed_actions')}`); "
|
||||
"`Tests (AMD)` uses actions from other repositories, so it fails to start"
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def is_actions_paused_by_github(*, fork: str, run: Runner) -> bool | None:
|
||||
"""
|
||||
Whether GitHub itself has paused GitHub Actions in ``fork``; None when the answer is unclear.
|
||||
|
||||
The fork's settings still read as enabled then. A dispatch to a ref that does not exist tells
|
||||
the two apart: it fails with "Actions has been disabled" instead of "No ref found".
|
||||
"""
|
||||
probe = run(
|
||||
[
|
||||
"gh",
|
||||
"api",
|
||||
f"repos/{fork}/actions/workflows/{CI_WORKFLOW_FILE}/dispatches",
|
||||
"-X",
|
||||
"POST",
|
||||
"-f",
|
||||
f"ref={PROBE_REF}",
|
||||
]
|
||||
)
|
||||
output = f"{probe.stdout}{probe.stderr}"
|
||||
if "Actions has been disabled" in output:
|
||||
return True
|
||||
if "No ref found" in output:
|
||||
return False
|
||||
return None
|
||||
|
||||
|
||||
def find_idle_fork_workflows(workflows_dir: Path) -> frozenset[str]:
|
||||
"""
|
||||
Workflow files that only a schedule or another workflow's run starts.
|
||||
|
||||
Fork CI runs on push, so in a fork these workflows only start runs that fork CI does not need.
|
||||
"""
|
||||
idle = set()
|
||||
for path in workflows_dir.glob("*.y*ml"):
|
||||
workflow = yaml.safe_load(path.read_text())
|
||||
# YAML 1.1 reads an unquoted `on` key as True.
|
||||
triggers = workflow.get("on", workflow.get(True)) or []
|
||||
automatic = set([triggers] if isinstance(triggers, str) else triggers) - {"workflow_dispatch"}
|
||||
if automatic and automatic <= {"schedule", "workflow_run"}:
|
||||
idle.add(path.name)
|
||||
return frozenset(idle)
|
||||
|
||||
|
||||
def run_audit(
|
||||
*, run: Runner, config: GateConfig, collaborators: frozenset[str], idle_workflows: frozenset[str]
|
||||
) -> list[CheckResult]:
|
||||
results: list[CheckResult] = []
|
||||
|
||||
gh_version = run(["gh", "--version"])
|
||||
@@ -305,26 +367,54 @@ def run_audit(*, run: Runner, config: GateConfig, collaborators: frozenset[str])
|
||||
if fork_ok and fork is not None:
|
||||
workflow = run(["gh", "api", f"repos/{fork}/actions/workflows/{CI_WORKFLOW_FILE}", "--jq", ".state"])
|
||||
state = _out(workflow) if _ok(workflow) else "missing" if _is_not_found(workflow) else None
|
||||
# Only the fork's owner can read these settings, and the audit runs as the owner.
|
||||
actions = run(["gh", "api", f"repos/{fork}/actions/permissions"]) if required is True else None
|
||||
actions_problem = (
|
||||
get_actions_problem(_out(actions), fork) if actions is not None and _ok(actions) else None
|
||||
)
|
||||
# An enabled workflow still never runs while Actions is off or limited, so this fix goes first.
|
||||
allow_actions: tuple[str, ...] = (
|
||||
(f"gh api -X PUT repos/{fork}/actions/permissions -F enabled=true -f allowed_actions=all",)
|
||||
if actions_problem
|
||||
else ()
|
||||
)
|
||||
if state is None:
|
||||
not_checked = f"could not check `Tests (AMD)` in {fork}: {(workflow.stderr or '').strip() or 'gh api failed'}"
|
||||
results.append(
|
||||
CheckResult(
|
||||
5,
|
||||
"fork workflows",
|
||||
CheckStatus.INFO,
|
||||
f"could not check `Tests (AMD)` in {fork}: {(workflow.stderr or '').strip() or 'gh api failed'}",
|
||||
5, "fork workflows", CheckStatus.FAIL, f"{actions_problem}; {not_checked}", allow_actions
|
||||
)
|
||||
if actions_problem
|
||||
else CheckResult(5, "fork workflows", CheckStatus.INFO, not_checked)
|
||||
)
|
||||
elif required is True and state != "active":
|
||||
enable: tuple[str, ...] = (f"gh workflow enable {CI_WORKFLOW_FILE} -R {fork}",)
|
||||
if state == "missing":
|
||||
enable = (f"gh repo sync {fork} --branch main", *enable)
|
||||
finding = f"`Tests (AMD)` is {state} in {fork}; enable it"
|
||||
if actions_problem:
|
||||
finding += f"; also, {actions_problem}"
|
||||
results.append(
|
||||
CheckResult(5, "fork workflows", CheckStatus.FAIL, finding, (*allow_actions, *enable))
|
||||
)
|
||||
elif actions is not None and not _ok(actions):
|
||||
results.append(
|
||||
CheckResult(
|
||||
5,
|
||||
"fork workflows",
|
||||
CheckStatus.INFO,
|
||||
f"could not check the GitHub Actions settings of {fork}: "
|
||||
f"{(actions.stderr or '').strip() or 'gh api failed'}",
|
||||
)
|
||||
)
|
||||
elif actions_problem:
|
||||
results.append(
|
||||
CheckResult(
|
||||
5,
|
||||
"fork workflows",
|
||||
CheckStatus.FAIL,
|
||||
f"`Tests (AMD)` is {state} in {fork}; enable it",
|
||||
enable,
|
||||
actions_problem,
|
||||
allow_actions,
|
||||
)
|
||||
)
|
||||
elif required is None:
|
||||
@@ -342,9 +432,33 @@ def run_audit(*, run: Runner, config: GateConfig, collaborators: frozenset[str])
|
||||
)
|
||||
)
|
||||
else:
|
||||
results.append(
|
||||
CheckResult(5, "fork workflows", CheckStatus.OK, f"`Tests (AMD)` is {state} in {fork}")
|
||||
)
|
||||
paused = is_actions_paused_by_github(fork=fork, run=run) if required is True else False
|
||||
if paused:
|
||||
results.append(
|
||||
CheckResult(
|
||||
5,
|
||||
"fork workflows",
|
||||
CheckStatus.FAIL,
|
||||
f"GitHub has paused GitHub Actions in {fork}, so `Tests (AMD)` never runs "
|
||||
"although the settings allow it; turn it back on from the banner on the fork's "
|
||||
"Actions page",
|
||||
(f"echo Open https://github.com/{fork}/actions and enable the workflows",),
|
||||
)
|
||||
)
|
||||
elif paused is None:
|
||||
results.append(
|
||||
CheckResult(
|
||||
5,
|
||||
"fork workflows",
|
||||
CheckStatus.INFO,
|
||||
f"`Tests (AMD)` is {state} in {fork}, but could not check whether GitHub "
|
||||
"has paused GitHub Actions there",
|
||||
)
|
||||
)
|
||||
else:
|
||||
results.append(
|
||||
CheckResult(5, "fork workflows", CheckStatus.OK, f"`Tests (AMD)` is {state} in {fork}")
|
||||
)
|
||||
|
||||
compare = run(
|
||||
[
|
||||
@@ -375,6 +489,47 @@ def run_audit(*, run: Runner, config: GateConfig, collaborators: frozenset[str])
|
||||
CheckResult(6, "fork main", CheckStatus.OK, f"{behind} commits behind apache/airflow")
|
||||
)
|
||||
|
||||
idle_result = None
|
||||
if fork_ok and fork is not None and required is True:
|
||||
listed = run(
|
||||
[
|
||||
"gh",
|
||||
"api",
|
||||
f"repos/{fork}/actions/workflows",
|
||||
"--paginate",
|
||||
"--jq",
|
||||
'.workflows[] | select(.state == "active") | .path',
|
||||
]
|
||||
)
|
||||
if not _ok(listed):
|
||||
idle_result = CheckResult(
|
||||
8,
|
||||
"idle fork workflows",
|
||||
CheckStatus.INFO,
|
||||
f"could not list the workflows of {fork}: {(listed.stderr or '').strip() or 'gh api failed'}",
|
||||
)
|
||||
else:
|
||||
active = {path.rsplit("/", 1)[-1] for path in _out(listed).splitlines()}
|
||||
to_disable = sorted(active & idle_workflows)
|
||||
idle_result = (
|
||||
CheckResult(
|
||||
8,
|
||||
"idle fork workflows",
|
||||
CheckStatus.FAIL,
|
||||
f"{fork} has workflows that start only on a schedule or after another workflow, so "
|
||||
"every run they start there is wasted; GitHub pauses GitHub Actions in forks that "
|
||||
"waste runs",
|
||||
tuple(f"gh workflow disable {name} -R {fork}" for name in to_disable),
|
||||
)
|
||||
if to_disable
|
||||
else CheckResult(
|
||||
8,
|
||||
"idle fork workflows",
|
||||
CheckStatus.OK,
|
||||
f"no workflow in {fork} starts runs that fork CI does not need",
|
||||
)
|
||||
)
|
||||
|
||||
branch = _out(run(["git", "rev-parse", "--abbrev-ref", "HEAD"]))
|
||||
tracking = run(["git", "rev-parse", "--abbrev-ref", "--symbolic-full-name", "@{u}"])
|
||||
if branch in ("main", "HEAD", ""):
|
||||
@@ -388,4 +543,6 @@ def run_audit(*, run: Runner, config: GateConfig, collaborators: frozenset[str])
|
||||
else:
|
||||
finding = f"`{branch}` tracks {_out(tracking)}"
|
||||
results.append(CheckResult(7, "branch", CheckStatus.INFO, finding))
|
||||
if idle_result is not None:
|
||||
results.append(idle_result)
|
||||
return results
|
||||
@@ -25,7 +25,14 @@ import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
from airflow_breeze.commands.ci_commands import audit
|
||||
from airflow_breeze.utils.ci_audit import CheckResult, CheckStatus, Role, run_audit, run_tool
|
||||
from airflow_breeze.utils.ci_audit import (
|
||||
CheckResult,
|
||||
CheckStatus,
|
||||
Role,
|
||||
find_idle_fork_workflows,
|
||||
run_audit,
|
||||
run_tool,
|
||||
)
|
||||
from airflow_breeze.utils.confirm import Answer
|
||||
from airflow_breeze.utils.fork_ci_gate import GateConfig
|
||||
|
||||
@@ -40,6 +47,12 @@ Output = tuple[int, str] | tuple[int, str, str]
|
||||
|
||||
NOT_A_MEMBER: Output = (1, "", "gh: Not Found (HTTP 404)")
|
||||
|
||||
ACTIONS_ALLOWED: Output = (0, json.dumps({"enabled": True, "allowed_actions": "all"}))
|
||||
|
||||
NO_REF: Output = (1, "", "gh: No ref found for: breeze-ci-audit..probe (HTTP 422)")
|
||||
|
||||
PAUSED: Output = (1, "", "gh: Actions has been disabled for this repository. (HTTP 422)")
|
||||
|
||||
|
||||
def _machine(**overrides: Output | None) -> dict[str, Output | None]:
|
||||
"""Outputs of every command the audit runs, keyed by a short name."""
|
||||
@@ -51,6 +64,9 @@ def _machine(**overrides: Output | None) -> dict[str, Output | None]:
|
||||
"membership": NOT_A_MEMBER,
|
||||
"private_membership": (1, ""),
|
||||
"workflow": (0, "active"),
|
||||
"actions": ACTIONS_ALLOWED,
|
||||
"dispatch": NO_REF,
|
||||
"fork_workflows": (0, ".github/workflows/ci-amd.yml\n"),
|
||||
"behind": (0, "12"),
|
||||
"branch": (0, "my-change"),
|
||||
"tracking": (0, "origin/my-change"),
|
||||
@@ -74,8 +90,14 @@ def _runner(machine):
|
||||
command = tuple(command)
|
||||
key = keys.get(command)
|
||||
if key is None and command[:2] == ("gh", "api"):
|
||||
if "/actions/workflows/" in command[2]:
|
||||
if command[2].endswith("/dispatches"):
|
||||
key = "dispatch"
|
||||
elif command[2].endswith("/actions/workflows"):
|
||||
key = "fork_workflows"
|
||||
elif "/actions/workflows/" in command[2]:
|
||||
key = "workflow"
|
||||
elif command[2].endswith("/actions/permissions"):
|
||||
key = "actions"
|
||||
elif "/compare/" in command[2]:
|
||||
key = "behind"
|
||||
elif command[2].startswith("repos/"):
|
||||
@@ -91,10 +113,12 @@ def _runner(machine):
|
||||
ENABLED = GateConfig(enabled=True)
|
||||
|
||||
|
||||
def _audit(machine, config=ENABLED, collaborators=frozenset()):
|
||||
def _audit(machine, config=ENABLED, collaborators=frozenset(), idle_workflows=frozenset({"stale.yml"})):
|
||||
return {
|
||||
result.number: result
|
||||
for result in run_audit(run=_runner(machine), config=config, collaborators=collaborators)
|
||||
for result in run_audit(
|
||||
run=_runner(machine), config=config, collaborators=collaborators, idle_workflows=idle_workflows
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
@@ -109,6 +133,7 @@ def test_healthy_external_setup():
|
||||
CheckStatus.OK,
|
||||
CheckStatus.OK,
|
||||
CheckStatus.INFO,
|
||||
CheckStatus.OK,
|
||||
]
|
||||
assert results[4].finding == f"you are a {Role.EXTERNAL.value}"
|
||||
|
||||
@@ -324,6 +349,209 @@ def test_fork_workflows_by_role(config, membership, collaborators, workflow, rol
|
||||
assert results[5].fix == fix
|
||||
|
||||
|
||||
ENABLE_ALL_ACTIONS = (
|
||||
"gh api -X PUT repos/someone/airflow/actions/permissions -F enabled=true -f allowed_actions=all",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("membership", "actions", "status", "finding", "fix"),
|
||||
[
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(0, json.dumps({"enabled": False})),
|
||||
CheckStatus.FAIL,
|
||||
"GitHub Actions is disabled in someone/airflow, so `Tests (AMD)` never runs",
|
||||
ENABLE_ALL_ACTIONS,
|
||||
id="actions-disabled",
|
||||
),
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(0, json.dumps({"enabled": True, "allowed_actions": "local_only"})),
|
||||
CheckStatus.FAIL,
|
||||
"someone/airflow limits the actions workflows may use (`local_only`); "
|
||||
"`Tests (AMD)` uses actions from other repositories, so it fails to start",
|
||||
ENABLE_ALL_ACTIONS,
|
||||
id="local-actions-only",
|
||||
),
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(0, json.dumps({"enabled": True, "allowed_actions": "selected"})),
|
||||
CheckStatus.FAIL,
|
||||
"someone/airflow limits the actions workflows may use (`selected`); "
|
||||
"`Tests (AMD)` uses actions from other repositories, so it fails to start",
|
||||
ENABLE_ALL_ACTIONS,
|
||||
id="selected-actions-only",
|
||||
),
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(1, "", "HTTP 502: Bad Gateway"),
|
||||
CheckStatus.INFO,
|
||||
"could not check the GitHub Actions settings of someone/airflow: HTTP 502: Bad Gateway",
|
||||
(),
|
||||
id="settings-not-read",
|
||||
),
|
||||
pytest.param(
|
||||
(0, ""),
|
||||
(0, json.dumps({"enabled": False})),
|
||||
CheckStatus.OK,
|
||||
"`Tests (AMD)` is active in someone/airflow",
|
||||
(),
|
||||
id="committer-actions-disabled",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_fork_actions_settings(membership, actions, status, finding, fix):
|
||||
results = _audit(_machine(membership=membership, actions=actions))
|
||||
|
||||
assert (results[5].status, results[5].finding, results[5].fix) == (status, finding, fix)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("workflow", "finding", "fix"),
|
||||
[
|
||||
pytest.param(
|
||||
(0, "disabled_manually"),
|
||||
"`Tests (AMD)` is disabled_manually in someone/airflow; enable it; also, "
|
||||
"GitHub Actions is disabled in someone/airflow, so `Tests (AMD)` never runs",
|
||||
(*ENABLE_ALL_ACTIONS, "gh workflow enable ci-amd.yml -R someone/airflow"),
|
||||
id="workflow-disabled",
|
||||
),
|
||||
pytest.param(
|
||||
(1, "", "gh: Not Found (HTTP 404)"),
|
||||
"`Tests (AMD)` is missing in someone/airflow; enable it; also, "
|
||||
"GitHub Actions is disabled in someone/airflow, so `Tests (AMD)` never runs",
|
||||
(
|
||||
*ENABLE_ALL_ACTIONS,
|
||||
"gh repo sync someone/airflow --branch main",
|
||||
"gh workflow enable ci-amd.yml -R someone/airflow",
|
||||
),
|
||||
id="workflow-missing",
|
||||
),
|
||||
pytest.param(
|
||||
(1, "", "HTTP 502: Bad Gateway"),
|
||||
"GitHub Actions is disabled in someone/airflow, so `Tests (AMD)` never runs; "
|
||||
"could not check `Tests (AMD)` in someone/airflow: HTTP 502: Bad Gateway",
|
||||
ENABLE_ALL_ACTIONS,
|
||||
id="workflow-not-read",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_actions_disabled_reported_whatever_the_workflow_state(workflow, finding, fix):
|
||||
results = _audit(_machine(workflow=workflow, actions=(0, json.dumps({"enabled": False}))))
|
||||
|
||||
assert (results[5].status, results[5].finding, results[5].fix) == (CheckStatus.FAIL, finding, fix)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("membership", "dispatch", "status", "finding", "fix"),
|
||||
[
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
PAUSED,
|
||||
CheckStatus.FAIL,
|
||||
"GitHub has paused GitHub Actions in someone/airflow, so `Tests (AMD)` never runs although "
|
||||
"the settings allow it; turn it back on from the banner on the fork's Actions page",
|
||||
("echo Open https://github.com/someone/airflow/actions and enable the workflows",),
|
||||
id="paused",
|
||||
),
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(1, "", "HTTP 502: Bad Gateway"),
|
||||
CheckStatus.INFO,
|
||||
"`Tests (AMD)` is active in someone/airflow, but could not check whether GitHub has paused "
|
||||
"GitHub Actions there",
|
||||
(),
|
||||
id="probe-failed",
|
||||
),
|
||||
pytest.param(
|
||||
(0, ""),
|
||||
PAUSED,
|
||||
CheckStatus.OK,
|
||||
"`Tests (AMD)` is active in someone/airflow",
|
||||
(),
|
||||
id="committer-not-probed",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_actions_paused_by_github(membership, dispatch, status, finding, fix):
|
||||
results = _audit(_machine(membership=membership, dispatch=dispatch))
|
||||
|
||||
assert (results[5].status, results[5].finding, results[5].fix) == (status, finding, fix)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("membership", "fork_workflows", "expected"),
|
||||
[
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(0, ".github/workflows/ci-amd.yml\n.github/workflows/stale.yml\n.github/workflows/ci-arm.yml\n"),
|
||||
(
|
||||
CheckStatus.FAIL,
|
||||
"someone/airflow has workflows that start only on a schedule or after another workflow, so "
|
||||
"every run they start there is wasted; GitHub pauses GitHub Actions in forks that waste runs",
|
||||
(
|
||||
"gh workflow disable ci-arm.yml -R someone/airflow",
|
||||
"gh workflow disable stale.yml -R someone/airflow",
|
||||
),
|
||||
),
|
||||
id="idle-workflows-active",
|
||||
),
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(0, ".github/workflows/ci-amd.yml\n"),
|
||||
(CheckStatus.OK, "no workflow in someone/airflow starts runs that fork CI does not need", ()),
|
||||
id="idle-workflows-disabled",
|
||||
),
|
||||
pytest.param(
|
||||
NOT_A_MEMBER,
|
||||
(1, "", "HTTP 502: Bad Gateway"),
|
||||
(
|
||||
CheckStatus.INFO,
|
||||
"could not list the workflows of someone/airflow: HTTP 502: Bad Gateway",
|
||||
(),
|
||||
),
|
||||
id="listing-failed",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_idle_fork_workflows(membership, fork_workflows, expected):
|
||||
results = _audit(
|
||||
_machine(membership=membership, fork_workflows=fork_workflows),
|
||||
idle_workflows=frozenset({"stale.yml", "ci-arm.yml"}),
|
||||
)
|
||||
|
||||
assert (results[8].status, results[8].finding, results[8].fix) == expected
|
||||
|
||||
|
||||
def test_idle_fork_workflows_not_checked_for_committers():
|
||||
results = _audit(_machine(membership=(0, ""), fork_workflows=(0, ".github/workflows/stale.yml\n")))
|
||||
|
||||
assert 8 not in results
|
||||
|
||||
|
||||
def test_find_idle_fork_workflows(tmp_path):
|
||||
workflows = {
|
||||
"cron.yml": "on:\n schedule:\n - cron: '0 0 * * *'\n",
|
||||
"cron-or-manual.yml": "on:\n schedule:\n - cron: '0 0 * * *'\n workflow_dispatch:\n",
|
||||
"after-ci.yml": "on:\n workflow_run:\n workflows: ['Tests (AMD)']\n",
|
||||
"quoted-on.yaml": '"on": [schedule]\n',
|
||||
"cron-and-push.yml": "on:\n schedule:\n - cron: '0 0 * * *'\n push:\n",
|
||||
"reusable.yml": "on:\n workflow_call:\n schedule:\n - cron: '0 0 * * *'\n",
|
||||
"manual-only.yml": "on: workflow_dispatch\n",
|
||||
"push.yml": "on: push\n",
|
||||
}
|
||||
for name, text in workflows.items():
|
||||
(tmp_path / name).write_text(f"{text}jobs: {{}}\n")
|
||||
|
||||
assert find_idle_fork_workflows(tmp_path) == {
|
||||
"cron.yml",
|
||||
"cron-or-manual.yml",
|
||||
"after-ci.yml",
|
||||
"quoted-on.yaml",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("behind", "status"),
|
||||
[
|
||||
|
||||
Reference in new issue
Block a user