* [main] CI: Upgrade important CI environment
* Ignore pydantic-ai's generic OpenAI-compatible provider in vendor drift checks
pydantic-ai-slim 2.63.0 adds a provider for any OpenAI-compatible endpoint. It needs a service name at construction and infer_model has no prefix for it, so a common.ai connection can never reach it and it is not a vendor the services page should list.
Generated-by: Claude Opus 5
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
ConfigMap, Secret and projected volumes keep their files in a hidden
"..<timestamp>" directory linked as "..data", and expose each entry as
a symlink through it. Walking both reached that directory twice, so the
recursive-loop check failed discovery for any such mounted folder.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
* [main] CI: Upgrade important CI environment
* Bump pypdf to 6.21.4 to fix high-severity advisories
The CI environment upgrade resolved pypdf 6.21.3, which the dependency
review gate rejects (GHSA-mfr5-77xh-9j7h, GHSA-cggf-277g-3q54,
GHSA-5849-834g-hvw5). The fixed 6.21.4 is still inside the 4-day
exclude-newer window, so add a temporary per-package cutoff for it.
Generated-by: Claude Opus 5
* Support pydantic-ai 2.60+ and opensandbox 1.2 in common.ai provider
pydantic-ai 2.60 records a durable streamed model request as the bare
response rather than a response-plus-events wrapper, so replaying or
tracking a streamed request in AirflowDurability failed to decode it.
The same release answers a tool retry with the call's own ToolReturnPart
(outcome "retried") instead of RetryPromptPart, which broke tests that
told retries from results by part type.
opensandbox 1.2 allows an egress rule with ports and no target. When the
enforced policy did not match the requested one, sorting the allowed
targets for the error message raised TypeError on that None, hiding the
real reason the sandbox was refused.
Both work with the older releases too, which the provider still supports.
Generated-by: Claude Opus 5
* Fix provider tests and type checks after the dependency upgrade
The tool-part predicates in the common.ai tests returned a plain bool, so
mypy could not narrow the message part they matched and the providers
type check failed.
On Python 3.14 the upgrade moves snowflake-connector-python to 5.1 and
snowflake-sqlalchemy to 2.1. connector 5.1's connect() no longer goes
through SnowflakeConnection, so patching that class stopped intercepting
the connection and the test tried a real login. snowflake-sqlalchemy
2.0.0 started percent-encoding "?" in the URL password, which the URI
test pinned to the older, unencoded form. Older Python versions keep the
earlier releases, so both tests have to accept either.
Generated-by: Claude Opus 5
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
* Stop parallel prek hooks racing on __pycache__ cleanup
prek runs hooks in parallel and every breeze invocation removes all
__pycache__ directories in the checkout on start-up. Any other hook
process importing from the source tree (breeze itself, prek hook
scripts, local mypy runs) can write a fresh .pyc between rmtree emptying
a directory and removing it, which fails the breeze hook with
"OSError: [Errno 39] Directory not empty" - a flaky Static checks
failure in CI.
Redirecting bytecode of every hook into a hidden, gitignored .build/
directory keeps the source tree free of new .pyc files while hooks run,
and a new check keeps future hooks from reintroducing the race. The
cleanup additionally tolerates ENOTEMPTY for writers prek does not
control, such as an IDE.
Generated-by: Claude Opus 5
* Set PYTHONPYCACHEPREFIX on the hooks added since the branch was cut
Eight hooks that landed on main after this branch was created did not redirect their bytecode, so the new check-prek-hooks-pycache-prefix hook failed after the rebase.
Generated-by: Claude Opus 5
* Update scripts/ci/prek/check_prek_hooks_pycache_prefix.py
Co-authored-by: Wei Lee <hello@wei-lee.me>
---------
Co-authored-by: Wei Lee <hello@wei-lee.me>
git-sync publishes each revision by repointing the Dags folder symlink
to a new checkout. When that happens during file discovery, child
directories resolve into the new checkout and no longer find the ignore
rules inherited from their parent, so files under ignored directories
are picked up for that scan.
* Require Python 3.11 in all distributions
Every pyproject.toml that declares requires-python now says >= 3.11 and
drops the Python 3.10 trove classifier, the tomli build requirement that
only existed for 3.10 and the per-version dependency markers for 3.10.
Lock files are regenerated.
With 3.10 gone, the tomli fallbacks in scripts, dev tooling and tests go
too, along with the tomli additional_dependencies of the prek hooks and
the blacken-docs py310 target. The tomllib imports that only sat inside
functions because of the fallback move to the top of their modules. The
four dev/registry and scripts/ci/prek modules left out of the earlier
UP017 pass also take their timezone.utc to datetime.UTC fixes here. The
minimum host Python checked by check-min-python-version, the default
Python of run_generate_constraints.py and the provider cut-off date
parsing (fromisoformat, which reads a trailing Z from 3.11) follow the
new baseline.
ruff infers py311 from the new requires-python, which enables the
PEP 646 star-index fix (UP044) applied here and makes two more rules
reportable: UP042 (str, Enum to StrEnum) alters what str() and
format() return for serialized enum values, and ASYNC109 means
dropping the public timeout parameter from async APIs. Both need
per-site judgement, so they go on the ignore list with that rationale.
The datamodel-codegen target moves to 3.11 too, with use-specialized-enum
turned off for the same reason: otherwise the generated Task SDK and
airflowctl enums would switch from (str, Enum) to StrEnum.
* Drop the Python 3.10 provider metadata shim
With provider distributions now requiring Python 3.11, breeze can derive
requires-python and classifiers from the default and allowed versions
again. The patch-exclusion test and the matching example in
MANAGING_PROVIDERS_LIFECYCLE.rst move from 3.10.0 to 3.11.0, the oldest
patch release CI still covers.
* Keep Breeze's locked dependency versions when dropping Python 3.10
Relocking Breeze for the Python 3.11 baseline ran while recently bumped versions were still inside the 4-day exclude-newer window, so it downgraded pyjwt (reverting #73969), prek, boto3/botocore and filelock. Relocking from the previous lock keeps them and only drops the packages Python 3.11 no longer needs.
Generated-by: Claude Opus 5
---------
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
* [main] CI: Upgrade important CI environment
* Cap azure-datalake-store below 1 in the Microsoft Azure provider
azure-datalake-store 1.x removed the lib.auth API that AzureDataLakeHook
uses, so the floor bump to 1.0.1 in the CI upgrade broke the hook and
every TestAzureDataLakeHook test. Supporting 1.x is tracked in #73795.
Generated-by: Claude Opus 5
* Regenerate provider requirement tables after the floor bumps
The floor bumps changed provider pyproject.toml files without regenerating
the README.rst and docs/index.rst requirement tables derived from them,
so update-providers-build-files and sync-provider-readme failed static checks.
Generated-by: Claude Opus 5
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
This reverts commit ecfa20322b.
Main is broken, so reverting this for now.
Errors:
```
azure-ai-projects 2.7.0 has requirement openai>=3.0.0, but you have openai 2.54.0.
```
* Identity-only DAG discovery and byte-importing
Discovery now filters purely by a source's identity rather than its
contents. find_file_dag_definitions and list_dag_definitions match on
extension and .airflowignore, skip __pycache__, and prefer a .py over a
side-by-side .pyc, without reading any file or taking a safe_mode flag.
The might_contain_dag heuristic instead runs a single time, in
import_definition; plain files were previously scanned both at discovery
and again at import, so this drops a redundant scan while still reporting
non-DAG sources through DagImportResult.skipped_definitions.
ZipImporter now processes one member at a time instead of a whole
archive. It enumerates individual members during discovery, applying the
same __pycache__ and .py/.pyc rules and the ZipSlip defence there, and
imports a single member per call, rather than listing whole .zip files
and walking their members inside import_definition.
Modules are loaded from a definition's bytes. Source and bytecode are
read through the definition with custom loaders in place of the
file-backed SourceFileLoader and SourcelessFileLoader, so archive members
are no longer extracted to a temporary file to be scanned or imported and
.py and .pyc members load in place. Accordingly, might_contain_dag now
takes a definition rather than a path, and PythonDagImporter is
parameterized over the file and zip-member definition types it handles.
Tests cover identity-only discovery, per-member zip import, import-time
skipping for both files and members, .py/.pyc dedup, .pyc import for a
file and a member, and corrupt-bytecode handling.
* Rework Task SDK DAG importers around a file-source abstraction
Discovery now filters only by identity (extension, .airflowignore,
__pycache__, .py-over-.pyc); DAG detection runs once at import for both
files and zip members, surfaced through skipped_definitions. ZipImporter
processes one member at a time, and modules load from a definition's
bytes instead of an extracted file.
File sources are modelled as an abstract FileDagDefinition with
FilesystemDagDefinition and ZipMemberDagDefinition backings; importers
bind to the interface, so PythonDagImporter handles archive members
without referencing zip. The shared might_contain_dag also accepts a
definition, reading its bytes in place and falling back to as_file() only
for a custom might_contain_dag_callable; existing path/zip callers are
unchanged.
* Rename private function
* Dedup .py/.pyc over discovered files
find_file_dag_definitions preferred a source over a side-by-side .pyc by
checking path.with_suffix(".py").exists(), a raw filesystem stat.
This dropped a .pyc whose .py sibling was on disk but never discovered,
excluded by .airflowignore, or not in the importer's supported extensions,
leaving nothing to import.
Decide the preference over the candidates the walk actually yields instead,
keyed by (directory, stem) so the pairing also matches case-insensitively on
the extension. Candidates keep the walk's order.
* Reject .pyc payloads that aren't code objects
_DefinitionBytecodeLoader.get_code passed whatever marshal.loads returned
straight to exec(), so a .pyc with a valid magic header but a payload that
unmarshals to a non-code object (e.g. a marshalled string) was run as source
instead of rejected. Validate isinstance(code, types.CodeType) and raise
ImportError otherwise, keeping the corrupt-.pyc handling meaningful past the
magic-number check.
* Reject sibling-resource reads from source loader
_DefinitionSourceLoader.get_data ignored its path argument and always returned
the definition's own bytes, so a pkgutil.get_data or __loader__.get_data call
for a sibling resource silently got the DAG source instead.
Raise FileNotFoundError when the requested path isn't the module's own
get_filename() value; the normal import path, which only asks for that value,
is unchanged.
* Report unreadable archives as import errors
list_dag_definitions is widened to yield DagDefinition | DagImportError so a
ZipImporter can be reported for the caller to forward into DagImportResult.
* Have definitions control the import environment
A zip member is imported correctly only with its archive on sys.path, but that
setup lived in ZipImporter.import_definition. A member routed through the
registry by its .py suffix went to PythonDagImporter, which imported it without
the archive on sys.path and failed cross-member imports.
Move the setup onto the definition: FileDagDefinition.import_context is no-op
by default; ZipMemberDagDefinition overrides it to put the archive on sys.path,
and PythonDagImporter wraps the load in it.
* Use os.sep form for zip member filelocs
ZIP_REGEX in airflow-core splits on os.sep, so we need to match. Join with
Path.joinpath instead (i.e. archive.zip/member.py) instead.
* Move content sniffing back to discovery
Identity-only discovery surfaced every extension match as a definition, so a
bundle of 10 DAGs beside a 40-module helper package became 50 definitions, and,
once wired, 50 parse processes. Content detection at import shrank each
process's work but not their count.
Move the lightweight might_contain_dag sniff back into discovery:
list_dag_definitions applies it (gated by a safe_mode parameter) so a source
that clearly holds no DAG never becomes a definition. might_contain_dag is a
method on AbstractDagImporter that returns True by default, overridden by
PythonDagImporter with the real heuristic; ZipImporter calls it on each
member's internal importer, so importers with no cheap check (non-Python ones)
keep every member. The import path no longer re-sniffs.
safe_mode moves from import_definition (where it was a redundant second check)
to list_dag_definitions, where the filter decision actually lives; it is
dropped from import_definition and the ABC.
* Harden Dag discovery for zip members and sourceless packages
Review of the identity-only discovery rewrite surfaced four cases where
the new byte-based paths behave differently from the file-backed ones
they replace.
The shipped configuration names the built-in heuristic in
might_contain_dag_callable, so the callable is never absent and every
scanned archive member was still written to a temporary file, which is
the cost this rewrite set out to remove. A member that failed to read
ended discovery for its whole archive, losing later valid Dags that the
file-backed path reported individually. The source-over-bytecode
preference was decided before importer lookup, so an unsupported source
could hide the bytecode beside it, and a member whose extension differed
only in case was never paired at all. A bytes-backed __init__.pyc was
treated as an ordinary module, so a sourceless package's relative
imports failed where the file-backed loader resolved them.
---------
Co-authored-by: LIU ZHE YOU <zhu424.dev@gmail.com>
* feat: Restructure the AbstractDagImporter/DagImporterRegistry and move the /importers base classes into the task SDK
* fix: run the checks
* Decouple Dag importers from Dag bundles in Task SDK
Pass bundle name and path as primitive parameters to importer methods instead of
requiring a BaseDagBundle instance. This effort saves us from passing whole bundle object into the DagBag for the import_definition() method.
* Fix core Airflow import violations in shared libraries and Task SDK Dag importers
* Harden ZipImporter against path traversal. Remove core dependencies from Task SDK DAG importers and standardize exceptions. Clean up test boilerplate and redundant checks.
* Pass Dag bundles directly to Task SDK Dag importers
* Reimplement Zip import def to extract on-demand
* Simplify ZipImporter's get_source_code
The whole-zip-file concat logic is removed since we are not supposed to treat
the entire zip file as one dag source. Airflow's mental model is to treat a zip
more like a directory, and only one member in the zip is the source file of a
given dag. This tradition should be kept. Viewing an entire zip should instead
be handled as a feature on the dag bundle level.
* Improve Dag importer resilience, discovery filtering, and bytecode support
* Pass conf explicitly to might_contain_dag instead of frame introspection
Resolving which config module to read (airflow.configuration vs
airflow.sdk.configuration) by walking the call stack only worked by
coincidence, since every current caller's immediate frame already sits
in an airflow-prefixed module. It silently breaks for anything else in
the chain (tests, wrapped calls) and re-walks the stack on every file
scanned in safe mode. Every caller already has the right conf object
in scope, so require it explicitly instead.
---------
Co-authored-by: Tzu-ping Chung <tp@astronomer.io>
Co-authored-by: ZHE YOU LIU <zhu424.dev@gmail.com>
When a deadline callback fires, it now fetches DagRun context from the
Execution API and passes it to the callback as context["deadline"] and
context["dag_run"]. This enables Jinja template rendering ({{ dag_run }},
{{ deadline.deadline_time }}) and enriched notifications.
Key changes:
- Add GetDagRun comms message and handler in callback supervisor
- Wire dag_id/run_id/deadline_id/deadline_time through the workload path
- Build context dict from DagRun response via build_context_from_dag_run
- Fail callback (for retry) when context fetch fails instead of running
degraded
- Restrict token:workload to read-only Execution API routes (GET only)
- Add hash verification in _ensure_bundle_module_registered to prevent
wrong-bundle loading when multiple bundles share a filename
- Run bundle initialization off triggerer event loop (asyncio.to_thread)
- Extract _load_mangled_module to airflow._shared.module_loading as a
shared helper for both triggerer and executor paths
* Isolate mypy prek hooks, cover all non-provider dirs, and clean up type errors
Each non-provider mypy prek hook now builds and caches its own virtualenv
at .build/mypy-venvs/<hook>/ and its own mypy cache at
.build/mypy-caches/<hook>/. UV_PROJECT_ENVIRONMENT redirects uv away from
the project's .venv, so running the hook never mutates a contributor's
regular development environment while still matching CI's frozen
dependency set. Mypy runs with --follow-imports=silent so each hook only
reports errors for files it owns; transitive code is covered by its own
hook and different venvs no longer produce divergent results on shared
code.
Adds mypy hooks for the non-provider directories that were previously
uncovered: airflow-ctl-tests, helm-tests, airflow-e2e-tests,
task-sdk-integration-tests, docker-tests, kubernetes-tests, and shared.
The mypy-shared hook iterates every shared/<dist> workspace distribution
and builds a separate venv + cache per distribution so each shared
library is type-checked against its own dependency set.
breeze down --cleanup-mypy-cache additionally removes
.build/mypy-venvs/ and .build/mypy-caches/ so all per-hook state is
wiped alongside the existing .mypy_cache and mypy-cache-volume.
Also fixes pre-existing type errors surfaced by the newly added and
cleaned-up checks: platform-specific ignores for Linux-only
os.posix_fadvise in the shared logging helper, narrower types and
type: ignore where appropriate in shared configuration/observability/
timezones/secrets_backend/secrets_masker, Liskov override markers on
the AirflowConfigParser subclass methods, and small correctness fixes
in dev/breeze and the docker-tests / kubernetes-tests helpers so the
full non-provider mypy suite runs clean on macOS and in CI.
* Move mypy prek hooks to their respective distribution configs
The new mypy hooks for airflow-ctl-tests, helm-tests, airflow-e2e-tests,
task-sdk-integration-tests, docker-tests, and kubernetes-tests now live
in each distribution's own .pre-commit-config.yaml, matching the pattern
already used by airflow-core, task-sdk, and airflow-ctl. New .pre-commit-
config.yaml files are added to distributions that didn't have one. prek
auto-discovers nested configs, so the hooks remain part of the default
check set.
mypy-dev (covers dev + scripts), mypy-devel-common, and mypy-shared stay
at the repo root: dev/scripts/devel-common don't have their own configs,
and mypy-shared iterates every shared/<dist> distribution so has no
single home.
* Split mypy-dev and mypy-scripts, each with its own pyproject.toml config
Previously the mypy-dev prek hook ran mypy against dev/ and scripts/ in
a single invocation under the dev project's virtualenv. The two now get
independent hooks — mypy-dev in dev/.pre-commit-config.yaml and
mypy-scripts in scripts/.pre-commit-config.yaml — so each can evolve its
own dependency set and check its own folder.
Copy the full [tool.mypy] section from the root pyproject.toml into both
dev/pyproject.toml and scripts/pyproject.toml so each sub-project owns its
mypy configuration. Paths inside mypy_path are rewritten from
$MYPY_CONFIG_FILE_DIR/ to $MYPY_CONFIG_FILE_DIR/../ so they still resolve
to the repo-root siblings from the sub-project location. The decorator/
outputs plugins are scoped to dev only (scripts does not author DAG code).
mypy_local_folder.py now passes --config-file <project>/pyproject.toml
when the folder maps to one of these sub-project configs, so mypy uses
the sub-project's configuration rather than the root one.
* Teach selective-checks about the new non-provider mypy hooks
Add FileGroupForCi entries and regex patterns for helm-tests,
airflow-e2e-tests, docker-tests, kubernetes-tests, scripts, and shared
Python files, then wire them into skip_prek_hooks so the corresponding
mypy-* prek hook is only kept when its folder changed:
- mypy-scripts (split off from the old combined mypy-dev)
- mypy-airflow-ctl-tests, mypy-helm-tests, mypy-airflow-e2e-tests,
mypy-task-sdk-integration-tests, mypy-docker-tests, mypy-kubernetes-tests
- mypy-shared
Update test_selective_checks.py skip-list constants and per-case inline
skip lists to include the new hooks. Targeted test cases for files under
the new-hook directories override skip-prek-hooks to leave the matching
hook out of the skip set, confirming it will run when its folder changes.
* Trim dev/scripts pyproject mypy_path to just relevant distributions
Drop the 200+ provider path entries that were blindly copied from the root
pyproject.toml. dev and scripts only import from other non-provider
workspace members, so listing every provider src/tests directory under
mypy_path just adds noise. The remaining non-provider entries cover
everything dev or scripts plausibly import from.
* Install mypy into per-hook venvs from uv.lock via a `mypy` dep group
Each non-provider distribution with a mypy prek hook now declares a
`mypy` dependency group in its pyproject.toml resolving to
`apache-airflow-devel-common[mypy]`. mypy_local_folder.py syncs each
dedicated virtualenv with `uv sync --frozen --project <X> --group mypy`
and runs mypy with `uv run --frozen --project <X> --group mypy` — so
mypy and its type stubs come from the workspace uv.lock, not from an
ephemeral `--with` overlay whose resolution is independent of the main
lockfile. uv.lock is refreshed to include the new group.
Covers airflow-core, task-sdk, airflow-ctl, devel-common, dev, scripts,
airflow-ctl-tests, helm-tests, airflow-e2e-tests, task-sdk-integration-
tests, docker-tests, kubernetes-tests, and every shared/<dist>
workspace member.
* Drop mypy_path from dev/scripts pyprojects — venv site-packages is enough
After the switch to installing mypy (and every transitive workspace
dependency) directly into each hook's virtualenv via the `mypy` dep
group, workspace packages like airflow, airflow.sdk, airflowctl,
airflow_breeze, tests_common are all available via the venv's
site-packages. mypy resolves them without needing mypy_path entries,
so drop the copied list and leave a short comment explaining why.
* Split mypy-shared into per-distribution hooks and enforce the pattern
Each shared/<dist> workspace member now owns a mypy-shared-<dist> prek hook
backed by its own shared/<dist>/.pre-commit-config.yaml. The single
mypy-shared iterator is gone — mypy_local_folder.py accepts shared/<dist>
as a first-class folder and the per-hook virtualenv now lives at
.build/mypy-venvs/shared-<dist>/ (slash in the folder name is replaced
with a dash in the venv/cache path).
Adds a new check-shared-mypy-hooks prek hook that fails when a
shared/<dist> workspace member is missing its dedicated .pre-commit-
config.yaml, printing the exact YAML to add. Selective-checks emits one
skip entry per dist, enumerated from shared/ at run time. Contributing
docs cover the two-step process for adding a new shared library.
* Pin minimum_prek_version to 0.3.4 consistently across all configs
All .pre-commit-config.yaml files now require prek >= 0.3.4 (the version
already declared by the root config). Previously the nested configs
pinned a mix of 0.2.0, 0.3.2, and 0.3.4, so a contributor could pass the
root's version check and still trip on stale subproject pins as they
moved between directories.
* Refresh uv.lock after rebase to reflect the `mypy` dep groups
The rebase onto main resolved the uv.lock conflict by taking main's
version, so `uv sync --group mypy` would fail against uv.lock until
the groups added to the per-distribution pyprojects were re-resolved.
Regenerates the lockfile to include them.
* Add explicit selective-checks test for per-shared-dist mypy hook skipping
Verifies that when a file under shared/logging/ changes, only
mypy-shared-logging is kept among the thirteen mypy-shared-* hooks;
all other shared distributions' hooks land in the skip list. Pins the
contract that the runtime enumeration over shared/*/pyproject.toml
works as intended.
* Refresh mypy docs to match the per-hook venv + --group mypy workflow
Fills in the docs that still referenced the pre-split workflow:
- AGENTS.md: mentions `mypy-shared-<dist>` per shared workspace member and
the `uv sync --group mypy` install path for mypy itself.
- scripts/ci/prek/AGENTS.md: clarifies that non-provider mypy hooks run
locally through mypy_local_folder.py (Breeze image only needed for the
providers hook).
- dev/breeze/doc/03_developer_tasks.rst: renames stale `mypy-airflow` to
`mypy-airflow-core`, and expands the cache note to cover the per-hook
virtualenvs and caches under .build/.
- dev/breeze/doc/ci/04_selective_checks.md: expands the file-group and
skip-reason lists so every new mypy hook (scripts, task-sdk, airflow-ctl,
the six test-dir hooks, and mypy-shared-<dist> enumerated at runtime)
is documented.
* Rename mypy_local_folder.py to run_mypy_full_dist_local_venv_or_breeze_in_ci.py
Updates every .pre-commit-config.yaml entry and prose references so they
point at the new script name. Two shared configs use YAML folded-scalar
entries to stay under the 110-char yamllint limit; updates the validation
script's expected template to match.
* Move ExecutorCallback execution into a supervised process and make the entire flow more generic to account for future workload types
* merge conflict whitespace oops
* moved runtime error into supervise_callback()
* add some context tot eh comment about not supporting message types
* Copilot review fixes
* Improve callback display name
* remove execute_callback from export
* Shivam fixes
* static checks
* fix Core/SDK barrier issues
* Ash fixes - create supervise_workload and move setproctitle in there as well
* merge conflict fixes adn mypy
* fix unit tests
* add running state back
* account for *args in accepts_context
* fix encoder
* fix context, use closure for subprocess, and plumb callback logs through to the scheduler
* fix dequeue cleanup
* Kaxil fixes rnd 2
- move bundle loading into start()
- add _make_process_nondumpable to supervise_callback
- fix context passing to callback_callable
- revert adding *args check to accepts_context
- parameterize tests
- top-leveled some imports
* move run_workload in to base_executor for SDK/Core split policy
* somehow missed this file in the commit
* mypy fix
* kaxil fixes rnd 3
* compat test fix
* replace mock with actual object
* fix celery compat issues
* switch from log to _log
Set `exclude-newer = "4 days"` in `[tool.uv]` to only consider package
versions uploaded at least 4 days ago. This provides supply-chain security
by giving the community time to identify and remediate compromised releases.
Also downgrade pyodps requirement for Python >= 3.13 from >=0.12.6 to
>=0.12.5.1 since 0.12.6 was published within the cooldown window.
IntelliJ IDEA .iml files could end up in sdist/wheel packages when
building distributions that lack their own .gitignore. Adding *.iml
to every distribution's .gitignore ensures these IDE metadata files
are excluded from built packages.
Directory-only patterns (ending with /) in .airflowignore were incorrectly
matching files inside the directory, not just the directory itself. This caused
negation patterns like !abc/def/ to un-ignore all files within abc/def/ instead
of only un-ignoring the directory for traversal purposes.
Added dir_only flag to _GlobIgnoreRule that skips file matching for
directory-only patterns, consistent with gitignore specification.
Closes: #62716
* Fix all build-system/requires including transitive dependencies
Add a feature to fix all build-system/requires to make build
reproducibility works. The build-system requires are managed
automatically by the `upgrade-important-versions` prek hook.
This should not only provide reproducibility, but also it should
prevent cases where transient dependency upgrade might break
building sdist (for example as it happened with virtualenv 21
breaking hatchling with https://github.com/pypa/hatch/issues/2193
* Fix mypy 'Cannot infer type of lambda' error in upgrade script
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* op template field via callable serialization fix
* removed duplicate MockOperator import
* template var serialization: use md5 fingerprint
* doc for template field callable serialization logic
* template var callable serialization: use md5 from airflow.utils.hashlib_wrapper
* template var callable serialization: use callable fully qualified name
* Apply suggestions from code review
---------
Co-authored-by: AndreiLeib <andrei.leibovski@appdirect.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
github-actions[bot]andJarek Potiuk
Haram Lee
Shahar EpsteinandJarek Potiuk
Ash Berlin-Taylor
Tzu-ping ChungandLIU ZHE YOU
Sean GhaeliandSean Ghaeli
Sebastian Daum
Jens Scheffler
D. Ferruzzi
Jason(Zhe-You) Liu
Wei Lee
Yoann
Kaxil Naik
Amogh Desai