* Java SDK: Read the Dag run conf into a class with typed fields
Signed-off-by: PoAn Yang <payang@apache.org>
* Document the clashing field names case of getConf
Signed-off-by: PoAn Yang <payang@apache.org>
* Reject nulls for primitives in getConf and document JsonProperty
Signed-off-by: PoAn Yang <payang@apache.org>
* Move getConf details from the Java SDK guide to the API reference
Signed-off-by: PoAn Yang <payang@apache.org>
---------
Signed-off-by: PoAn Yang <payang@apache.org>
The Default Model Provider field was a blank text box, so users had to know
pydantic-ai's provider names (`openai-chat` vs `openai`) by heart. The field's
schema now lists common ones as `examples`, which the connection form renders as
suggestions on a text input: any other provider pydantic-ai recognizes can still
be typed. An `enum` dropdown would reject providers pydantic-ai adds until a new
release of this provider. A test keeps every suggestion resolvable.
The suggestions ship with the provider, but the installed pydantic-ai can
recognize more. The field's help text and the connection docs now say the list
is common choices, not every provider that works. A test also keeps each
suggestion among the connection's declared external services, so the form never
suggests a vendor the docs and registry don't list.
* Keep the model provider help text accurate on the registry and share test lookups
The registry's connection builder shows a conn-field's description but not its
examples, so the help text now says any provider the installed pydantic-ai
recognizes works instead of describing suggestions. The docs paragraph no longer
repeats part of the list by hand. The suggestion checks reuse one connection-type
lookup and one prefix-to-module mapping, and the two pydantic-ai checks on this
field are one test.
The example Dag's ConnectionError rules used the builtin ConnectionError, which requests.exceptions.ConnectionError does not subclass, so they never matched what the example task raises. The docs marked RESCHEDULE as added in 3.5.0 (it shipped in 3.4.0), quoted a chain reason the page's own policy cannot produce, said every policy decision is logged (only decisions with a reason are), and listed only AirflowFailException as skipping the policy.
Several constraints-<VERSION> tags ship constraints files for Python
versions the release does not support, for example 3.13 for Airflow
3.0.x. Following the documented install command with such a file pins
packages that have no wheels for that interpreter, so users hit a
grpcio source build failure instead of a clear missing-file error.
Generated-by: Claude Opus 5
* Use only the configured auth type when creating tokens in the FAB auth manager
Generated-by: Claude Opus 5
* Mock security manager with specific class for tests
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
The PROD image build first built the Airflow and provider wheels, but only after the CI image and
generate-constraints had finished, although the wheels need neither. In a RunsOn pull request run
that put them on the chain that ends the run: CI image to minute 9.1, constraints to 15.2, wheels
15.5 to 17.9, PROD image to 29.6, Kubernetes tests to 41.3. The CI workflows now build the wheels
in their own job right after build-info and tell the PROD image build they are already uploaded;
the other callers of prod-image-build.yml still build them as before.
Generated-by: Claude Opus 5
* Run the dependency upgrade summary beside the tests instead of after them
The "Deps" jobs need only the CI images, but as part of finalize-tests they waited for every test
to finish and then ran for up to 17 minutes: on a canary, finalize-tests started at minute 59 and
the run ended at minute 78, with the image-cache push done at minute 69. In their own workflow,
started once the CI images are built, they finish long before the tests. The CI passed gate and
check-timed-out-jobs need them, so a failing summary still fails the run as before.
Generated-by: Claude Opus 5
* Read the Deps job from its new workflow in the dependency report tests
Generated-by: Claude Opus 5
The example_dms system test intermittently fails in await_task_start
because DMS stops the replication task a few seconds after starting it,
before it reaches the running state. The StopReason added to the sensor
failure message in #75240 only said "Stop Reason NORMAL", with no
failure message, so the DMS API does not explain the stop. The DMS task
logs are the next place to look.
Copying the task logs into the Airflow task log puts them next to the
rest of the system test output, and deleting the log group afterwards
keeps the hourly runs from leaving one behind each time.
The --only-idle argument read its default from config when the parser was built, so a config-enabled only_idle looked like an explicitly passed flag and tripped the guard requiring a positive --num-runs, which defaults to -1. It also made the option impossible to switch off from the command line.
Generated-by: Claude Opus 5
Co-authored-by: Eason09053360 <185830721+Eason09053360@users.noreply.github.com>
The AWS console setup steps told users to set AIRFLOW__ECS_EXECUTOR__*
and AIRFLOW__LAMBDA_EXECUTOR__* variables, but the executors read the
aws_ecs_executor and aws_lambda_executor config sections. Airflow
silently ignored the documented variables, so ECS tasks started without
the security groups and subnets needed to reach the metadata database,
and the Lambda executor had no SQS queue URL.
* UI: Reduce initial JavaScript payload
Airflow currently loads code for every UI page before rendering the active route, increasing download, parse, and execution work during startup.
* UI: Show progress during route navigation
Cold route chunks can take time to arrive on slow connections, leaving navigation without visible feedback.
* UI: Recover from failed lazy route loading
Browsers can retain an older entrypoint across a deployment and request route chunks that no longer exist, leaving navigation stuck on an unrecoverable error.
* UI: Fix stale route chunk recovery
* UI: Scope lazy route test setup
* UI: Keep lazy route tests stable
The fork CI monitor posted a Slack item each time reading a fork run's
jobs hit a transient GitHub server error (HTTP 502, a reset HTTP/2
stream). The item carried the error text, so a changed error re-posted
it, and one successful run in between re-posted it too.
Count the consecutive runs whose job statuses failed per pull request,
report only after three in a row, and use fixed wording so the item is
posted once while the failure persists. The run log keeps each error.
Generated-by: Claude Opus 5
Cache FTPS clients only after connection setup completes. Close clients on setup failure without masking the original exception, and verify that subsequent calls create a fresh connection.
Co-authored-by: drewrukin <drewrukin@users.noreply.github.com>
Co-authored-by: Przemysław Mirowski <17602603+Miretpl@users.noreply.github.com>
* Add migration guide from Git-Sync sidecar to GitDagBundle
The Git-Sync sidecar (dags.gitSync) is planned for removal in Helm
Chart 2.0 in favour of the Airflow-native GitDagBundle. This adds a
chart docs page mapping the old dags.gitSync values to their
dagProcessor.dagBundleConfigList equivalents, with authentication
patterns and a step-by-step migration.
* Address reviewer feedback: remove inaccurate Git-Sync claim
Veyu noted Git-Sync could checkout any commit via rev/ref params, so the statement that it always checked out the latest commit was not accurate. Removed that sentence per the preferred fix, keeping the GitDagBundle versioning note.
* docs: address review feedback on Git-Sync to GitDagBundle migration guide
- Fix spellcheck: favour -> favor (en_US)
- Generalize Git-Sync sidecar description (persistence-setting dependent)
- Prereq: latest Helm Chart 1.x release with Airflow 3
- credentialsSecret row lists both GITSYNC_* and GIT_SYNC_* key names
- Clarify what kwargs.git_conn_id refers to
- Note the Airflow Connection UI/CLI alternative to extraSecrets/extraEnv
---------
Co-authored-by: Przemysław Mirowski <17602603+Miretpl@users.noreply.github.com>
A docs-only change under dev/breeze/ drafted the pull request and waited for a maintainer as if it changed CI, although only the files that decide which jobs run and on which runners (workflows, breeze sources, the breeze installation, and scripts/ci/prek/ whose provider dependencies script selective checks runs) can let a pull request steer its own CI. Such pull requests still wait for a maintainer to run project CI, but are no longer drafted, so they stay visible for review. Changes to the other files CI uses are no longer held, but are still pointed out to the reviewer in a one-time comment. The provider dependencies script now ignores uv configuration from the checkout, so a pull request cannot redirect where its dependencies come from.
Generated-by: Claude Opus 5
The UI gained a Versions tab on the Dag page that compares two stored
versions of a Dag, but the UI overview page did not mention it. Add a
section next to the other Dag tabs describing how to pick versions, what
the change table shows, who can see values, and the Max changes limit,
with dark and light screenshots.
* Show which skills an agent was offered and used in the AI Model tab
When an agent behaves differently from one run to the next, the skills it
had and the ones it loaded are the first thing to check, and a skill pulled
from Git can change between runs without the Dag changing. Recording the
cloned commit next to each skill turns "it behaved differently yesterday"
into "the skill changed between these two commits".
* Keep skill calls made before an approval pause or a durable retry
A run resumed after a tool approval starts in a new process, and a durable retry
replays earlier tool calls without running them, so counting calls as they pass
through the toolset showed loaded skills as unused. Both get the earlier calls
back as message history, so the counts now come from there. A second skills
toolset on the same agent no longer replaces the first one's skills either.
Loops, the tasks inside them and the task groups around them were left out of
the Grid multi-selection because a looped task's cell sums up every pass while
its link opens only the newest one. The bulk Clear, Mark as and Delete actions
now address each pass on their own, so a loop cell can stand for every pass it
covers, the way a mapped task's cell stands for all its mapped task instances.
The bulk endpoint reports a task instance that has a region by its id rather
than by its coordinates, so the action bar also maps those ids back to the
selected cells to deselect them once they are updated.
* Run Kerberos worker sidecars as Kubernetes native sidecars
* Enhance tests for Kerberos sidecar to assert restart policy and startup probe
* Refactor Kerberos initialization tests to use parameterization for better coverage and clarity
* Update Kerberos init container logic and descriptions to clarify behavior with sidecar settings
* Remove Kerberos init container in favor of native worker sidecars
* Describe Kerberos sidecar startup probe per worker component
* remove unnecessary test assertion
* restore _helpers.yaml
* Update chart/tests/helm_tests/airflow_aux/test_pod_template_file.py
Co-authored-by: Przemysław Mirowski <17602603+Miretpl@users.noreply.github.com>
* Refactor Kerberos sidecar tests to improve clarity and coverage for enabled/disabled states
* Remove deprecated kerberosInitContainer configuration and update schema to reflect changes
* Remove unused test for kerberos sidecar pod override reconciliation
* Refactor test for removed kerberosInitContainer settings to improve error handling and clarity
* Drop Kerberos init container rejection from chart values schema
* Add documentation for Kerberos worker sidecars as Kubernetes native sidecars
* Refactor documentation for Kerberos worker sidecars to clarify container configuration and requirements
---------
Co-authored-by: Przemysław Mirowski <17602603+Miretpl@users.noreply.github.com>
safe_call_command interpolated exception and server error text straight
into Rich markup. A message containing a tag-like fragment such as
"[/x]" made rich.print raise MarkupError, so the user saw a traceback
instead of the real failure and the exit code was no longer 1.
Escape the text in the AirflowCtl, protocol, timeout, server response
and HTTP status handlers.
* Allow deleting queued partitioned asset events through the REST API
The DELETE queuedEvents endpoints only cleared queued events of
non-partitioned assets, so a partitioned event that was wrong or no
longer relevant could not be removed and would still trigger the
downstream Dag run. They now accept an optional partition_key to delete
the queued events of that partition whose Dag run has not been created
yet.
* Match partition_key against the asset event partition key
The partition_key on the DELETE queued events endpoints now refers to
the partition key of the queued asset event, the same key used by
POST /assets/{asset_id}/materialize, instead of the partition key of
the Dag run the event would create. This lets a single wrong upstream
partition be cleared when a partition mapper rolls several of them up
into one Dag run.
* Test that deleting queued partitioned events respects Dag read access
* Mention the partition key when no queued partitioned event is found
* Keep events of deleted Dag runs when deleting queued partitioned events
A partitioned asset event row outlives its partition record once the Dag
run it created is deleted, since only the partition record cascades.
Those rows were matched as queued events, so the endpoint deleted them
and returned 204 although nothing was queued. Only events of a partition
still waiting for its Dag run are deleted now.
* Test that Dag scoped queued event deletes leave other Dags' partitions alone
* Scope the pending partition cleanup to the partitions the call clears
Deleting queued partitioned events swept every empty pending partition in the
table, so clearing one asset's key could drop a partition of another Dag that
the request never matched. The partitions to clear are now collected before
anything is deleted, and both deletes are restricted to them.
The asset route now deletes what the caller may edit rather than what it may
read, so its test has to tell the two apart.
* Lock the partitions a queued event delete is about to clear
Two calls clearing different assets of the same partition could each still see
the other's uncommitted events under READ COMMITTED, so neither would notice
the partition had been emptied and it would stay pending with nothing left to
wait for.
* Name the queued event 404 detail helper with an action verb
* Say which partition key a queued event delete matches
A partition mapper can map an asset event's partition key to a different
Dag run partition key, so a caller reading only the parameter name cannot
tell which of the two the filter applies to.
* Build the partitioned queued event filter with the shared helper
The non-partitioned and partitioned queue tables carry the same asset,
Dag and timestamp columns, so two copies of the same filter drift apart
as soon as one of those columns changes.
* Regenerate the API spec and UI client
Keeps the generated clients carrying the same parameter documentation the
route declares.
* Say that one asset event can feed several partitioned Dag runs
A fan-out mapper turns one upstream partition key into several downstream
keys, and a rollup Dag run is fed by more than one event, so describing
the filter against a single Dag run's key mis-states what it matches.
Reviews for the provider currently route to three owners. Adding two active
contributors spreads that load and lets GitHub request them automatically
instead of relying on manual pings.
airflowctl sent the two filters as start_date and end_date query
parameters. The Dag runs endpoint only reads range parameters such as
start_date_gte and end_date_lte, and FastAPI drops parameters it does
not declare. The command returned all Dag runs with no error.
A bundle is deployed as a whole, so its pages should show every file and
the full counts rather than whatever slice of Dags the caller can read.
Filtering by readable Dags made the counts differ between users and hid
a bundle whose only file failed before defining a Dag, which is the case
the page most needs to show. Since that exposes every deployed file, the
bundle pages need operator-level access instead of Dag read.
The API client logs "Server error" and the tenacity retry warning
through structlog, which prints to stdout when nothing configures it.
A single retried 5xx then puts two log lines in front of the
`-o json` output, while the command still exits 0. The shared error
handler also printed its messages to stdout. #76133 and #76134 moved
the messages of a few commands to stderr, but these two paths are
shared by every command.
* Simplify the masked Variable merge and drop unused merge depth arguments
The restore runs from a single place, so keeping it inline mirrors the
Connection update it is modelled on. Merging now walks every depth like
redaction does, so the depth arguments threaded through _merge no longer
bound anything and only suggested a cutoff that does not exist.
* Read the stored Variable value once and clarify the merge max_depth docstring
* Use bulk ingest in AdbcHook.insert_rows when fast_executemany is set
AdbcHook.insert_rows built an Arrow RecordBatch per commit_every chunk
and bound it with cursor.executemany. Several ADBC drivers run a bound
stream once per row, so the previous fast_executemany path was a no-op
that set cursor.fast_executemany inside suppress(AttributeError).
Route fast_executemany (now the default) to cursor.adbc_ingest in
append mode. Keep executemany when fast_executemany is False, and
always bind when replace=True because bulk ingest cannot carry the
dialect's conflict clause. Chunking, commits, target_fields, and
schema.table splitting are unchanged.
Co-authored-by: Aldo <aoelvp94@gmail.com>
* Fall back to binding when the ADBC driver has no bulk ingest
Co-authored-by: Aldo <aoelvp94@gmail.com>
* Use Logger spec and patch decorators in ADBC insert_rows tests
Match AGENTS.md: spec every mock, and prefer @mock.patch.object over
a with-block for the replace=True insert_rows tests.
Co-authored-by: Aldo <aoelvp94@gmail.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
GitHub answers HTTP 502 on every attempt when it lists 100 jobs per page of
some large fork runs that were re-run, while smaller pages of the same run
list fine. The fork CI monitor then never read the re-run's jobs, so job
statuses of the failed first attempt stayed red on the pull request after
the re-run passed, as on #74696.
Generated-by: Claude Opus 5
Magpie 0.9.0.dev202611021858 was released and changed the code-review skill's surface. Without the refresh, contributors' Magpie skill pre-flights keep proposing a project-wide reconcile against the old version.
Generated-by: Claude Opus 5
* Work around mapped inlet population bug
* Tighten limit and expected_count checks
* Make sensor 3.2-compatible
Specific arguments that only make sense in 3.4+ get additional checks
at parse time to be rejected. Importing and using the sensor works on
3.2 and 3.3 as long as you do not use the 3.4+ parameters.
* Remove dotted process_events format
* Better BaseModel dump
* Improve AssetEventSensor docs and example
* Add AssetEventSensor to standard provider
* Scope asset event sensor/trigger tests to Airflow 3.4+ and regenerate provider build files
* Address review: dedicated exception, validation, docstring, and tests for AssetEventSensor
* Coerce datetime after/before to ISO strings, drop type: ignore (Copilot review)
* Add AssetEventSensor howto docs and example DAG
* Fix docs build: drop unresolved asset ref and reword to fix spellcheck
* Drop deferrable support from AssetEventSensor
The execution API calls backing asset event fetching (GetAssetEventByAsset
and GetAssetEventByAssetAlias) are not available in the triggerer, so the
deferrable path could not work in practice. Ship the synchronous sensor
only; deferrable support can follow once the triggerer can fetch asset
events and InletEventsAccessor exposes a native async interface.
Removes AssetEventTrigger and its tests, folds the shared fetch/count/
serialize helpers into the sensor module, and drops the now-unused
asset sensor exceptions.
* Declare asset sensor dependencies and support minimum event counts
Task-level asset waits need visible lineage without changing the Dag schedule. Explicit count policies let consumers wait for a minimum number of events without missing the condition when several events arrive between pokes.
* Reject ambiguous asset sensor targets and invalid event counts
Conflicting selectors can silently wait for a different asset, while fractional counts and invalid limits can produce misleading waits. Result processing also needs clear limits so consumers know which events their callback can inspect.
* Use UTC in asset sensor tests
* Unify sensor asset argument
* Support run-relative asset event bounds
Asset waits need time bounds tied to a run without requiring Jinja expressions
or limiting Dag authors to a fixed set of anchors. A runtime resolver contract
lets authors derive custom bounds from task context.
* Fix AssetEventSensor documentation heading
* Add explicit asset event filters and stable run bounds
* Allow empty extra filters on older Airflow versions
An empty extra dictionary adds no query predicate, so it should not require Airflow 3.4 when another filter is present.
---------
Co-authored-by: Tzu-ping Chung <tp@astronomer.io>
* Load Java Dags from a bundle path that contains a colon
The JVM splits -classpath on the path separator (':' on Linux) and has
no way to escape it. When a Dag bundle's path contains one, for example
a version directory named after a timestamp like
2026-11-02T15:52:20Z, JavaCoordinator passed absolute JAR paths, the
JVM split them apart, never loaded the JAR and exited with code 1, so
the Dag failed to parse with "exited with code 1 without a parse
result".
JavaCoordinator now starts the JVM in the bundle root and puts the
JARs on the classpath relative to it, so the bundle's own path never
appears there. SubprocessCoordinator gets a _get_working_directory hook
that both the Dag parse (chdir before exec) and task execution (Popen
cwd) use. A JAR whose path inside the bundle still holds the separator
is rejected with an error that names it.
* Prefix relative Java classpath entries with ./
The java launcher reads an argument that starts with "@" as an @argfile,
including the value after -classpath, so a bundle whose first JAR sat
under a directory like "@scope" failed to start. Every entry now starts
with "./".
* Test a Java bundle path holding the path separator end to end
Run the task and Dag parse launch tests in a bundle directory named after
a timestamp too, so the working directory and the relative classpath are
checked together for both paths.
Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
---------
Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
* Fix Dag id collision in the Java SDK example bundle
The interface example registered its task handlers as a Java Dag, so the
example jar declared a Dag with the same id as the Python Dag that owns
those tasks. When the jar is also a Dag bundle, the jar's Dag replaced
the Python one and its runs failed. The new java_task_handler_ prefix
also tells these Dags apart from the Java-defined java_native_ ones.
Co-Authored-By: Claude <noreply@anthropic.com>
* Register the interface example handlers inline
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Send the Python task's execution_timeout to the supervisor
* Load the stub task only inside the is_stub branch of ti_run
Only a task a language SDK runs needs its definition from the serialized Dag, so keeping every use of it under that branch makes it clear a Python task never loads one.
* Let a stub task ask for no execution_timeout
A language SDK runtime has to tell a task that sets no timeout, which gets the worker's default, from one that asks for none. Sending zero for the latter needs a negative timeout to stay negative, so the supervisor, not the server, turns it into the grace period alone.
* Drop the note on where the default execution_timeout is read
* Set the run context's execution_timeout only in the coordinator
* Address review nits on execution_timeout docs and schema description
* Reword execution_timeout comment for runtimes that send SetExecutionTimeout
* Address review nits on ti_run stub task lookup
DuckDB reports a denied S3 request only as "HTTP 403", without saying
which credentials it used, so the AWS identity is the first thing to
check. sts:GetCallerIdentity returns it and needs no IAM permission.
The lookup is only a debugging aid and must never affect the task. It
has short timeouts and no retries, and a failure is logged at debug
level and ignored.
* [main] CI: Upgrade important CI environment
* Ignore pydantic-ai's generic OpenAI-compatible provider in vendor drift checks
pydantic-ai-slim 2.63.0 adds a provider for any OpenAI-compatible endpoint. It needs a service name at construction and infer_model has no prefix for it, so a common.ai connection can never reach it and it is not a vendor the services page should list.
Generated-by: Claude Opus 5
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Rishabh
PoAn Yang
Tzu-ping Chung
Kaxil Naik
Jarek Potiuk
Rahul Vats
Ramit Kataria
Aaron Chen
Pavan Nandi
safaehar
dependabot[bot]
Rakesh PaiandPrzemysław Mirowski
Yuseok Jo
Jed Cunningham
Pierre Jeambrun
Henry Chen
Christos Bisias
Andrew Chang
ZJ-lucky-0915
Wei Lee
AldoandCursor Agent
Amogh Desai
Hussein Awala
Jens Scheffler
ChickenBenny
Niko Oliveira
github-actions[bot]andJarek Potiuk