Hard SDK validation can go stale when the server changes the allowed ID
rules, and the server already validates authoritatively, so the SDK only
warns. Per review, the check runs at the interface based syntax, which
both authoring styles compile down to: a checkAirflowBundle verification
task in the check lifecycle runs BundleInspector (the ADR-0003 shape)
against the assembled Bundle and prints a warning for every ID the
server would reject. Warnings never fail the build; a bundle that cannot
be built at all does.
* Java SDK: Record the class that declares each Dag and describe it on request
DagDef remembers the outermost class that built it, and a generated
@Builder.Dag builder names the annotated class instead of itself.
Server accepts --describe-sources <file> to write each Java-declared Dag's
declaring class and return without connecting, for the Gradle plugin.
* Java SDK: Pack each native Dag's source file into the bundle JAR
The packDagSources task runs mainClass in describe mode, finds each Dag's
source file through the class file's SourceFile attribute, and packs each
unique file with a sources.json mapping under META-INF/airflow/. The JAR
manifest names it with Airflow-Java-SDK-Sources.
* Java SDK: Bound the describe run and keep its payload to the bundle JAR
Every `Jar` task gained a dependency on `packDagSources`, so a sources or
javadoc JAR carried the Dag sources and claimed the
`Airflow-Java-SDK-Sources` attribute that the Python side picks a bundle
JAR out of a directory by. Only `jar` and `shadowJar` carry them now.
The describe run had no time bound, so a `main` that does not route
through `Server.create(args)` and starts serving instead held the build
open with no way out. It now runs under a timeout and falls back to
packing the entrypoint's source alone, which is what every other
describe failure already does.
Say in `serve` and `serveAsync` that a `--describe-sources` server
returns without connecting, and record in `sources.json` that its paths
are relative to the source directory, which is also where they sit under
`META-INF/airflow/sources/`.
* Java SDK: Show each Dag's own source and find it outside the package layout
The Java importer now passes the Dag ID to the source lookup, so the Code
view reads dag_source_paths instead of always showing the entrypoint.
The source locator falls back to a unique match on the SourceFile name
when the package does not match the directory, using an index built only
when the package path misses. An unresolvable entrypoint or Dag class is
now logged as a warning.
The describe task is no longer up to date after a failed run, so a
transient failure does not stick until a source changes. Its hint now
points at Server.create(args), which is what answers --describe-sources.
Add a shadowJar test, correct the comment on which JARs carry the sources
payload, and document that a Dag maps to the class that constructed it.
* Java SDK: Use a describe run's output even when the run itself failed
* Java SDK: Serve and describe sources from their own Server classes
* Java SDK: Pack the sources into the one JAR the bundle is built from
* Java SDK: Attach the plugin's documentation to the plugin class
* Java SDK: Drop the duplicated classes dir from the describe classpath
* Java SDK: Let a bundle set how long the describe run may take
* Replace coordinator artifact roots with task_handler_bundle_name
ADR-0013 locates Lang-SDK task handlers through a named Dag bundle, so the
unversioned jars_root, executables_root and bundles_root directories go away
instead of living beside it. dag_bundle_name becomes task_handler_bundle_name
because the native-Dag work will use the old name with a different meaning. It
has not shipped in a release, so there is no shim.
A coordinator reads handlers from the named bundle at its current version, or
from the task's own Dag bundle at the run's version, pinned for the task either
way. CoordinatorManager.from_config validates the name of every spec without
constructing coordinators, so a typo fails at config load instead of on the
first task.
* Serve lang-SDK harness artifacts from Dag bundles
The compose e2e and the k8s lang-SDK test configured coordinators with the
explicit artifact roots, which no longer exist. The Java e2e needs three
bundles because one bundle is one classpath, and its three coordinators each
need their own dependencies.
Artifacts stay mounted on the workers only, because only the workers run them;
the other components need just the bundle registration for the name to
resolve. The k8s init container keeps staging from S3 because the S3 download
drops the Go binary's execute bit, which the coordinator requires.
* Document artifact Dag bundles for language SDK coordinators
The Go, Java and TypeScript guides, the bundle specs, the READMEs and the
contributor guide still pointed users at the removed jars_root,
executables_root and bundles_root kwargs.
A Dag bundle now decides what a coordinator sees, so two of its limits reach
users. For Java, every JAR in the bundle goes on one classpath, so conflicting
dependency versions cannot share a bundle, and a second Main-Class in it makes
the entry point ambiguous. For Go, the coordinator skips files without the
execute bit, so an object-store Dag bundle that drops it leaves the
coordinator with no executable to find.
* Tidy names and mocks in the artifact bundle tests
Review flagged a Java test fixture still named after the removed jars_root
kwarg, an e2e helper without a verb, and two execute_task patches that were
not autospecced like the ones beside them.
* Say artifact Dag bundles are registered on every component
The guides said only the workers need the artifact Dag bundle registered, which
reads as permission to register it there alone. The [sdk] configuration is also
loaded by the scheduler under KubernetesExecutor, and it is rejected wherever the
name is missing, so the bundle belongs in the shared bundle list.
* Drop em-dashes from the rewritten doc sentences
Two sentences rewritten for artifact Dag bundles kept their em-dashes, which the
docs style for this work avoids.
* Say the Dag processor needs the language SDK setup too
In Airflow 3.4 the Dag processor checks a Python Dag's stub tasks against
the task handlers the language SDK artifacts register, so a deployment that
follows the old worker-only advice fails that check. task_handler_bundle_name
only serves those stub tasks, which the docs did not say.
* Validate task_handler_bundle_name only for routed coordinators
A coordinator no queue routes to is never used, so its kwargs are not checked.
Add from_config tests for an accepted name, an unrouted coordinator and a spec without the kwarg.
* Add newsfragment for the removed coordinator artifact roots
* Mark ADR-0013 as accepted
The Java SDK build trusted whatever its configured repositories returned, so a
replaced or shadowed artifact could execute during a build -- including in the
buildSrc code that configures every published module -- without anything
failing. Pinning checksums turns that substitution into a build error, and
declaring repositories in one place gives reviewers a single spot to audit
where bytes may come from.
Checksums alone are trust-on-first-use: they prove an artifact has not changed
since it was recorded, not that it was ever trustworthy. The README therefore
documents the independent cross-check required when regenerating the metadata,
and names the inputs this deliberately does not cover -- the auto-provisioned
JDK toolchain and the downloaded Supervisor Schema -- so the guarantee is not
read as broader than it is.
Andrew Chang
Jason(Zhe-You) Liu
Tzu-ping Chung