* Add consolidated Language SDK compatibility matrix Users comparing Language SDKs had to read each SDK's README, and the per-SDK manifests lagged behind what the SDKs actually ship. One generated matrix, split by authoring mode, shows what each SDK supports for a Dag definition and for a Python Dag with stub TaskHandler. * Add callbacks, listeners and provider operators to the SDK matrix Users comparing Language SDKs could not see where callbacks, Airflow's listener API and Python provider operators stand, because the matrix had no dimension for any of them. Callbacks ask different things of an SDK per authoring mode, so they are declared per mode like the other shared capabilities. * Address review on the Language SDK matrix generator The umbrella capability and the authoring mode both read `dag-definition`, which gives one name two meanings in the same manifest. A row's kind also belongs in its entry key: state and capability names happen not to collide today, so the lookup worked by accident rather than by design. * Rename the dynamic-task-mapping dimension to mapped-tasks The dimension is about whether a task can be a mapped task, which the shorter name says directly and keeps the matrix row readable next to the other task-level dimensions. * Split callbacks into Dag-level and task-level dimensions The callbacks of a Python Dag run whichever SDK executes its stub tasks, so ``dag-callbacks`` is a Dag-definition concern only, while ``task-callbacks`` stays declared per authoring mode. Drop ``provider-operators`` until ADR-0009 is accepted, and reuse the shared ``capabilities_doc`` fixture in the per-SDK README tests. * Fix the hook attribution in the matrix generation diagram ``index.rst`` is written by ``update-lang-sdk-central-matrix`` from every SDK in ``LANG_SDKS``, not by the per-SDK README hook. ---------
11 KiB
Airflow Java SDK
Write Apache Airflow Dags and tasks in Java, Kotlin, Scala or any other JVM language.
Status: Beta · Java 11+
Installation
Apply the Gradle plugin, and add the SDK and its annotation processor:
plugins {
id("org.apache.airflow.sdk") version "<version>"
}
dependencies {
annotationProcessor("org.apache.airflow:airflow-sdk-processor:<version>")
implementation("org.apache.airflow:airflow-sdk:<version>")
}
airflowBundle {
mainClass = "com.mycompany.airflow.hello.Main"
}
Replace <version> with a release from Maven Central. See the guide for Maven.
Quick start
Declare a Dag:
package com.mycompany.airflow.hello;
import org.apache.airflow.sdk.*;
@Airflow.Dag(id = "java_hello", schedule = "@daily", queue = "java")
public class HelloDag {
@Airflow.Task
public long extract() {
return 42L;
}
@Airflow.Task
public void report(long rows) {
System.out.println("extracted " + rows + " rows");
}
@Airflow.Deps
static class Wiring implements HelloDagDeps {
void depends() {
report(extract());
}
}
}
Serve it from the main class:
package com.mycompany.airflow.hello;
import org.apache.airflow.sdk.Bundle;
import org.apache.airflow.sdk.Server;
public class Main {
public static void main(String[] args) {
Server.create(args).serve(new Bundle().register(HelloDag.class));
}
}
Build the bundle into build/bundle/:
./gradlew bundle
Copy build/bundle/ into a Dag bundle, such as the default dags-folder bundle, and send the java queue to
the Java coordinator in airflow.cfg:
[sdk]
coordinators = {"java": {"classpath": "airflow.sdk.coordinators.java.JavaCoordinator"}}
queue_to_coordinator = {"java": "java"}
Airflow reads the bundle like any other Dag file: java_hello shows up in the UI, runs on its schedule, and
report receives the value extract returned.
Learn more
The Java SDK guide covers:
| Topic | Guide section |
|---|---|
| Task inputs, options, order-only dependencies, task groups, branching, triggering another Dag | Declaring a Dag in Java |
@Airflow.TaskHandler for the @task.stub tasks of a Python Dag, and how arguments bind |
Implementing stub tasks of a Python Dag |
Client, Context, and what a task returns |
Writing tasks |
| Building a bundle with Gradle or Maven | Building a bundle |
Where to deploy a bundle, and the JavaCoordinator options |
Deploying |
| What a Dag declared in Java cannot do yet | Limitations |
See example/ for a working project.
Compatibility matrix
Which Airflow TaskInstance states and capabilities this SDK supports. This table is generated from
capabilities.yaml; the conformance dimensions are defined in the
Language SDK conformance spec.
Do not edit the table by hand — edit capabilities.yaml and let the update-java-sdk-readme-matrix
prek hook regenerate it.
Min. Airflow version: 3.4.0 · supervisor schema: 2026-10-30
| Dimension | Tier | Supported | Since (Airflow) | Since (SDK) | Notes |
|---|---|---|---|---|---|
| TaskInstance states | |||||
state: success |
MUST | ✓ | 3.4.0 | 1.0.0 | |
state: failed |
MUST | ✓ | 3.4.0 | 1.0.0 | |
state: up_for_retry |
MUST | ✓ | 3.4.0 | 1.0.0 | RetryTask |
state: skipped |
SHOULD | ✓ | 3.4.0 | 1.0.0 | a TriggerDagRun task that skips an existing run |
state: deferred |
MAY | ✓ | 3.4.0 | 1.0.0 | a TriggerDagRun task that waits with deferrable |
state: up_for_reschedule |
MAY | ✗ | – | – | runtime does not emit RescheduleTask yet |
state: awaiting_input |
MAY | ✗ | – | – | runtime does not emit AwaitInputTask yet |
state: removed |
MAY | ✓ | 3.4.0 | 1.0.0 | |
| Task runtime | |||||
capability: task-logging |
MUST | ✓ | 3.4.0 | 1.0.0 | SLF4J + JPL bridged to the task log |
capability: xcom-read-write |
MUST | ✓ | 3.4.0 | 1.0.0 | |
capability: connection-read |
MUST | ✓ | 3.4.0 | 1.0.0 | |
capability: variable-read-write |
MUST | ✓ | 3.4.0 | 1.0.0 | |
capability: self-contained-bundle |
MUST | ✓ | 3.4.0 | 1.0.0 | Airflow metadata embedded in the jar artifact |
capability: retry-policy |
MAY | ✗ | – | – | no task-facing retry-policy API yet |
capability: task-state-store |
MAY | ✓ | 3.4.0 | 1.0.0 | Client.getTaskStateStore() get/set/delete/clear |
capability: asset-state-store |
MAY | ✗ | – | – | no task-facing asset state-store API yet |
capability: asset-event-emit |
MAY | ✗ | – | – | runtime does not emit asset events yet |
capability: asset-event-read |
MAY | ✗ | – | – | no task-facing asset-event API yet |
capability: object-store |
MAY | ✗ | – | – | no object-storage API yet |
capability: listeners |
MAY | ✗ | – | – | no hook into the Python listener API, so a task emits no OpenLineage facets of its own |
| Dag definition | |||||
capability: dag-definition-authoring |
SHOULD | ✓ | 3.4.0 | 1.0.0 | |
capability: dag-args |
MUST † | ✓ | 3.4.0 | 1.0.0 | |
capability: task-args |
MUST † | ✓ | 3.4.0 | 1.0.0 | |
capability: dag-params |
MUST † | ✗ | – | – | |
capability: dag-callbacks |
MAY † | ✗ | – | – | |
capability: explicit-dependencies |
MUST † | ✓ | 3.4.0 | 1.0.0 | before / after, or @Airflow.Deps |
capability: edge-labels |
MAY † | ✓ | 3.4.0 | 1.0.0 | Label.of(text).on(task) |
capability: taskflow-dependencies |
MUST † | ✓ | 3.4.0 | 1.0.0 | |
capability: task-group |
MAY † | ✓ | 3.4.0 | 1.0.0 | |
capability: task-loops |
MAY † | ✗ | – | – | |
capability: if-else-branching |
SHOULD † | ✓ | 3.4.0 | 1.0.0 | DagDef.If or @Airflow.If |
capability: switch-case-branching |
SHOULD † | ✓ | 3.4.0 | 1.0.0 | DagDef.Switch or @Airflow.Switch |
capability: trigger-dag-run |
MAY † | ✓ | 3.4.0 | 1.0.0 | TriggerDagRun |
capability: dynamic-dag-generation |
MAY † | ✓ | 3.4.0 | 1.0.0 | |
capability: human-in-the-loop |
MAY † | ✗ | – | – | |
capability: dag-test |
SHOULD † | ✗ | – | – | |
capability: asset-definition |
MAY † | ✗ | – | – | |
capability: asset-scheduling |
MAY † | ✗ | – | – | |
capability: mapped-tasks |
MAY † | ✗ | – | – | |
capability: asset-inlets-outlets |
MAY † | ✗ | – | – | |
capability: task-callbacks |
MAY † | ✗ | – | – | no SDK-defined task callbacks |
| Python Dag with stub TaskHandler | |||||
capability: mixed-lang-stub-target |
MUST | ✓ | 3.4.0 | 1.0.0 | @task.stub |
capability: taskflow-binding |
MUST | ✓ | 3.4.0 | 1.0.0 | mapped stubs excluded |
capability: mapped-tasks |
MAY | ✗ | – | – | Airflow does not bind arguments to mapped stubs yet |
capability: asset-inlets-outlets |
MAY | ✗ | – | – | runtime does not report task outlets yet |
capability: task-callbacks |
MAY | ✗ | – | – | a Python-declared task callback does not run for a stub task |
Marks: ✓ supported · ✗ not supported · n/a not applicable. A tier marked † applies only when dag-definition-authoring is supported.
Links
- Java SDK guide
- API reference (generated from the sources)
- Maven Central
- Source: the
java-sdk/directory of the Apache Airflow monorepo - Issues: bug reports and feature requests
- Developing this SDK (building, testing, and the release process)