* Add consolidated Language SDK compatibility matrix Users comparing Language SDKs had to read each SDK's README, and the per-SDK manifests lagged behind what the SDKs actually ship. One generated matrix, split by authoring mode, shows what each SDK supports for a Dag definition and for a Python Dag with stub TaskHandler. * Add callbacks, listeners and provider operators to the SDK matrix Users comparing Language SDKs could not see where callbacks, Airflow's listener API and Python provider operators stand, because the matrix had no dimension for any of them. Callbacks ask different things of an SDK per authoring mode, so they are declared per mode like the other shared capabilities. * Address review on the Language SDK matrix generator The umbrella capability and the authoring mode both read `dag-definition`, which gives one name two meanings in the same manifest. A row's kind also belongs in its entry key: state and capability names happen not to collide today, so the lookup worked by accident rather than by design. * Rename the dynamic-task-mapping dimension to mapped-tasks The dimension is about whether a task can be a mapped task, which the shorter name says directly and keeps the matrix row readable next to the other task-level dimensions. * Split callbacks into Dag-level and task-level dimensions The callbacks of a Python Dag run whichever SDK executes its stub tasks, so ``dag-callbacks`` is a Dag-definition concern only, while ``task-callbacks`` stays declared per authoring mode. Drop ``provider-operators`` until ADR-0009 is accepted, and reuse the shared ``capabilities_doc`` fixture in the per-SDK README tests. * Fix the hook attribution in the matrix generation diagram ``index.rst`` is written by ``update-lang-sdk-central-matrix`` from every SDK in ``LANG_SDKS``, not by the per-SDK README hook. ---------
7.4 KiB
Module sdk
Write Apache Airflow Dags and tasks in Java, Kotlin, Scala or any other JVM language.
It requires Java 11 or later.
Quick start
Apply the Gradle plugin, and add the SDK and its annotation processor:
plugins {
id("org.apache.airflow.sdk") version "<version>"
}
dependencies {
annotationProcessor("org.apache.airflow:airflow-sdk-processor:<version>")
implementation("org.apache.airflow:airflow-sdk:<version>")
}
airflowBundle {
mainClass = "com.mycompany.airflow.hello.Main"
}
Replace <version> with a release from Maven Central.
Declare a Dag with @Airflow.Dag, its tasks with @Airflow.Task, and its graph in an @Airflow.Deps class.
Passing the value one task returns into another is how the tasks depend on each other:
package com.mycompany.airflow.hello;
import org.apache.airflow.sdk.*;
@Airflow.Dag(id = "java_hello", schedule = "@daily", queue = "java")
public class HelloDag {
@Airflow.Task
public long extract() {
return 42L;
}
@Airflow.Task
public void report(long rows) {
System.out.println("extracted " + rows + " rows");
}
@Airflow.Deps
static class Wiring implements HelloDagDeps {
void depends() {
report(extract());
}
}
}
Serve it from the main class:
package com.mycompany.airflow.hello;
import org.apache.airflow.sdk.Bundle;
import org.apache.airflow.sdk.Server;
public class Main {
public static void main(String[] args) {
Server.create(args).serve(new Bundle().register(HelloDag.class));
}
}
Build the bundle with ./gradlew bundle, and see the Java SDK guide for deploying it, and for everything
a Dag declared in Java can do.
Language SDK compatibility matrix
Which Airflow TaskInstance states and capabilities the Java SDK currently supports. The normative meaning of each dimension is defined in the Language SDK conformance specification.
Min. Airflow version: 3.4.0 · supervisor schema: 2026-10-30
| Dimension | Tier | Supported | Since (Airflow) | Since (SDK) | Notes |
|---|---|---|---|---|---|
| TaskInstance states | |||||
state: success |
MUST | ✓ | 3.4.0 | 1.0.0 | |
state: failed |
MUST | ✓ | 3.4.0 | 1.0.0 | |
state: up_for_retry |
MUST | ✓ | 3.4.0 | 1.0.0 | RetryTask |
state: skipped |
SHOULD | ✓ | 3.4.0 | 1.0.0 | a TriggerDagRun task that skips an existing run |
state: deferred |
MAY | ✓ | 3.4.0 | 1.0.0 | a TriggerDagRun task that waits with deferrable |
state: up_for_reschedule |
MAY | ✗ | – | – | runtime does not emit RescheduleTask yet |
state: awaiting_input |
MAY | ✗ | – | – | runtime does not emit AwaitInputTask yet |
state: removed |
MAY | ✓ | 3.4.0 | 1.0.0 | |
| Task runtime | |||||
capability: task-logging |
MUST | ✓ | 3.4.0 | 1.0.0 | SLF4J + JPL bridged to the task log |
capability: xcom-read-write |
MUST | ✓ | 3.4.0 | 1.0.0 | |
capability: connection-read |
MUST | ✓ | 3.4.0 | 1.0.0 | |
capability: variable-read-write |
MUST | ✓ | 3.4.0 | 1.0.0 | |
capability: self-contained-bundle |
MUST | ✓ | 3.4.0 | 1.0.0 | Airflow metadata embedded in the jar artifact |
capability: retry-policy |
MAY | ✗ | – | – | no task-facing retry-policy API yet |
capability: task-state-store |
MAY | ✓ | 3.4.0 | 1.0.0 | Client.getTaskStateStore() get/set/delete/clear |
capability: asset-state-store |
MAY | ✗ | – | – | no task-facing asset state-store API yet |
capability: asset-event-emit |
MAY | ✗ | – | – | runtime does not emit asset events yet |
capability: asset-event-read |
MAY | ✗ | – | – | no task-facing asset-event API yet |
capability: object-store |
MAY | ✗ | – | – | no object-storage API yet |
capability: listeners |
MAY | ✗ | – | – | no hook into the Python listener API, so a task emits no OpenLineage facets of its own |
| Dag definition | |||||
capability: dag-definition-authoring |
SHOULD | ✓ | 3.4.0 | 1.0.0 | |
capability: dag-args |
MUST † | ✓ | 3.4.0 | 1.0.0 | |
capability: task-args |
MUST † | ✓ | 3.4.0 | 1.0.0 | |
capability: dag-params |
MUST † | ✗ | – | – | |
capability: dag-callbacks |
MAY † | ✗ | – | – | |
capability: explicit-dependencies |
MUST † | ✓ | 3.4.0 | 1.0.0 | before / after, or @Airflow.Deps |
capability: edge-labels |
MAY † | ✓ | 3.4.0 | 1.0.0 | Label.of(text).on(task) |
capability: taskflow-dependencies |
MUST † | ✓ | 3.4.0 | 1.0.0 | |
capability: task-group |
MAY † | ✓ | 3.4.0 | 1.0.0 | |
capability: task-loops |
MAY † | ✗ | – | – | |
capability: if-else-branching |
SHOULD † | ✓ | 3.4.0 | 1.0.0 | DagDef.If or @Airflow.If |
capability: switch-case-branching |
SHOULD † | ✓ | 3.4.0 | 1.0.0 | DagDef.Switch or @Airflow.Switch |
capability: trigger-dag-run |
MAY † | ✓ | 3.4.0 | 1.0.0 | TriggerDagRun |
capability: dynamic-dag-generation |
MAY † | ✓ | 3.4.0 | 1.0.0 | |
capability: human-in-the-loop |
MAY † | ✗ | – | – | |
capability: dag-test |
SHOULD † | ✗ | – | – | |
capability: asset-definition |
MAY † | ✗ | – | – | |
capability: asset-scheduling |
MAY † | ✗ | – | – | |
capability: mapped-tasks |
MAY † | ✗ | – | – | |
capability: asset-inlets-outlets |
MAY † | ✗ | – | – | |
capability: task-callbacks |
MAY † | ✗ | – | – | no SDK-defined task callbacks |
| Python Dag with stub TaskHandler | |||||
capability: mixed-lang-stub-target |
MUST | ✓ | 3.4.0 | 1.0.0 | @task.stub |
capability: taskflow-binding |
MUST | ✓ | 3.4.0 | 1.0.0 | mapped stubs excluded |
capability: mapped-tasks |
MAY | ✗ | – | – | Airflow does not bind arguments to mapped stubs yet |
capability: asset-inlets-outlets |
MAY | ✗ | – | – | runtime does not report task outlets yet |
capability: task-callbacks |
MAY | ✗ | – | – | a Python-declared task callback does not run for a stub task |
Marks: ✓ supported · ✗ not supported · n/a not applicable. A tier marked † applies only when dag-definition-authoring is supported.