nprasad2077/airflow

0 Fork 0

mirror of https://github.com/apache/airflow.git synced 2026-11-04 15:19:48 +00:00

Files

T

Jason(Zhe-You) LiuandClaude 31b558a607 Fix Dag id collision in the Java SDK example bundle (#76019 )

* Fix Dag id collision in the Java SDK example bundle

The interface example registered its task handlers as a Java Dag, so the
example jar declared a Dag with the same id as the Python Dag that owns
those tasks. When the jar is also a Dag bundle, the jar's Dag replaced
the Python one and its runs failed. The new java_task_handler_ prefix
also tells these Dags apart from the Java-defined java_native_ ones.

Co-Authored-By: Claude <noreply@anthropic.com>

* Register the interface example handlers inline

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

2026-11-03 02:40:54 +00:00

4.5 KiB

Raw Blame History

Java SDK examples

Example Dags for the Apache Airflow Java SDK. See the Java SDK guide for the full documentation.

Build

Publish the SDK to the local Maven repository, then build the example bundle into build/bundle/:

cd java-sdk
./gradlew publishToMavenLocal -PskipSigning=true

cd example
../gradlew bundle

Configure Airflow

export AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[
  {"name": "dags-folder", "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle", "kwargs": {}},
  {
    "name": "java-task-handlers",
    "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
    "kwargs": {"path": "/absolute/path/to/airflow/java-sdk/example/build/bundle"}
  }
]'
export AIRFLOW__SDK__COORDINATORS='{
  "java": {
    "classpath": "airflow.sdk.coordinators.java.JavaCoordinator",
    "kwargs": {"task_handler_bundle_name": "java-task-handlers"}
  }
}'
export AIRFLOW__SDK__QUEUE_TO_COORDINATOR='{"java": "java"}'

The workers and the Dag processor need Java 11 or later on PATH.

Dag definition

These Dags are declared entirely in Java, in src/java/org/apache/airflow/example/nativedag/:

Dag	Source	Shows
`java_native_annotation_example`	`AnnotationExample.java`	The annotation style: task inputs, a task group, `@Airflow.If`, `@Airflow.Switch`, and `TriggerDagRun`
`java_native_interface_example`	`InterfaceExample.java`	A similar Dag in the interface style, with `DagDef`
`java_native_target_example`	`TargetExample.java`	The Dag the other two trigger

airflow dags trigger java_native_annotation_example
airflow dags trigger java_native_interface_example

Python Dag with stub TaskHandler

These Dags are declared in Python, in src/resources/dags/java_examples.py, and their stub tasks are implemented in Java:

Dag	Source	Shows
`java_task_handler_annotation_example`	`AnnotationExample.java`	`@Airflow.TaskHandler` methods, with arguments by position and by name
`java_task_handler_interface_example`	`InterfaceExampleBuilder.java`	`Task` and `InputTask` classes registered by Dag and task id
`java_task_handler_xcom_casting_example`	`XComCastingExample.java`	How values passed between tasks convert to the declared Java types

Copy src/resources/dags/java_examples.py into your Airflow Dags folder, and create the Connection and Variable the examples read:

export AIRFLOW_CONN_TEST_HTTP='{
    "conn_type": "http",
    "login": "user",
    "password": "pass",
    "host": "example.com",
    "port": 1234,
    "extra": {"param1": "val1", "param2": "val2"}
}'
export AIRFLOW_VAR_MY_VARIABLE=123

airflow dags trigger java_task_handler_annotation_example
airflow dags trigger java_task_handler_interface_example
airflow dags trigger java_task_handler_xcom_casting_example

4.5 KiB Raw Blame History