mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
Gabriel
e48911e417
### What problem does this PR solve? Flight SQL exposes VARIANT as UTF8, preventing ADBC clients from receiving native binary Variant V2 values. On branch-4.1, return Variant V2 through the `arrow.parquet.variant` extension by default, with `struct<metadata: binary not null, value: binary not null>` storage. SQL NULL is a null struct; Variant null is an encoded non-null value. Clients without a Variant extension implementation can access the physical struct and field metadata. Support **Variant V2 only**. Legacy Variant returns an unsupported error, including nested legacy fields, constants, all-null columns and empty results. Reject legacy types during BE Flight schema conversion and defensively in the legacy SerDe. Remove the output-format session variable and result-sink flag entirely. Users who need text can explicitly `CAST(... AS STRING)`. Keep execution, GetTables, GetSchema and prepared-statement schemas consistent, including nested Variant fields. Native Variant output does not depend on a heartbeat capability or a cluster-wide BE support check. Retain the Variant V2 type requirement and reject non-native execution schemas. Reuse the V2 binary representation and compact dictionaries per selected row, preserving physical scalar types and decimal scales without repeatedly copying unrelated dictionary keys. Keep the value helper private to `data_type_variant_v2_serde.cpp`. Native encoding accepts up to 128 nested levels. Iceberg output keeps its existing behavior. Document the V2 requirement and client decoding limitations in the Python Flight sample README; parsing JSON does not bypass the configured Variant representation. ### Release note Arrow Flight SQL / ADBC returns Variant V2 as native Arrow Variant binary by default, without a session switch. Legacy Variant is unsupported. Use an explicit SQL cast to STRING for text output. ### Validation - The native-output update passed all 46 selected Arrow/Flight tests using a focused ASAN binary; the binary conversion code is unchanged by the capability removal. - For the capability removal, regenerated the heartbeat Thrift bindings, compiled the affected FE sources and BE heartbeat translation unit, and passed all 6 focused FE tests covering native schemas, legacy/UTF8 rejection and the existing Paimon heartbeat capability. - FE Checkstyle, clang-format 16 and diff checks passed. - The updated Groovy regression script and Python sample syntax were checked in the preceding update. - A full production build and live cluster regression/ADBC execution were not rerun for this update; CI validation is requested. ### Check List (For Author) - Test - [x] Regression test updated - [x] Unit Test - Behavior changed: - [x] Yes. Variant V2 uses native binary by default; legacy Variant requires an explicit text cast. - Does this need documentation? - [x] Yes, usage, V2-only scope and client decoding limitations are documented in the Python Flight sample README.