[test](be) Extend Variant V1/V2 segment benchmarks (#66668)

ColumnVariantV2 needs a reproducible component benchmark that exercises real segment write/read paths, scales beyond one segment, distinguishes materialized and sparse exact-path reads, and measures the current production compaction pipeline without inventing a benchmark-only V2 compaction mode. ### What is changed? - Keep 1,000,000 rows as the checked-in default and add environment-controlled total rows and rows per segment. Input is generated in 4K batches, while parse/append/finalize remain in the timed region. - Add multi-segment ingestion for `Sparse16`, `Doc16`, and `Full` layouts. - Add exact-path read cases corresponding to the storage-scan portion of a literal `SELECT v['path']` after subpath pruning: - `Materialized100pct`: materialized `LEAF` route. - `Sparse6pct`: sparse-bucket `BINARY_EXTRACT` route with one hit per 17 rows. - Add a scan-and-rewrite pipeline benchmark. It queries key plus whole Variant from the same prevalidated V1-written source Segment as `ColumnVariant` or `ColumnVariantV2`, then passes that queried Block directly to a new `SegmentWriter` and reports every pipeline stage separately. - Add a current-production cumulative-compaction benchmark with ten overlapping rowsets written by Variant V2. It calls public `CumulativeCompaction::execute_compact()`, disables and verifies the ordered-link shortcut, and validates the physically rewritten output. - Keep correctness checks outside timing: full row/layout/route checks, bounded whole-value canonical samples, and full exact-path checksums/hit counts. - Do not use whole-column read performance as a gate in this round. The compaction benchmark intentionally does **not** label the current compactor as V2. `TabletColumn::variant_is_v2` is a transient read marker and is not persisted through schema protobuf copies, so the benchmark records `persisted_compaction_variant_v2=0` instead of creating a misleading V1/V2 pair. ### Release benchmark evidence - Default 1M ingestion, five samples per representation: | Layout | V1 median (ms) | V2 median (ms) | V2/V1 | |---|---:|---:|---:| | Sparse16 | 9,212.349 | 8,202.836 | 0.8904 | | Doc16 | 7,917.702 | 8,286.466 | 1.0466 | | Full | 10,231.216 | 9,195.444 | 0.8988 | - Final exact-path 1M, five samples per representation: | Route | V1 median (ms) | V2 median (ms) | V2/V1 | |---|---:|---:|---:| | Materialized `LEAF` | 3.630 | 2.403 | 0.6619 | | Sparse `BINARY_EXTRACT` | 28.254 | 27.006 | 0.9558 | #### Scan and rewrite: 1M rows The source is the same V1-written physical Segment for both sides. The only switched dimension is the requested query representation and therefore the column handed to the destination writer. Every group contains five V1 and five V2 samples in fixed ABBA order. Lower is better. | Layout | Representation | Total median (ms) | Read median (ms) | Append median (ms) | Finalize median (ms) | Total CV | |---|---|---:|---:|---:|---:|---:| | Sparse16 | V1 | 8,429.347 | 3,347.014 | 1,580.744 | 3,501.255 | 0.554% | | Sparse16 | V2 | 11,275.605 | 5,869.060 | 2,005.063 | 3,401.644 | 0.543% | | Doc16 | V1 | 7,035.376 | 2,719.646 | 699.392 | 3,610.837 | 0.238% | | Doc16 | V2 | 11,568.249 | 6,106.403 | 1,977.956 | 3,485.382 | 0.095% | | Full | V1 | 8,023.795 | 2,010.926 | 1,166.440 | 4,804.266 | 0.212% | | Full | V2 | 13,154.337 | 4,567.923 | 4,201.698 | 4,400.651 | 0.799% | V2/V1 total latency is `1.3377` for Sparse16, `1.6443` for Doc16, and `1.6394` for Full. The gap is in the query/read and append stages; destination finalize is comparable or slightly faster for V2 (`0.92x`-`0.97x`). Destination segment size and physical layout match for V1 and V2 in every layout.
2026-11-04 07:31:17 +00:00 · 2026-08-20 15:39:08 +08:00
1 parent 4b07b5f594
commit 243df79a0c
@@ -31,6 +31,7 @@
 #include "benchmark_fmod.hpp"
 #include "benchmark_hll_merge.hpp"
 #include "benchmark_json_extract.hpp"
+#include "benchmark_variant_segment.hpp"
 #include "benchmark_zone_map_index.hpp"
 #include "binary_cast_benchmark.hpp"
 #include "common/config.h"
@@ -0,0 +1,124 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euo pipefail
+
+if [[ $# -lt 2 || $# -gt 3 ]]; then
+    echo "Usage: $0 <benchmark-binary> <result-dir> [benchmark-filter]" >&2
+    exit 2
+fi
+
+benchmark_binary=$1
+result_dir=$2
+benchmark_filter=${3:-'^BM_Variant(IngestToSegment|ReadWholeColumn|ReadExactPath|ScanAndRewriteSegment)/'}
+script_path=$(realpath "${BASH_SOURCE[0]}")
+repo_root=$(git -C "$(dirname "${script_path}")" rev-parse --show-toplevel)
+be_config=${repo_root}/conf/be.conf
+benchmark_root=${DORIS_VARIANT_BENCHMARK_ROOT:-/tmp}
+thirdparty_installed_path=$(realpath "${repo_root}/thirdparty/installed")
+
+if [[ ! -x "${benchmark_binary}" ]]; then
+    echo "Benchmark binary is not executable: ${benchmark_binary}" >&2
+    exit 2
+fi
+if [[ ! -f "${be_config}" ]]; then
+    echo "Benchmark config does not exist: ${be_config}" >&2
+    exit 2
+fi
+if [[ ! -d "${benchmark_root}" || ! -w "${benchmark_root}" ]]; then
+    echo "Benchmark root must be a writable directory: ${benchmark_root}" >&2
+    exit 2
+fi
+benchmark_binary=$(realpath "${benchmark_binary}")
+benchmark_root=$(realpath "${benchmark_root}")
+java_library_path=${JAVA_HOME:?JAVA_HOME must be set}/lib/server
+runtime_library_path="${java_library_path}${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}"
+
+mkdir -p "${result_dir}"
+
+{
+    date --iso-8601=seconds
+    echo "repo_root=${repo_root}"
+    echo "invocation_pwd=${PWD}"
+    echo "git_head=$(git -C "${repo_root}" rev-parse HEAD)"
+    git -C "${repo_root}" status --short --branch
+    echo "benchmark_binary=${benchmark_binary}"
+    sha256sum "${benchmark_binary}"
+    echo "JAVA_HOME=${JAVA_HOME}"
+    echo "runtime_LD_LIBRARY_PATH=${runtime_library_path}"
+    LD_LIBRARY_PATH="${runtime_library_path}" ldd "${benchmark_binary}"
+    echo "be_config=${be_config}"
+    sha256sum "${be_config}"
+    echo "variant_storage_parse_mode_config:"
+    grep -nE '^[[:space:]]*variant_storage_parse_mode[[:space:]]*=' "${be_config}" ||
+        echo "<compiled default>"
+    echo "thirdparty_installed=${thirdparty_installed_path}"
+    while IFS= read -r fingerprint; do
+        echo "$(basename "${fingerprint}")=$(< "${fingerprint}")"
+    done < <(find "${thirdparty_installed_path}" -maxdepth 1 -type f \
+        -name '*-build-fingerprint.txt' | sort)
+    if [[ -f "${repo_root}/be/build_Release/CMakeCache.txt" ]]; then
+        sha256sum "${repo_root}/be/build_Release/CMakeCache.txt"
+        grep -n '^CMAKE_BUILD_TYPE:' "${repo_root}/be/build_Release/CMakeCache.txt"
+    fi
+    echo "benchmark_root=${benchmark_root}"
+    df -h "${benchmark_root}"
+    findmnt -T "${benchmark_root}" -o TARGET,SOURCE,FSTYPE,OPTIONS
+    lsblk -o NAME,TYPE,SIZE,ROTA,MODEL,MOUNTPOINTS
+    free -h
+    swapon --show
+    uname -a
+    lscpu
+    echo "load_before=$(< /proc/loadavg)"
+    echo "rows=${DORIS_VARIANT_BENCHMARK_ROWS:-1000000}"
+    echo "rows_per_segment=${DORIS_VARIANT_BENCHMARK_ROWS_PER_SEGMENT:-1000000}"
+    echo "filter=${benchmark_filter}"
+    echo "cpu=${DORIS_BENCHMARK_CPU:-unbound}"
+} >"${result_dir}/environment.txt"
+
+git -C "${repo_root}" diff --binary HEAD -- >"${result_dir}/source.diff"
+sha256sum "${result_dir}/source.diff" >>"${result_dir}/environment.txt"
+
+command=("${benchmark_binary}"
+         "--benchmark_filter=${benchmark_filter}"
+         "--benchmark_counters_tabular=true"
+         "--benchmark_out=${result_dir}/raw.json"
+         "--benchmark_out_format=json")
+
+if [[ -n "${DORIS_BENCHMARK_CPU:-}" ]]; then
+    command=(taskset -c "${DORIS_BENCHMARK_CPU}" "${command[@]}")
+fi
+
+DORIS_HOME="${repo_root}" \
+    DORIS_VARIANT_BENCHMARK_ROOT="${benchmark_root}" \
+    LD_LIBRARY_PATH="${runtime_library_path}" \
+    /usr/bin/time -v -o "${result_dir}/resource.txt" "${command[@]}" \
+    >"${result_dir}/stdout.txt" 2>"${result_dir}/stderr.txt"
+
+echo "load_after=$(< /proc/loadavg)" >>"${result_dir}/environment.txt"
+
+if ! grep -q '"name": "BM_Variant' "${result_dir}/raw.json"; then
+    echo "No Variant benchmark cases matched; see ${result_dir}/raw.json" >&2
+    exit 1
+fi
+
+if grep -q '"error_occurred": true' "${result_dir}/raw.json"; then
+    echo "One or more benchmark cases failed; see ${result_dir}/raw.json" >&2
+    exit 1
+fi
@@ -2316,8 +2316,6 @@ Status parse_and_materialize_variant_columns(Block& block, const std::vector<uin
            { return _parse_and_materialize_variant_columns(block, variant_pos, configs); });
 }

-namespace {
-
 ParseConfig::ParseTo select_storage_variant_parse_target(const TabletColumn& column,
                                                         const ParseConfig& config) {
    // NestedGroup consumes the parse-time subcolumn tree to build nested storage structures, so it
@@ -2357,8 +2355,6 @@ ParseConfig::ParseTo select_storage_variant_parse_target(const TabletColumn& col
    }
 }

-} // namespace
-
 Status parse_and_materialize_variant_columns(Block& block, const TabletSchema& tablet_schema,
                                             const std::vector<uint32_t>& column_pos) {
    std::vector<uint32_t> variant_column_pos;
@@ -256,6 +256,10 @@ Status parse_and_materialize_variant_columns(Block& block, const std::vector<uin
 void parse_json_to_variant(IColumn& column, const StringRef& jsons, JsonParser* parser,
                           const ParseConfig& config);

+// Select the V1 parse target used by the storage write path for the given schema column.
+ParseConfig::ParseTo select_storage_variant_parse_target(const TabletColumn& column,
+                                                         const ParseConfig& config);
+
 // Parse variant columns by picking variant positions from `column_pos` and generating ParseConfig
 // based on tablet schema settings (flatten nested / doc snapshot mode).
 Status parse_and_materialize_variant_columns(Block& block, const TabletSchema& tablet_schema,
@@ -146,6 +146,39 @@ private:
    int32_t _old_value;
 };

+TEST(VariantUtilTest, SelectStorageVariantParseTargetPrecedence) {
+    TabletSchema plain_schema = _make_variant_schema(false, false);
+    TabletSchema doc_schema = _make_variant_schema(true, false);
+    TabletSchema nested_doc_schema = _make_variant_schema(true, true);
+    ParseConfig parse_config;
+
+    {
+        ScopedVariantStorageParseMode parse_mode(0);
+        EXPECT_EQ(select_storage_variant_parse_target(plain_schema.column(0), parse_config),
+                  ParseConfig::ParseTo::OnlyDocValueColumn);
+
+        parse_config.deprecated_enable_flatten_nested = true;
+        EXPECT_EQ(select_storage_variant_parse_target(plain_schema.column(0), parse_config),
+                  ParseConfig::ParseTo::OnlySubcolumns);
+        EXPECT_EQ(select_storage_variant_parse_target(doc_schema.column(0), parse_config),
+                  ParseConfig::ParseTo::OnlyDocValueColumn);
+        EXPECT_EQ(select_storage_variant_parse_target(nested_doc_schema.column(0), parse_config),
+                  ParseConfig::ParseTo::OnlySubcolumns);
+    }
+
+    parse_config.deprecated_enable_flatten_nested = false;
+    {
+        ScopedVariantStorageParseMode parse_mode(1);
+        EXPECT_EQ(select_storage_variant_parse_target(plain_schema.column(0), parse_config),
+                  ParseConfig::ParseTo::OnlySubcolumns);
+    }
+    {
+        ScopedVariantStorageParseMode parse_mode(2);
+        EXPECT_EQ(select_storage_variant_parse_target(plain_schema.column(0), parse_config),
+                  ParseConfig::ParseTo::OnlyDocValueColumn);
+    }
+}
+
 TEST(VariantUtilTest, NumericConflictResolvedAsJsonbPreservesOriginalNumberText) {
    // Values distributed to different tablets can reach the reader as a JSONB conflict. JSONB
    // preserves the original number text instead of formatting the value through DOUBLE.