mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
Gabriel
cc2d6a82a5
## What problem does this PR solve? File Scanner V2 can use full multi-column predicates for Row Group statistics, but Page Index pruning previously handled only predicates that referenced one slot. Compound predicates such as the multi-column `OR` expressions in TPC-DS Q28 and the DNF expressions in Q88 therefore decoded the complete Row Group before residual filtering. ## What is changed? - Evaluate multi-column `AND`/`OR` predicate trees against Parquet Page Index metadata. - Intersect child candidate row ranges for `AND` and union them for `OR`, following the predicate tree rather than constructing decoder-level branch masks. - Cache decoded page zone maps per slot while evaluating multiple leaves from the same column. - Fall back conservatively: an unavailable `AND` child contributes no pruning, while an unavailable `OR` branch retains the complete Row Group range. - Keep the complete compound expression as a row-level residual predicate after all referenced columns are materialized, preserving SQL NULL and expression semantics. - Preserve the existing single-column dictionary, raw-filter, lazy-materialization, and Page Index paths unchanged. - Remove the auxiliary-reader and decoder branch-mask implementation from the earlier revision. ## Performance The Release microbenchmark scans the same 1,048,576-row PLAIN fixture in both variants and changes only the Doris Page Index switch. Both variants evaluate the same two-column `OR` residual and return exactly 209,712 rows. Results were pinned to one CPU, preceded by three warmups, and collected in ABBA order with 10 repetitions and a 0.5-second minimum per repetition. | Variant | Median CPU time | CPU time per raw row | Selected rows | |---|---:|---:|---:| | Page Index off | 12.51 ms | 11.93 ns | 209,712 | | Page Index on | 9.05 ms | 8.63 ns | 209,712 | | **Change** | **-27.67%** | **-27.67%** | **identical** | The single-column Page Index path is structurally unchanged; only conjuncts that do not resolve to a single slot enter the new evaluator. ## Test - 17 focused ASAN tests passed for native Parquet statistics and scanner behavior. - Coverage includes multi-column OR union, AND intersection, nested DNF, missing-index fallback, exact residual results, and existing single-column Page Index/range-gap pruning. - 13 Parquet benchmark scenario tests passed. - Release benchmark build succeeded and all 169 `ParquetReader` cases were registered. - The dedicated Page Index on/off benchmark completed without errors or row-count mismatches.