minghong
e143cb2019
### What problem does this PR solve? Issue Number: close #xxx Related PR: #62205 Problem Summary: PR #62205 introduced the `length(str)` OFFSET optimization for nested column pruning. When `length()` is applied to a string-like nested field, the pruner marks the access path with an OFFSET suffix so BE can read only the offset array instead of full element data. However, the OFFSET path dedup logic operates at **slot granularity**: if ANY non-OFFSET path exists for the same slot, ALL OFFSET paths are stripped. This is incorrect for `array<struct<...>>` columns where different struct fields have independent access patterns. **Bug 1 (crash):** Given a query like: ```sql SELECT array_match_all(x -> length(struct_element(x, 'str_field')) > 0, arr), struct_element(element_at(arr, 1), 'int_field') FROM t ``` The `int_field` non-OFFSET path causes the `str_field` OFFSET path to be stripped. BE never reads `str_field` data → crash or wrong results. **Fix:** Replace slot-level OFFSET dedup with per-field bidirectional prefix matching. An OFFSET path `P+["OFFSET"]` is only stripped when a non-OFFSET path Q shares a prefix relationship with P (covering the same container or ancestor), not when Q accesses a sibling struct field. **Bug 2 (wrong pruning):** When `isStringOffsetOnly=true` AND `accessPartialChild=true` (e.g., `cardinality(arr) + arr[*].f1`), `pruneDataType()` returns the full type instead of pruning unused fields. **Fix:** Add `!accessPartialChild` guard to the `isStringOffsetOnly` check. ### Release note Fix a BE crash caused by nested column pruning incorrectly stripping OFFSET access paths for array<struct> columns when different struct fields have independent access patterns (e.g., length() on one field + direct access on another).
Guide for test cases
General Case
-
Write "def" before variable names; otherwise, they will be global variables and may be affected by other cases running in parallel.
Problematic code:
ret = ***Correct code:
def ret = *** -
Avoid setting global session variables or modifying cluster configurations in cases, as it may affect other cases.
Problematic code:
sql """set global enable_pipeline_x_engine=true;"""Correct code:
sql """set enable_pipeline_x_engine=true;""" -
If it is necessary to set global variables or modify cluster configurations, specify the case to run in a nonConcurrent manner.
-
For cases involving time-related operations, it is best to use fixed time values instead of dynamic values like the
now()function to prevent cases from failing after some time.Problematic code:
sql """select count(*) from table where created < now();"""Correct code:
sql """select count(*) from table where created < '2023-11-13';""" -
After streamloading in a case, add a sync to ensure stability when executing in a multi-FE environment.
Problematic code:
streamLoad { ... } sql """select count(*) from table """Correct code:
streamLoad { ... } sql """sync""" sql """select count(*) from table """ -
For UDF cases, make sure to copy the corresponding JAR file to all BE machines.
-
Do not create the same table in different cases under the same directory to avoid conflicts.
-
Cases injected should be marked as nonConcurrent and ensured injection to be removed after running the case.
-
Docker case run in a docker cluster. The docker cluster is new created and independent, not contains history data, not affect other cluster.
Docker case can add/drop/start/stop/restart fe and be, and specify fe and be num.
Example will see demo_p0/docker_action.groovy
Read the annotation carefully in the example file.
Also read the doris-compose readme.
Compatibility case
Refers to the resources or rules created on the initial cluster during FE testing or upgrade testing, which can still be used normally after the cluster restart or upgrade, such as permissions, UDF, etc.
These cases need to be split into two files, load.groovy and xxxx.groovy, placed in a folder, and tagged with the restart_fe group label, example.