mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
Gabriel
99ae82ac36
### What problem does this PR solve?
Lance vector searches cannot configure the SDK's query parallelism from
Doris, and second-phase row-ID fetch profiles omit available read
counters. Materialization also nests two timers for the same `ExecTime`
counter, counting the synchronous fetch wait twice.
This PR targets `branch-4.1` and:
- Adds optional `vector_search("query_parallelism" = "4", ...)` support
through FE validation, Thrift, and the existing Lance-C API. Accepts
`-1`, `0`, and positive integers; omitting it preserves the SDK default.
EXPLAIN shows an explicit setting.
- Carries row-ID fetch call counts, returned row counts, and per-dataset
Foyer cache/origin logical read bytes through the fetch RPC into each
`RowIDFetcher` profile, preserving their units. Statistics are captured
after the reader closes.
- Removes the nested materialization execution timer; the operator
framework continues to count the fetch wait once.
The read-byte counters exclude metadata/index IO and block-alignment
amplification. The current take API does not expose physical request
counts or a separate decode timer; zero Foyer counters on bypassed paths
do not mean zero physical IO. These boundaries are documented in
`docs/lance-ann-profile.md`.
No third-party dependency or patch changes are required.
### Validation
- 43 focused FE tests passed, including option bounds, omitted defaults,
Thrift round trips, and EXPLAIN propagation.
- FE Checkstyle passed with zero violations.
- Clang-format 16 checks passed for every changed C++ source/header.
- C++ syntax checks passed for all changed translation units and both
affected BE test files using regenerated Thrift definitions.
- Groovy compilation passed for the changed regression suite.
- Added BE tests for parameter validation, result preservation,
row-fetch counters, RPC profile propagation, and avoiding duplicate
execution timing.
- Added external regression coverage comparing indexed results across
parallelism settings and rejecting invalid values. Full BE UT and
external regression execution remain for CI; local C++ checks were
syntax-only.
### Release note
Add configurable Lance vector query parallelism and row-ID fetch read
counters, and correct materialization execution-time accounting.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- Behavior changed:
- [x] Yes. Explicit query parallelism is now accepted; existing defaults
are preserved. Materialization profiles no longer double-count fetch
execution time.
- Does this need documentation?
- [x] Yes. Updated `docs/lance-ann-profile.md` in this PR.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label