Jack
88511083a6
### What problem does this PR solve? Issue Number: N/A Related PR: #69292 (depends on this fix) Problem Summary: A CTE fails to plan when the optimizer inlines it after one of its scans got a virtual column: ```sql WITH c AS ( SELECT id, score() AS s FROM t WHERE c1 MATCH 'apple' ORDER BY s DESC LIMIT 2 ) SELECT * FROM c; -- ERROR: A expression contains slot not from children ``` A score or ANN TopN pushed into a scan, and a MATCH in the select list, add a virtual column to the scan. After rewrite, the optimizer inlines a CTE that has one consumer left (`Optimizer.normalizeCtePlan` → `CTEInliner`) by deep copying its rewritten body, so the copy includes these scans: - `score()` and `l2_distance_approximate()` are nondeterministic, so the rewrite does not inline a CTE that calls them, even with one consumer. Once the TopN pushdown has replaced them with a virtual column of the scan, the optimizer inlines the CTE. So every CTE with `ORDER BY score() LIMIT n` or an ANN TopN fails, with default settings. - A CTE with MATCH in its select list reaches the optimizer when the rewrite removes all but one of its consumers, e.g. `... UNION ALL SELECT * FROM c WHERE 1 = 0`. `LogicalPlanDeepCopier` copied a scan with virtual columns wrongly: - it gave each virtual column two new ids: the slots above the scan were mapped to the first one, and the copied scan output the second one; - it copied the virtual columns in list order, so a virtual column that refers to a later one (CSE with `experimental_enable_virtual_slot_for_cse=true`, e.g. `[v2 := v1 * v1, v1 := x + y]`) kept the old id of the later one; - the copied scan still output the original virtual column, with its original id, next to the new one; - the score and ANN order keys of the copied scan still used the original slots. So the plan above the copied scan read a slot that the scan does not output. The fix: - The copier gives every virtual column its new id first and then copies their expressions, which reuse those ids, so virtual columns may refer to each other in any order. It also maps the score and ANN order keys to the new slots. Relations without virtual columns are copied as before. - `LogicalOlapScan.withVirtualColumns` replaces the virtual columns of the scan, but kept their old slots in the output next to the new ones. It now replaces them there too and keeps the other output slots, which the copier has already mapped. Computing the output of the copy again would not work: it gives VARIANT subpath slots new ids. `AdjustNullable`, the other caller that replaces existing virtual columns, no longer keeps two slots for a virtual column whose nullability changes. #69292 binds `score()` to its scan before CTEs are inlined, so it needs this fix; it carries the same change until this PR is merged.
🌍 Read this in other language
English • العربية • বাংলা • Deutsch • Español • فارسی • Français • हिन्दी • Bahasa Indonesia • Italiano • 日本語 • 한국어 • Polski • Português • Română • Русский • Slovenščina • ไทย • Türkçe • Українська • Tiếng Việt • 简体中文 • 繁體中文
?style=for-the-badge){kind=icon}
?style=for-the-badge){kind=icon}
Apache Doris is an open-source, real-time analytics and search database built on MPP architecture. It provides fast SQL analytics, lakehouse query acceleration, and hybrid search across structured, text, and vector data.
Explore the official website for the latest product overview, use cases, ecosystem updates, blogs, and user stories. For version updates, see all release notes.
📈 Use Cases
| Use Case | What it provides |
|---|---|
| Customer-Facing Analytics | Ship sub-second interactive analytics to external users. |
| Data Warehousing | Build one real-time warehouse across business domains. |
| Observability | Analyze high-throughput logs, events, and metrics with SQL. |
| Doris for AI | Use vector, text, JSON, and structured search in one SQL engine. |
🚀 Core Capabilities
Apache Doris is built around three core capabilities. The website is the source of truth for detailed product descriptions and examples.
| Capability | What it provides |
|---|---|
| Real-Time Analytics | Streaming ingestion, incremental transformation, and sub-second queries under high concurrency. |
| Lakehouse Analytics | Fast SQL analytics over open table formats such as Iceberg, Delta Lake, and Hudi. |
| Hybrid Search | SQL-native analytics across JSON, full-text, and vector data for AI and search workloads. |
🔌 Ecosystem
Doris sits at the center of the modern data stack. It connects upstream databases, streaming systems, and lakehouse storage with downstream BI, AI, analytics, and observability tools.
For the latest ecosystem coverage, visit the official website and the connection and integration documentation.
👣 Get Started
🧱 Architecture
Apache Doris supports both compute-storage coupled and compute-storage decoupled deployments. In decoupled mode, stateless compute groups run over shared object storage, so you can scale compute on demand and isolate workloads.
Learn more in the deployment guide and deployment mode guide.
📣 Project Updates
| Resource | What it provides |
|---|---|
| Community Report | Weekly updates on community activity, merged PRs, contributors, and feature progress. |
| Roadmap 2026 | The 2026 planning discussion for AI and hybrid search, query engine, storage, and data lake work. |
🧩 Components
Doris provides connectors and tools for common data engineering workflows.
- Doris Flink Connector
- Doris Spark Connector
- Doris Kafka Connector
- Doris Stream Loader
- Doris Kubernetes Operator
👨👩👧👦 Users
Apache Doris is used in production by thousands of companies worldwide across internet services, finance, retail, logistics, manufacturing, energy, telecommunications, AI, and other industries.
🙌 Contributors
Apache Doris graduated from the Apache Incubator and became an Apache Top-Level Project in June 2022. Thanks to all community contributors who help build Doris.
🌈 Community and Support
💬 Contact Us
🧰 Links
- Apache Doris official website: doris.apache.org
- X: @doris_apache
- Medium: @ApacheDoris
📜 License
Note
Some licenses of the third-party dependencies are not compatible with Apache 2.0 License. So you need to disable some Doris features to comply with Apache 2.0 License. For details, refer to the
thirdparty/LICENSE.txt