mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
loser
a623df21d7
### What problem does this PR solve? Issue Number: close #66516 Related PR: #68349 Problem Summary: Lance catalogs are currently read-only from Doris. This PR adds DDL support backed by the Lance Namespace API for both filesystem and REST catalogs. This PR retargets the original branch-4.1 PR to branch-4.2 because branch-4.1 is code frozen. - Add create/drop namespace operations, including FORCE semantics and root-namespace protection. - Add create/drop/rename table operations and ENGINE=LANCE / SHOW CREATE TABLE support. - Convert Doris schemas to Arrow and create empty Lance datasets through Arrow IPC. - Add top-level ADD, MODIFY, RENAME, and DROP COLUMN schema evolution. - Refresh Doris metadata, schema caches, and the upstream Lance table-access cache after successful mutations. - Reject unsupported defaults, positions, comments, complex type changes, nested paths, and multi-clause schema changes explicitly. The current Lance Java SDK does not implement explicit backfill/virtual-column operations for DirectoryNamespace or RestNamespace, so this PR exposes only operations that the pinned SDK can execute reliably and documents that limitation. ### Release note Support namespace, table, and top-level column DDL for Lance external catalogs. ### Test Plan - Compile the FE reactor against JDK 17. - Run focused Lance DDL, type conversion, namespace mutation, metadata cache, and ALTER TABLE command tests. - Run Checkstyle and Markdownlint. - Add a filesystem regression suite covering successful DDL and unsupported operations. ### Test Result - FE reactor compilation: passed. - 65 focused unit tests: passed. - Checkstyle: passed with 0 violations. - Markdownlint for docs/lance-ddl.md: passed. - The Doris + MinIO regression suite was added but not executed locally because this environment does not provide that cluster. Signed-off-by: wangzhaobo957-cloud <wangzhaobo957@gmail.com> Signed-off-by: trae-solo-agent <solo-agent@trae.ai>
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # fe-common This module is used to store some common classes of other modules. # spark-dpp This module is Spark DPP program, used for Spark Load function. Depends: fe-common # fe-core This module is the main process module of FE. Depends: fe-common, spark-dpp