mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
924060929
11bfb73889
### What problem does this PR solve? Related PR: #69501, #69629, #69632, #69720 Problem Summary: Complete guarded cleanup of stale external table lookup state after confirmed remote absence, preserving refreshed or same-name recreated tables and unrelated numeric lookup routes. External metadata reads can discover a dropped table after resolving a cached handle while FE names, table objects and numeric ID routes remain stale. A refreshed names list can also omit a resident old object. In case-folding modes, targeted cleanup previously cleared unrelated numeric routes. Route confirmed provider absence through a typed SPI exception carrying the exact remote database/table identity. Capture the existing lookup observation before handle resolution and retain it through base/system schemas (including pinned system schemas), latest/pinned row counts, ordinary write-schema binding and row-level DML validation, partition-values and partition-name metadata (SHOW PARTITIONS and PARTITIONS), eager/deferred scan partition views, MVCC partition/freshness reads and deferred loads, and scan column/property/filter/estimation and synchronous/deferred split reads. System relations repair their shared source identity; query TVFs have no shared lookup identity. Missing files, transport/permission failures and unrelated resources do not authorize cleanup. Preserve live views, cache TTLs, HMS exception wrapping and comment display fallbacks. Runtime, names, object and metadata-refresh guards reject delayed cleanup after refresh or same-name recreation. Catalog-wide, metadata-only and explicit lookup refreshes use the catalog-owned generation and active count across their full interval, protecting replacement database/table objects admitted by normal eviction or published during hot/cold invalidation. Database refresh and truncate replay include their cold-cache branches; rename replay, overlapping refreshes and errors use the same owner guard. Existing per-table metadata generations remain part of the observation. Ordinary writes remain metadata-only and do not reload table names. Targeted wide invalidation retires only the target's ROOT-folded ID aliases while retaining unrelated numeric routes. Provider IO remains outside the catalog monitor. Pinned system schema reads use the existing schema-session lifecycle hooks. Their live statement owns the borrowed scope until all aliases and scans finish; the schema read does not prematurely close it. The guarded resolver preserves master's view classification before table-endpoint access, including connectors that return non-404 errors for views; live view deletion still permits guarded cleanup. Hive file-size estimates preserve confirmed table absence until guarded FE cleanup, then retain the unknown estimate fallback. Sampled ANALYZE file-size reads retain the same observation and their existing failure/session-closure behavior. Ordinary listing failures still degrade statistics to unknown. Native Hive table and table-scoped partition-name RPCs preserve exact-source absence under the existing HmsClientException boundary; both ordinary/fresh and cancellation-aware listings use the same RPC normalization. Individual partition absence, ordinary failures and successful empty lists keep their existing behavior. MTMV partition warmup retains its pre-handle MetadataRead through snapshot binding and partition enumeration. The combined interface retains current master's DEFAULT-column and partition-filter APIs. Native Iceberg write-shaping loads preserve exact-source table absence through the existing shared converter, while ordinary write exceptions, resource tracking and statement ownership keep their prior contracts. The shared absence exception bumps the connector plugin API to 29.0. No cache owner framework, persisted ID format, or journal format changes. ### Release note Repair stale external-table lookup state after confirmed table/view absence while preserving refreshed or recreated tables and unrelated numeric lookups.
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # fe-common This module is used to store some common classes of other modules. # spark-dpp This module is Spark DPP program, used for Spark Load function. Depends: fe-common # fe-core This module is the main process module of FE. Depends: fe-common, spark-dpp