mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
### What problem does this PR solve? Related PR: #69312 Port #69312 from `branch-4.2` to current `master`. - Remove Hive ACID read transactions, shared read locks, write-ID snapshots, delta-file listing and the BE transactional reader. Reject full ACID and insert-only reads in the Hive connector. - Adapt the changes to master's connector modules and unified reader initialization interfaces. Preserve ordinary Hive scanning and Iceberg/Paimon ORC position-delete and deletion-vector handling. - Reject standalone Avro and deprecated table markers before file access or metadata-only counts. Keep LakeSoul unsupported and remove its obsolete test service and examples. - Keep the retired `skip_checking_acid_version_file` name as a compatibility no-op for client session initialization, using the existing empty-string fallback for variable reads. - Preserve Iceberg/Hudi Avro metadata dependencies. The Thrift changes add deprecation comments only; every definition and numeric ID is unchanged. ### Release note Remove support for Hive transactional reads, including insert-only tables, and standalone Avro file reads. LakeSoul remains unsupported. Iceberg and Hudi Avro metadata parsing remains supported. The retired `skip_checking_acid_version_file` session variable remains accepted as a compatibility no-op. ### Validation - Hive connector: all 490 tests passed; HMS client: all 155 tests passed. - 20 targeted FE core tests passed: deprecated-read rejection, partition-path parsing, generic file scan planning and Hive streaming scans. - Review follow-up: reproduced the legacy ACID session-variable failure before the fix; all 8 focused compatibility, variable-export and unsupported-read tests passed after the fix. - FE Checkstyle passed with zero violations for every affected FE module. - All 12 affected C++ source/header files passed clang-format 16; build hygiene and shell syntax checks passed. - Syntax-only compilation passed for all 6 affected BE production translation units and all 4 affected BE test translation units. Isolated checks relaxed pre-existing unused-constant/shadow warnings. Full BE linking and runtime tests remain for CI. - Verified that Thrift changes are comments only and checked the affected definitions against branch-3.1/4.0/4.1/4.2. `git diff --check` passed. ### Check List (For Author) - Test - [x] Unit Test - [x] Manual test: formatting, syntax, build hygiene and Thrift compatibility checks described above. - Behavior changed: - [x] Yes. Reject unsupported external reads explicitly. - Does this need documentation? - [x] No. This ports the existing removal to master.
177 lines
3.8 KiB
Bash
Executable File
177 lines
3.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
# or more contributor license agreements. See the NOTICE file
|
|
# distributed with this work for additional information
|
|
# regarding copyright ownership. The ASF licenses this file
|
|
# to you under the Apache License, Version 2.0 (the
|
|
# "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing,
|
|
# software distributed under the License is distributed on an
|
|
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
# KIND, either express or implied. See the License for the
|
|
# specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
##############################################################
|
|
# This script is used to build for Apache Doris Release
|
|
##############################################################
|
|
|
|
set -eo pipefail
|
|
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
|
|
|
|
export DORIS_HOME="${ROOT}"
|
|
|
|
# Check args
|
|
usage() {
|
|
echo "
|
|
Usage: $0 --version version <options>
|
|
Optional options:
|
|
[no option] build with avx2
|
|
--noavx2 build without avx2
|
|
--tar pack the output
|
|
|
|
Eg.
|
|
$0 --version 1.2.0 build with avx2
|
|
$0 --noavx2 --version 1.2.0 build without avx2
|
|
$0 --version 1.2.0 --tar build with avx2 and pack the output
|
|
"
|
|
exit 1
|
|
}
|
|
|
|
if ! OPTS="$(getopt \
|
|
-n "$0" \
|
|
-o '' \
|
|
-l 'noavx2' \
|
|
-l 'tar' \
|
|
-l 'version:' \
|
|
-l 'help' \
|
|
-- "$@")"; then
|
|
usage
|
|
fi
|
|
|
|
eval set -- "${OPTS}"
|
|
|
|
_USE_AVX2=1
|
|
TAR=0
|
|
HELP=0
|
|
VERSION=
|
|
if [[ "$#" == 1 ]]; then
|
|
_USE_AVX2=1
|
|
else
|
|
while true; do
|
|
case "$1" in
|
|
--noavx2)
|
|
_USE_AVX2=0
|
|
shift
|
|
;;
|
|
--tar)
|
|
TAR=1
|
|
shift
|
|
;;
|
|
--version)
|
|
VERSION="$2"
|
|
shift 2
|
|
;;
|
|
--help)
|
|
HELP=1
|
|
shift
|
|
;;
|
|
--)
|
|
shift
|
|
break
|
|
;;
|
|
*)
|
|
echo "Internal error"
|
|
exit 1
|
|
;;
|
|
esac
|
|
done
|
|
fi
|
|
|
|
if [[ "${HELP}" -eq 1 ]]; then
|
|
usage
|
|
fi
|
|
|
|
if [[ -z ${VERSION} ]]; then
|
|
echo "Must specify version"
|
|
usage
|
|
fi
|
|
|
|
echo "Get params:
|
|
VERSION -- ${VERSION}
|
|
USE_AVX2 -- ${_USE_AVX2}
|
|
TAR -- ${TAR}
|
|
"
|
|
|
|
ARCH="$(uname -m)"
|
|
|
|
if [[ "${ARCH}" == "aarch64" ]]; then
|
|
ARCH="arm64"
|
|
elif [[ "${ARCH}" == "x86_64" ]]; then
|
|
ARCH="x64"
|
|
else
|
|
echo "Unknown arch: ${ARCH}"
|
|
exit 1
|
|
fi
|
|
|
|
echo "ARCH: ${ARCH}"
|
|
|
|
ORI_OUTPUT="${ROOT}/output"
|
|
rm -rf "${ORI_OUTPUT}"
|
|
|
|
FE="fe"
|
|
BE="be"
|
|
CLOUD="ms"
|
|
TOOLS="tools"
|
|
PACKAGE="apache-doris-${VERSION}-bin-${ARCH}"
|
|
|
|
if [[ "${_USE_AVX2}" == "0" ]]; then
|
|
PACKAGE="${PACKAGE}-noavx2"
|
|
fi
|
|
|
|
OUTPUT="${ORI_OUTPUT}/${PACKAGE}"
|
|
OUTPUT_FE="${OUTPUT}/${FE}"
|
|
OUTPUT_BE="${OUTPUT}/${BE}"
|
|
OUTPUT_CLOUD="${OUTPUT}/${CLOUD}"
|
|
OUTPUT_TOOLS="${OUTPUT}/${TOOLS}"
|
|
|
|
echo "Package Name:"
|
|
echo "FE: ${OUTPUT_FE}"
|
|
echo "BE: ${OUTPUT_BE}"
|
|
echo "CLOUD: ${OUTPUT_CLOUD}"
|
|
|
|
sh build.sh --clean &&
|
|
USE_AVX2="${_USE_AVX2}" sh build.sh &&
|
|
USE_AVX2="${_USE_AVX2}" sh build.sh --be --meta-tool
|
|
|
|
echo "Begin to pack"
|
|
rm -rf "${OUTPUT}"
|
|
mkdir -p "${OUTPUT_FE}" "${OUTPUT_BE}" "${OUTPUT_CLOUD}" "${OUTPUT_TOOLS}"
|
|
|
|
# FE
|
|
cp -R "${ORI_OUTPUT}"/fe/* "${OUTPUT_FE}"/
|
|
|
|
# BE
|
|
cp -R "${ORI_OUTPUT}"/be/* "${OUTPUT_BE}"/
|
|
|
|
# CLOUD
|
|
cp -R "${ORI_OUTPUT}"/ms/* "${OUTPUT_CLOUD}"/
|
|
|
|
if [[ "${TAR}" -eq 1 ]]; then
|
|
echo "Begin to compress"
|
|
cd "${ORI_OUTPUT}"
|
|
tar -cf - "${PACKAGE}" | xz -T0 -z - >"${PACKAGE}".tar.xz
|
|
cd -
|
|
fi
|
|
|
|
# TOOL
|
|
cp -R "${ORI_OUTPUT}"/tools/* "${OUTPUT_TOOLS}"/
|
|
|
|
echo "Output dir: ${OUTPUT}"
|
|
exit 0
|