Files
doris/build-for-release.sh
T
Gabriel b68d1d5052 [refactor](external) Remove Hive ACID, Avro and LakeSoul readers on master (#69601)
### What problem does this PR solve?

Related PR: #69312

Port #69312 from `branch-4.2` to current `master`.

- Remove Hive ACID read transactions, shared read locks, write-ID
snapshots, delta-file listing and the BE transactional reader. Reject
full ACID and insert-only reads in the Hive connector.
- Adapt the changes to master's connector modules and unified reader
initialization interfaces. Preserve ordinary Hive scanning and
Iceberg/Paimon ORC position-delete and deletion-vector handling.
- Reject standalone Avro and deprecated table markers before file access
or metadata-only counts. Keep LakeSoul unsupported and remove its
obsolete test service and examples.
- Keep the retired `skip_checking_acid_version_file` name as a
compatibility no-op for client session initialization, using the
existing empty-string fallback for variable reads.
- Preserve Iceberg/Hudi Avro metadata dependencies. The Thrift changes
add deprecation comments only; every definition and numeric ID is
unchanged.

### Release note

Remove support for Hive transactional reads, including insert-only
tables, and standalone Avro file reads. LakeSoul remains unsupported.
Iceberg and Hudi Avro metadata parsing remains supported. The retired
`skip_checking_acid_version_file` session variable remains accepted as a
compatibility no-op.

### Validation

- Hive connector: all 490 tests passed; HMS client: all 155 tests
passed.
- 20 targeted FE core tests passed: deprecated-read rejection,
partition-path parsing, generic file scan planning and Hive streaming
scans.
- Review follow-up: reproduced the legacy ACID session-variable failure
before the fix; all 8 focused compatibility, variable-export and
unsupported-read tests passed after the fix.
- FE Checkstyle passed with zero violations for every affected FE
module.
- All 12 affected C++ source/header files passed clang-format 16; build
hygiene and shell syntax checks passed.
- Syntax-only compilation passed for all 6 affected BE production
translation units and all 4 affected BE test translation units. Isolated
checks relaxed pre-existing unused-constant/shadow warnings. Full BE
linking and runtime tests remain for CI.
- Verified that Thrift changes are comments only and checked the
affected definitions against branch-3.1/4.0/4.1/4.2. `git diff --check`
passed.

### Check List (For Author)

- Test
    - [x] Unit Test
- [x] Manual test: formatting, syntax, build hygiene and Thrift
compatibility checks described above.
- Behavior changed:
    - [x] Yes. Reject unsupported external reads explicitly.
- Does this need documentation?
    - [x] No. This ports the existing removal to master.
2026-10-27 10:29:08 +08:00

177 lines
3.8 KiB
Bash
Executable File

#!/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
##############################################################
# This script is used to build for Apache Doris Release
##############################################################
set -eo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
export DORIS_HOME="${ROOT}"
# Check args
usage() {
echo "
Usage: $0 --version version <options>
Optional options:
[no option] build with avx2
--noavx2 build without avx2
--tar pack the output
Eg.
$0 --version 1.2.0 build with avx2
$0 --noavx2 --version 1.2.0 build without avx2
$0 --version 1.2.0 --tar build with avx2 and pack the output
"
exit 1
}
if ! OPTS="$(getopt \
-n "$0" \
-o '' \
-l 'noavx2' \
-l 'tar' \
-l 'version:' \
-l 'help' \
-- "$@")"; then
usage
fi
eval set -- "${OPTS}"
_USE_AVX2=1
TAR=0
HELP=0
VERSION=
if [[ "$#" == 1 ]]; then
_USE_AVX2=1
else
while true; do
case "$1" in
--noavx2)
_USE_AVX2=0
shift
;;
--tar)
TAR=1
shift
;;
--version)
VERSION="$2"
shift 2
;;
--help)
HELP=1
shift
;;
--)
shift
break
;;
*)
echo "Internal error"
exit 1
;;
esac
done
fi
if [[ "${HELP}" -eq 1 ]]; then
usage
fi
if [[ -z ${VERSION} ]]; then
echo "Must specify version"
usage
fi
echo "Get params:
VERSION -- ${VERSION}
USE_AVX2 -- ${_USE_AVX2}
TAR -- ${TAR}
"
ARCH="$(uname -m)"
if [[ "${ARCH}" == "aarch64" ]]; then
ARCH="arm64"
elif [[ "${ARCH}" == "x86_64" ]]; then
ARCH="x64"
else
echo "Unknown arch: ${ARCH}"
exit 1
fi
echo "ARCH: ${ARCH}"
ORI_OUTPUT="${ROOT}/output"
rm -rf "${ORI_OUTPUT}"
FE="fe"
BE="be"
CLOUD="ms"
TOOLS="tools"
PACKAGE="apache-doris-${VERSION}-bin-${ARCH}"
if [[ "${_USE_AVX2}" == "0" ]]; then
PACKAGE="${PACKAGE}-noavx2"
fi
OUTPUT="${ORI_OUTPUT}/${PACKAGE}"
OUTPUT_FE="${OUTPUT}/${FE}"
OUTPUT_BE="${OUTPUT}/${BE}"
OUTPUT_CLOUD="${OUTPUT}/${CLOUD}"
OUTPUT_TOOLS="${OUTPUT}/${TOOLS}"
echo "Package Name:"
echo "FE: ${OUTPUT_FE}"
echo "BE: ${OUTPUT_BE}"
echo "CLOUD: ${OUTPUT_CLOUD}"
sh build.sh --clean &&
USE_AVX2="${_USE_AVX2}" sh build.sh &&
USE_AVX2="${_USE_AVX2}" sh build.sh --be --meta-tool
echo "Begin to pack"
rm -rf "${OUTPUT}"
mkdir -p "${OUTPUT_FE}" "${OUTPUT_BE}" "${OUTPUT_CLOUD}" "${OUTPUT_TOOLS}"
# FE
cp -R "${ORI_OUTPUT}"/fe/* "${OUTPUT_FE}"/
# BE
cp -R "${ORI_OUTPUT}"/be/* "${OUTPUT_BE}"/
# CLOUD
cp -R "${ORI_OUTPUT}"/ms/* "${OUTPUT_CLOUD}"/
if [[ "${TAR}" -eq 1 ]]; then
echo "Begin to compress"
cd "${ORI_OUTPUT}"
tar -cf - "${PACKAGE}" | xz -T0 -z - >"${PACKAGE}".tar.xz
cd -
fi
# TOOL
cp -R "${ORI_OUTPUT}"/tools/* "${OUTPUT_TOOLS}"/
echo "Output dir: ${OUTPUT}"
exit 0