mirror of
https://github.com/apache/doris.git
synced 2026-11-04 16:03:23 +00:00
Calvin Kirs
5e1c5427b3
## Proposed changes The `apache_hdfs_broker` daemon under `fs_brokers/` has not been modified for nearly two years, and users have migrated to the built-in S3 / HDFS TVF, so the standalone broker component is no longer needed. The in-tree SDKs under `sdk/` (`go-doris-sdk`, `java-doris-sdk`) are now maintained out-of-tree in a separate `doris-sdk` repo and no longer need to live here. ### Removed - `fs_brokers/apache_hdfs_broker/` — broker daemon module - `sdk/` — `go-doris-sdk` + `java-doris-sdk` - `docker/runtime/broker/` — broker daemon docker image, and the `hdfs-broker-doris` compose demo ### Build / CI / docker references cleaned up `build.sh` (`--broker` / `BUILD_BROKER`), `build-for-release.sh`, `.github/workflows/build-extension.yml` (`build-broker` job), `.github/workflows/code-checks.yml`, `.gitignore`, `docker/runtime/docker-build.sh`, `docker/runtime/all-in-one/Dockerfile` + `entry_point.sh`, `docker/runtime/README.md`, and `threat-model.md`. ### Kept (intentionally out of scope) - `fs_brokers/cdc_client/` (not a broker; still built into BE) - All FE/BE broker **functionality**: `BrokerMgr`, `BrokerDesc`, broker load, `fe-filesystem-broker`, the BE filesystem broker module, and `PaloBrokerService.thrift` This PR only removes the standalone broker daemon component and the in-tree SDK sources; the broker load/storage feature in FE/BE is unchanged, so there is no metadata/edit-log compatibility impact.
5.1 KiB
5.1 KiB
Building the Docker image
Download and copy the Doris code repo
clone Doris code repo and enter the doris/docker/runtime path. your workspace should like this:
.
├── runtime
│ ├── README.md
│ ├── all-in-one
│ ├── base-image
│ ├── be
│ ├── broker
│ ├── docker-compose-demo
│ ├── doris-compose
│ ├── fe
│ ├── ms
Download, extract archive and copy into resource directories
- Go to the doris official website to download the binary package you need(pay attention to selecting the doris version and architecture you need), and extract archive binary
Here we take the build of x64 (avx2) and arm64 platforms of 3.0.2 version as an example.
$ wget https://apache-doris-releases.oss-accelerate.aliyuncs.com/apache-doris-3.0.2-bin-x64.tar.gz && tar -zxvf apache-doris-3.0.2-bin-x64.tar.gz
# or
$ wget https://apache-doris-releases.oss-accelerate.aliyuncs.com/apache-doris-3.0.2-bin-arm64.tar.gz && tar -zxvf apache-doris-3.0.2-bin-arm64.tar.gz
- You need to copy the corresponding directories to different directories in sequence as below in the table:
amd64(avx2) platform:
| doris type | doris package | docker file path |
|---|---|---|
| fe | apache-doris-3.0.2-bin-x64/fe | runtime/fe/resource/amd64/apache-doris-3.0.2-bin-x64/fe |
| be | apache-doris-3.0.2-bin-x64/be | runtime/be/resource/amd64/apache-doris-3.0.2-bin-x64/be |
| ms | apache-doris-3.0.2-bin-x64/ms | runtime/ms/resource/amd64/apache-doris-3.0.2-bin-x64/ms |
arm64 platform:
| doris type | doris package | docker file path |
|---|---|---|
| fe | apache-doris-3.0.2-bin-arm64/fe | runtime/fe/resource/arm64/apache-doris-3.0.2-bin-arm64/fe |
| be | apache-doris-3.0.2-bin-arm64/be | runtime/be/resource/arm64/apache-doris-3.0.2-bin-arm64/be |
| ms | apache-doris-3.0.2-bin-arm64/ms | runtime/ms/resource/arm64/apache-doris-3.0.2-bin-arm64/ms |
NOTICE
Only after doris 3.0, the storage-computing separation mode requires the ms (meta-service) component.
Build base image
NOTICE
The below images depend on the base image apache/doris:base-latest. If your environment cannot access it, you can pre-build the base image. The base image contains the basic environment for doris to run, including JDK, openssl, etc.
- As mentioned in the preparation steps above, the Dockerfile of the Base image is under the runtime/base-image path. To build it, make sure you have pulled the Doris code repo and execute the following command.
$ cd doris/runtime/base-image && docker build . -t doris-base:latest -f Dockerfile_base
- Adjust the base image name used by the Dockerfile of the doris component and replace the base image you built yourself, As shown in the following example:
# Adjust the base image here
FROM doris-base:latest:latest
ARG TARGETARCH
ARG DORIS_VERSION="x.x.x"
Build Doris docker image
as the following commands, Docker will automatically confirm the architecture
$ cd doris/runtime/fe && docker build . -t doris.fe:3.0.2 -f Dockerfile --build-arg DORIS_VERSION=3.0.2
$ cd doris/runtime/be && docker build . -t doris.be:3.0.2 -f Dockerfile --build-arg DORIS_VERSION=3.0.2
$ cd doris/runtime/ms && docker build . -t doris.ms:3.0.2 -f Dockerfile --build-arg DORIS_VERSION=3.0.2
Latest update time
2025-02-12