### What problem does this PR solve? Problem Summary: This PR adds Iceberg v3 support in Doris. Main changes: - support hidden row lineage columns `_row_id` and `_last_updated_sequence_number` - support querying and filtering on row lineage columns - forbid explicitly writing row lineage columns in INSERT - support Iceberg v3 deletion vectors in DELETE/MERGE - support rewrite_data_files for Iceberg v3 tables - improve v2-to-v3 compatibility for query and DML paths - add FE/BE/unit/regression tests for Parquet and ORC ### Release note Support Iceberg v3 row lineage and deletion vectors in Doris.
Doris Develop Environment based on docker
Preparation
-
Download the Doris code repo
$ cd /to/your/workspace/ $ git clone https://github.com/apache/doris.git $ cd doris $ git submodule update --init --recursiveYou can remove the
.gitdir indoris/to make the dir size smaller. So that the following generated docker image can be smaller. -
Copy Dockerfile
$ cd /to/your/workspace/ $ cp doris/docker/Dockerfile ./
After preparation, your workspace should like this:
.
├── Dockerfile
├── doris
│ ├── be
│ ├── bin
│ ├── build.sh
│ ├── conf
│ ├── DISCLAIMER-WIP
│ ├── docker
│ ├── docs
│ ├── env.sh
│ ├── fe
│ ├── ...
Build docker image
$ cd /to/your/workspace/
$ docker build -t doris:v1.0 .
dorisis docker image repository name andv1.0is tag name, you can change them to whatever you like.
Use docker image
This docker image you just built does not contain Doris source code repo. You need to download it first and map it to the container. (You can just use the one you used to build this image before)
$ docker run -it -v /your/local/path/doris/:/root/doris/ doris:v1.0
$ docker run -it -v /your/local/.m2:/root/.m2 -v /your/local/doris-DORIS-x.x.x-release/:/root/doris-DORIS-x.x.x-release/ doris:v1.0
Then you can build source code inside the container.
$ cd /root/doris/
$ sh build.sh
NOTICE
The default JDK version is openjdk 11, if you want to use openjdk 8, you can run the command:
$ alternatives --set java java-1.8.0-openjdk.x86_64
$ alternatives --set javac java-1.8.0-openjdk.x86_64
$ export JAVA_HOME=/usr/lib/jvm/java-1.8.0
The version of jdk you used to run FE must be the same version you used to compile FE.
Latest update time
2022-1-23