mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
Gabriel
28cbcfc233
### What problem does this PR solve?
The third-party Arrow build leaves `PARQUET_REQUIRE_ENCRYPTION` at its
default `OFF`. Parquet still installs encryption APIs and exports some
encryption symbols, but its archive contains
`encryption_internal_nossl.cc.o`. Reading a standard PME-encrypted
Parquet file with the correct key then throws:
```text
Calling encryption method in Arrow/Parquet built without OpenSSL
```
Linking `libssl.a` and `libcrypto.a` into the consumer does not replace
those compiled stubs. This change supplies the build prerequisites for
native encrypted Iceberg reads.
The change:
- Enables `PARQUET_REQUIRE_ENCRYPTION` and selects static OpenSSL.
- Pins OpenSSL headers and both archives to the Doris third-party
prefix, and fails before CMake if any required input is missing.
- Installs `encryption_internal.h` and `internal_file_decryptor.h` from
the same pinned Arrow source. Arrow normally excludes these internal
headers, but module-level decryption is needed to retain the native
Parquet scan path.
Existing third-party prebuilt packages must be rebuilt to pick up these
changes. SQL-level encrypted Iceberg read integration remains separate
work.
### Release note
None
### Check List (For Author)
- Test:
- [x] Manual test (details below)
- Behavior changed:
- [x] Yes. Third-party Parquet is built with PME enabled; missing owned
OpenSSL inputs fail before configure.
- Does this need documentation?
- [x] No. This is the third-party build prerequisite, with no new
SQL-facing configuration.
#### Validation
On Linux x86_64, extracted Arrow 24.0.0 from the archive pinned by
`thirdparty/vars.sh`, applied the current Doris Arrow patches, and
executed the changed `build_arrow()` function in an isolated prefix
seeded with existing third-party dependencies. The complete Arrow build
and install succeeded. CMake selected PME `ON` and OpenSSL 1.1.1s from
that prefix. The rebuilt archive contains `encryption_internal.cc.o` /
`openssl_internal.cc.o`, with no `nossl` object.
A standalone C++ reader linked directly to the checked/rebuilt static
archives, including the same prefix's OpenSSL archives, used the public
Apache [Parquet test
fixtures](https://github.com/apache/parquet-testing/tree/master/data):
| Check | Before: checked PME-disabled package | After: rebuilt package
|
| --- | --- | --- |
| `uniform_encryption.parquet.encrypted`, correct public test key |
OpenSSL-disabled runtime error | All-column scan succeeds: 50 rows |
| Same file, wrong key | OpenSSL-disabled runtime error | Authentication
failure: `Failed decryption finalization` |
| Same file, no key | Missing decryption-properties error | Missing
decryption-properties error |
| `alltypes_plain.parquet` | All-column scan succeeds: 8 rows |
All-column scan succeeds: 8 rows |
Additional checks:
- Module-level AES-GCM round trip and rejection of a different data-page
AAD pass.
- Both installed internal headers match the source and compile/link with
`InternalFileDecryptor` and the module decryptors.
- Removing each required OpenSSL input rejects the build before CMake.
The baseline function reaches CMake for the same incomplete inputs.
- Captured Linux and Darwin invocations contain the PME option and
explicit static OpenSSL paths; this is an argument check, not a macOS
build.
- `bash -n thirdparty/build-thirdparty.sh` and `git diff --check` pass.
Validation limits: no full Doris BE build, Doris SQL test, AArch64 build
or native macOS build was run. No runtime reader behavior is enabled by
this PR alone.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label