mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
Gabriel
707c39170b
### What problem does this PR solve?
A `VARBINARY(16)` column becomes an unspecified-length `VARBINARY` after
a Block serialization round trip. Iceberg FIXED/UUID writers then reject
it with `Iceberg fixed width does not match Doris VARBINARY length:
expected 16, got -1`, including when inserting omitted columns with
write defaults.
`PTypeDesc` already preserves the declared length, but Blocks use
`PColumnMeta`, which had no length field. The receiver consequently
reconstructed every VARBINARY column with the default length of -1.
Add an optional `varbinary_length` field to `PColumnMeta`, serialize it
from `DataTypeVarbinary`, and restore it in `DataTypeFactory`. The field
defaults to -1 so metadata from older senders retains its existing
behavior. Nested and nullable types reuse the same metadata path.
Iceberg width validation remains unchanged.
### Release note
Preserve VARBINARY declared lengths across Block serialization to
prevent Iceberg FIXED/UUID write failures.
### Check List (For Author)
- Test
- [x] Unit Test
- Behavior changed:
- [x] Yes. Serialized Blocks retain VARBINARY lengths, including nested
fields.
- Does this need documentation?
- [x] No.
### Validation
New tests cover:
- Full compressed Block and protobuf wire round trips for lengths -1,
16, and 32, with nullable and non-nullable columns, NULL values, and
embedded zero bytes.
- Nested array/map/struct metadata retaining independent binary lengths.
- Legacy metadata without the new field restoring length -1.
Before the fix, the Block and nested metadata tests fail with lengths
restored as -1; the legacy test passes.
After the fix, all 17 `DataTypeVarbinaryTest.*` tests pass in an ASAN
test binary. The focused build regenerated `data.proto` and rebuilt the
affected type/Block implementations and test source, reusing existing
dependency objects.
Also verified old/new protobuf wire compatibility and passed
clang-format 16 checks on all changed C++ files.
Full distributed Iceberg regression remains for CI. Old senders still
omit lengths, so the fix does not recover information already lost by an
older BE.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label