mirror of
https://github.com/apache/doris.git
synced 2026-11-04 07:31:17 +00:00
HappenLee
f40c80dbf8
### What problem does this PR solve? Issue Number: None The existing SSB, TPCH and TPCDS tools require generating local data files and running separate creation, import, statistics and benchmark commands. Add one entry point per suite, backed by a shared runner: ```bash tools/ssb-tools/bin/run-ssb.sh -s 1 -d ssb_sf1 tools/tpch-tools/bin/run-tpch.sh -s 1 -d tpch_sf1 tools/tpcds-tools/bin/run-tpcds.sh -s 1 -d tpcds_sf1 ``` Rows are generated on demand through Doris's Trino connector and imported with `INSERT INTO ... SELECT ...`, without intermediate data files or a separate Trino server. SSB uses the included native-dbgen plugin. TPCH/TPCDS use complete upstream Trino 435 plugin distributions, pinned by SHA-256 and downloaded during operator installation. Plugins must be installed on every FE and BE before use. The runner executes the original scale-specific table DDL and query SQL: - SSB prepares its five base tables and supports the original `lineorder_flat` workflow through `--mode ssb|flat|both`. Both suites have 13 query files. - TPCH prepares its eight base tables and `revenue0` view and runs 22 query files. - TPCDS prepares its 24 base tables and runs 103 scale-specific query files, including the second statements of templates 14, 23, 24 and 39. Full statistics are collected synchronously before query timing. Every query runs three times with SQL and BE query result caches disabled. `--queries-only` repeats measurements against already prepared data. Each run preserves SQL results, errors, preparation logs, settings and CSV timings in a fresh directory. Parallel TPCH/TPCDS tables default to `max(10, SCALE)` generator splits: 10/100/1000/10000 for SF1/100/1000/10000. `--splits COUNT` overrides that count; invalid values fail before connecting. TPCH region/nation and TPCDS tables below the pinned generator's one-million-record parallelization threshold use one split, including with an override. Small tables share a single-split catalog; `--splits 1` reuses the main catalog. Per-table effective counts are logged. Preparation stderr is saved in `prepare.log`, while failures still print the log directory to the terminal. SSB preserves the old generator's ten lineorder partitions, enables its existing 64-bit path and fixes SSB random stream usage-counter bounds. Each SSB scan owns and closes its generator process. Imports explicitly project target column names, including TPCH's reordered lineitem columns and Trino 435's `p_response_targe` spelling for TPCDS promotion. TPCH uses standard prefixed column names and DECIMAL values. A fresh database prevents duplicate appends on retries. INSERTs use strict mode with zero filtered rows and must return a metadata `status` of `VISIBLE`. SQL text or labels containing `COMMITTED` cannot cause a false failure; missing or non-visible status stops preparation. Query errors stop the suite and are not recorded as successful timings.