Calvin Kirs
9614286a2d
### What problem does this PR solve?
`enable_all_http_auth` gates authentication and privilege checks on part
of the FE HTTP surface. It shipped **off**, so those endpoints answered
without checking credentials. This PR turns it **on by default on FE**,
makes it non-mutable, and fixes the bugs that made the flag unusable
when turned on.
**BE is deliberately not changed.** Its default stays `false`. BE 8040
is an internal port that operators are already required to keep off
end-user networks, and the compatibility cost of flipping it (metrics
scrapers point at it) outweighs the gain. Extending the flip to BE is a
separate decision.
#### What changes
1. **FE default flipped to `true`.** The FE `/api/**` and `/rest/v2/**`
REST surface now authenticates and enforces privileges in the shipped
configuration.
2. **The flag is no longer runtime-mutable.** `mutable = true` is
dropped from the `@ConfField`, so `ADMIN SET FRONTEND CONFIG
("enable_all_http_auth" = ...)` is rejected. Turning authentication off
requires editing `fe.conf` and restarting. Rationale: disabling
authentication should be a recorded, on-disk decision that survives a
restart, not a runtime command. It also means `fe.conf` is authoritative
when triaging a report — you can tell what a cluster was actually
running.
3. **Three bugs that only surface with the flag on.** These make the
recommended configuration unusable today, so they are fixed here:
- `BaseController.checkWithCookie()` never populated
`authInfo.userIdentity`. Every `checkAdminAuth(authInfo.userIdentity)`
built on its return value threw `NullPointerException`, taking out the
whole `/api/meta/**` path and with it the Doris-to-Doris external
catalog.
- `/api/get_small_file` demanded a user password on top of the cluster
token. BE (kafka certificates) and the CDC client (SSL CA) call it with
the token only, so both started downloading a 401 body instead of the
file — surfacing as `Small file MD5 mismatch`. The token is the
credential for this endpoint, the same way an auth token is on the BE
HTTP interfaces.
- `/api/bootstrap` had the same problem for an FE joining the cluster,
which presents cluster id + token rather than a password.
4. **Security documentation.** `threat-model.md` moves FE HTTP
*authentication* from a disclaimed property to a default-config one:
unauthenticated access to FE 8030 is now a valid finding. Authorization
is documented as a separate, narrower property — this change adds no
centralized privilege check, and the password-only handlers that remain
(`AddStoragePolicyAction`, `ESCatalogAction`, `ImportAction`,
`StatisticAction`, `query_schema`) are recorded as a known, tracked gap
rather than implied to be covered. BE 8040, FE `/metrics`, FE
`/api/health` and the cluster-token endpoints stay explicitly
disclaimed, with the asymmetry spelled out so triagers do not over-apply
the change. `SECURITY.md` states the security-testing baseline:
assessments of the FE HTTP surface must run with the flag on, verified
against the *effective* config (`fe_custom.conf` is read after `fe.conf`
and overwrites it).
5. **Regression tests.** No change to the shared `curl()` /
`http_client()` helpers — endpoints that need credentials get them at
their own call sites, so the helpers keep their anonymous mode and never
carry the JDBC password to non-FE destinations. The auth suites drop
their `ADMIN SET ... CONFIG` toggles, which the non-mutable config no
longer accepts; `test_http_api_auth` exercises the disabled path with a
second docker cluster started with the flag off in `fe.conf`, and both
clusters assert against an endpoint whose behaviour actually depends on
the flag. New `auth_p0/test_http_legacy_meta_auth` gives the legacy
`/api/meta/**` controller its first regression coverage (the existing
`test_http_meta_*` suites exercise `/rest/v2/api/meta/**`, a different
class).
### Release note
FE now enables `enable_all_http_auth` by default, so the FE `/api/**`
and `/rest/v2/**` HTTP surface requires credentials and enforces the
caller's privileges. Callers that previously polled those endpoints
anonymously (health probes, ops scripts, management tooling) will
receive 401 until they present credentials. FE `/metrics` is unaffected
— it is public by design and not gated by this flag — and the BE default
is unchanged. The flag is no longer runtime-mutable: to fall back during
a migration, set `enable_all_http_auth = false` in `fe.conf` and
restart. That is a temporary aid, not a supported steady state.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [ ] No.
- [x] Yes. FE HTTP endpoints that previously answered anonymously now
require credentials; `enable_all_http_auth` can no longer be changed
with `ADMIN SET`.
- Does this need documentation?
- [ ] No.
- [x] Yes. Upgrade guide and the `enable_all_http_auth` config reference
need updating in apache/doris-website.
1.8 KiB
Security Policy
Apache Doris security findings should be reported to
security@apache.org. The Apache Security Team will route reports to
the Doris project maintainers.
For security scope, trust boundaries, attacker roles, explicit
non-goals, and vulnerability triage classification, use
threat-model.md as the canonical source for this repository. Security
scanners, review agents, and vulnerability triagers should read
threat-model.md before classifying findings.
Findings that are out of model or by design under threat-model.md
should be reported with that disposition instead of being treated as
Doris vulnerabilities.
Before testing the FE HTTP surface, confirm enable_all_http_auth is
true on the running FE — it is the default, and it must not be
turned off for the test run. Confirm it against the effective
runtime value, not against fe.conf alone: FE reads fe.conf and
then ${custom_config_dir}/fe_custom.conf, and the latter overwrites
the former. In releases where this flag was mutable, ADMIN SET FRONTEND CONFIG (...) PROPERTIES("persist" = "true") could write
enable_all_http_auth=false into fe_custom.conf; making the flag
non-mutable does not remove or migrate such a value, so an upgraded
cluster can have no false entry anywhere in fe.conf and still be
running with authentication off. Read the value back from
/api/show_config (or ADMIN SHOW FRONTEND CONFIG) and check both
files. With it off, FE serves part of its HTTP surface (metadata,
statistics and import REST endpoints) without checking credentials,
and every finding that depends on it being off is out of model. On BE
the same flag still defaults to false; BE 8040 is an internal port
that operators are required to keep off end-user networks, so findings
there are disclaimed rather than valid. See the security-testing
baseline in §4.5a of threat-model.md.