mirror of
https://github.com/apache/airflow.git
synced 2026-11-04 07:12:48 +00:00
Wei Lee
8ee2365717
* Find providers in the registry by the external services they reach The names people type into registry search are often the service they want to talk to, not the Airflow distribution that talks to it -- "openai" or "ollama" rather than "common-ai". Those names were declared in provider.yaml and rendered in a table on the provider page, but neither search path read them, so the queries most likely to lead someone to Common AI returned nothing. Both search paths now read the same deduped list off the same field, so a query that works in the global search box also works in the /providers/ filter box. * Badge the external service that actually matched the search query The badge is meant to name the service that surfaced a provider, but it took the first entry the query appeared anywhere inside, so "ai" credited OpenAI and "llm" credited vLLM. It also compared raw text while the two other search paths collapse runs of "-_\s" first, so a hyphenated query like "mistral-ai" matched no service at all and got no badge. Matching a word start after the same collapse gives all three paths the same notion of a match. That normalize() now has a third hand-written copy, so the sync note in each of the three files names the other two. * Let the ai-ml explore category match a provider's external services The filter box on /providers/ already matched the services a provider's connections reach, but the category dropdown beside it did not, so a provider that gets to OpenAI only through external-services could be found by typing and not by filtering. Feeding those names to every category would misfile providers instead. A bare service name like "Google" collides with the cloud category, whose keywords are provider distribution ids -- an identity claim, not something a provider reaches. Only ai-ml keys on integration names, the same axis external services describe, so it is the one category that opts in and the default stays off. * Stop badging unrelated services on search hits that matched by name The service badge next to a search result exists to show which external service a provider was found through, when that is not obvious from the provider's own name. It was derived purely from the query string, with no check for whether the query already matched the provider's own identity: searching "ai" surfaced Common AI on its own name and then badged it "Mistral AI", and searching "a" badged it "Anthropic" — services the query had nothing to do with. Skipping the service lookup whenever the provider's own name or id already explains the hit removes those misleading badges without losing any of the genuine ones. * Replace the ai-ml opt-in comment's false rule with a measured fact The comment explaining why only ai-ml opts into external-service matching is the only written rationale for that flag anywhere, and it claimed every other category's keywords are this-distribution-only identity claims. That is false in both directions: dingtalk, dataflow, couchbase and twilio are not directory ids either, while 7 of ai-ml's own 13 keywords are. What replaces it is falsifiable: turning the flag on for every category, ai-ml included, changes exactly one other — cloud would gain Common AI via `google` matching "Google" and "Google Vertex AI". Common AI is not a cloud platform, so that is the false positive the opt-in stays scoped to avoid. * Document the external-service opt-in in the category matcher's README The registry README is the only place the explore-category matcher is explained, and it still described keywords as matching a provider's id and declared integration names only. Since a category can now opt in to also matching the provider's declared external services, anyone reading the README would have had no way to know the option exists, what it defaults to, or where it is declared. * Keep a provider out of the search text for services another provider owns Indexing every external service name a provider reaches gives short records an advantage that BM25's length normalisation turns into a ranking win over the provider that actually implements the service: Common AI's record reaches for "AWS Bedrock" and "Google Vertex AI" in a fraction of the words Amazon and Google spend describing their own. Compound service names whose words all belong to another provider's own integrations are now left out of the indexed text, while the badge and the /providers/ filter still see the full list. Bare vendor names stay indexed: dropping them on an exact-name match would have cost the discovery this PR adds, since a query like "ollama" or "groq" has no other provider to find.