Skip to content

Add a generated supported services page for the common.ai provider - #72939

Draft
Lee-W wants to merge 1 commit into
apache:mainfrom
astronomer:common-ai-supported-services-page
Draft

Lee-W wants to merge 1 commit into
apache:mainfrom
astronomer:common-ai-supported-services-page

Conversation

@Lee-W

@Lee-W Lee-W commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

Why

Nothing told a user what common.ai actually reaches — the answer was spread across the toolsets guide, the index page, seven connection pages and provider.yaml, so the differentiating half (the Airflow hooks, MCP servers, SQL warehouses, DataFusion tables, sandboxes and vendor-managed agents a toolset gives you over a raw SDK call) was effectively unwritten.

The model list had also drifted: it was a hand-copied snapshot of nine vendors while the connection resolves anything pydantic_ai.providers can construct, so services we already support — Snowflake Cortex and OpenRouter among them — were absent from the page a customer searches.

What

  • provider.yaml becomes the one source the docs page reads, and — for connection types — the registry too.
  • pydanticai and mcp connection types get their external-services lists completed, and a new toolsets[].external-services key (schema entry plus a section in the provider-hook-migration contributing doc) records what each toolset module reaches.
  • provider-connection-services and provider-toolset-services, render those lists as tables on a new Supported services page, linked from the provider index
  • the LangChain connection page drops its hand-maintained vendor sentence and points at it instead, and toolsets.rst gains howto/toolset:<module> anchors so every row links to the toolset it names.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: [Claude] following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Lee-W
Lee-W force-pushed the common-ai-supported-services-page branch 3 times, most recently from a5292df to 5479098 Compare September 17, 2026 09:25
@Lee-W
Lee-W marked this pull request as ready for review September 17, 2026 09:26
@kaxil

kaxil commented Sep 17, 2026

Copy link
Copy Markdown
Member

@Lee-W Screenshots would be very helpful

@Lee-W
Lee-W force-pushed the common-ai-supported-services-page branch from 5479098 to 0595000 Compare September 18, 2026 08:00
@Lee-W

Lee-W commented Sep 18, 2026 •

Copy link
Copy Markdown
Member Author
01-supported-services-full
02-connections-table
03-toolsets-table
04-index-link
05-langchain-supported-providers

Comment thread contributing-docs/23_provider_hook_migration_to_yaml.rst Outdated
Comment thread devel-common/src/sphinx_exts/providers_extensions.py Outdated
Comment thread providers/common/ai/tests/unit/common/ai/test_provider_metadata.py Outdated
@Lee-W
Lee-W force-pushed the common-ai-supported-services-page branch from 0595000 to 9c161ee Compare September 19, 2026 01:27
@Lee-W
Lee-W marked this pull request as draft September 19, 2026 07:54
@Lee-W
Lee-W marked this pull request as ready for review September 19, 2026 07:56

@kaxil kaxil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I am concerned about is -- "toolsets" is the concept of Common AI provider and doesn't apply more generally to other providers. So calling it toolsets is weird.

I wonder if you can do something specific to only Common AI provider for now.. even if needs to be manually maintained (ideally not)

@Lee-W
Lee-W marked this pull request as draft September 20, 2026 23:34
Readers had no single place to see what this provider reaches. The
answer was spread across the toolsets guide, the index page, seven
connection pages and provider.yaml, so the differentiating half — the
Airflow hooks, MCP servers, SQL warehouses, DataFusion tables,
sandboxes and vendor-managed agents a toolset gives you over a raw SDK
call — was effectively unwritten.

The model list had also drifted. It was a hand-copied snapshot of nine
vendors while the connection actually resolves anything
pydantic_ai.providers can construct, so services we already support,
Snowflake Cortex and OpenRouter among them, were absent from the page a
customer searches.

Making provider.yaml the one source both the page and the registry read
keeps them from disagreeing, and a test that derives the reachable set
from pydantic-ai turns a drift into a red build naming the vendor
rather than a quiet omission.

That list is derived from a current pydantic-ai, while the provider
supports pydantic-ai-slim from 2.23.0 upward and CI exercises exactly
that floor. Snowflake Cortex and Crusoe reach back only to 2.27.0 and
2.28.0, so a vendor being absent downstream is a version difference
rather than drift. The test therefore asserts on every supported
version that an installed vendor is declared, and defers the opposite
direction to the release that carries all of them.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants