metric-create: new skill for fact tables and fact metrics

[Berenger-Wooclap]

Opening this at @jdorn's suggestion on growthbook/growthbook-mcp#66 — that PR added create_fact_table / create_fact_metric as hand-written MCP tools, which #63 made the wrong shape. This is the same competence, remapped to a skill.

What it adds

metric-create — create a fact metric, and the fact table underneath it when one doesn't exist yet. Fourth write skill, after flag-create, experiment-launch, and experiment-stop.

It fills the gap metric-search currently points at: its read-only guardrail said to route metric creation to the GrowthBook UI. Now it routes here.

Covers all six metric types, row filters, aggregate filters, quantile settings, and the fact-table path (data source resolution → SQL → async column detection → metric).

Scope stops at the metric definition. Conversion windows, capping, priors, regression adjustment, risk thresholds, sample size, and MDE inherit the org defaults and are tuned in the UI. A ~25-field request body isn't something an agent fills in well, and those fields are exactly the ones where a wrong guess is invisible until the experiment reads wrong.

Guardrails came from the handlers, not the spec

Following the top-of-file rule in CLAUDE.md, every guardrail was checked against packages/back-end/src/api/fact-{metrics,tables}/post*.ts and packages/shared/src/validators/fact-{metrics,table}.ts. Three of them disagree with the OpenAPI descriptions, in the direction that matters — the docs describe constraints the API doesn't enforce:

Documented	Actual behavior
numerator.column "must be empty for proportion and dailyParticipation"	getCreateMetricPropsFromBody overwrites it — $$distinctUsers for proportion and retention, $$distinctDates for dailyParticipation — and clears aggregation for all three. A supplied column is silently discarded, not rejected.
denominator "only when metricType is 'ratio'"	Not enforced either way. A ratio metric with no denominator is accepted and creates a metric that can't compute; a denominator on a non-ratio metric is accepted and stored unused.
quantileSettings "mandatory if metricType is quantile"	.optional() in the schema. Omitting it yields quantileSettings: null rather than a 400.

I mention it because I originally implemented these as blocking validation in the MCP PR — which would have rejected requests GrowthBook accepts. Reading the handlers is what caught it. If you'd rather the descriptions moved toward the behavior, that's a separate change in the main repo and I'm happy to open it.

Two more worth having written down, also from the handlers:

• POST /fact-tables queues refreshFactTableColumns, so the create response's columns[] is empty. The skill re-fetches before defining a metric rather than guessing column names.
• userIdTypes is validated against datasource.settings.userIdTypes and fails with Invalid userIdType: <value>. The skill reads them from GET /api/v1/data-sources first instead of assuming user_id.

Files kept in sync

• skills/metric-create/SKILL.md + the scripts/gb-call symlink
• skills/metric-search — creation now routes here; added to Handoffs
• README.md — Product analytics table, the chaining list, the file tree, and the "What these skills do not do" bullet (narrowed to datasource creation and analysis tuning)
• CHANGELOG.md — under [Unreleased]

Structure follows the contract in CLAUDE.md: frontmatter with routing-oriented description, allowed-tools pinned to gb-call and nothing else, numbered workflow with literal bash and JSON, Guardrails, Endpoints used, Handoffs.

Happy to split the fact-table path into its own skill if you'd rather keep one responsibility per skill — I kept them together because creating a fact table is almost never the user's actual goal, it's a prerequisite they hit on the way to a metric.

🤖 Generated with Claude Code