Skip to content

Latest commit

 

History

History
355 lines (280 loc) · 15.1 KB

File metadata and controls

355 lines (280 loc) · 15.1 KB

Schema Namespaces

Table of Contents

SchemaBot uses namespaces to organize declarative schema files. A namespace maps to the database-specific grouping concept:

Database Namespace maps to How to list
MySQL Schema name SHOW DATABASES
Vitess Keyspace SHOW KEYSPACES
PostgreSQL Schema \dn

Schema Directory Structure

The schema directory is the source of truth. Each subdirectory is a namespace containing SQL files and optional configuration.

MySQL — Single schema name

The simplest case. One database, one schema name:

myapp/schema/
├── schemabot.yaml
└── testapp/
    ├── users.sql
    ├── orders.sql
    └── products.sql
# schemabot.yaml
database: testapp
type: mysql

MySQL — Multiple schema names on the same database

A single MySQL database server can have multiple schema names (SHOW DATABASES). Each subdirectory is a namespace, all managed under one schemabot.yaml:

myapp/schema/
├── schemabot.yaml
├── app_primary/
│   ├── users.sql
│   └── sessions.sql
└── app_analytics/
    ├── events.sql
    └── metrics.sql
# schemabot.yaml
database: myapp
type: mysql

SchemaBot plans each namespace independently — app_primary and app_analytics each produce their own SchemaChange with separate table changes.

MySQL — Different databases entirely

When an app talks to separate database servers, each gets its own schema directory with its own schemabot.yaml:

myapp/
├── primary-schema/
│   ├── schemabot.yaml          # database: app_primary, type: mysql
│   └── app_primary/
│       ├── users.sql
│       └── sessions.sql
└── analytics-schema/
    ├── schemabot.yaml          # database: app_analytics, type: mysql
    └── app_analytics/
        ├── events.sql
        └── metrics.sql

These are completely independent SchemaBot configurations — different targets, different credentials, different plans.

Vitess — Multiple keyspaces

Vitess databases have multiple keyspaces under a single schemabot.yaml. Each keyspace subdirectory is a namespace:

myapp/schema/
├── schemabot.yaml
├── commerce/
│   ├── orders_seq.sql
│   ├── products_seq.sql
│   └── vschema.json
└── commerce_sharded/
    ├── orders.sql
    ├── products.sql
    └── vschema.json
# schemabot.yaml
database: commerce
type: vitess

SchemaBot plans all keyspaces together — a single plan can contain changes across commerce and commerce_sharded. This is necessary because DDL and VSchema changes across keyspaces may need to be deployed atomically (e.g., moving a table between keyspaces).

Vitess — VSchema changes

vschema.json controls Vitess routing (which tables live in which keyspace, sharding strategy, vindexes, etc.). When a vschema.json file changes:

  1. Plan detects the diff and includes it as a FileChange for display
  2. Apply sends the full vschema.json content to the branch (not the diff)
  3. The PlanetScale engine applies both DDL and VSchema atomically

A plan can have DDL-only changes, VSchema-only changes, or both.

A VSchema change that removes a vindex, a table routing entry, or a table's column-vindex association is an unsafe change and requires the same --allow-unsafe acknowledgment as destructive DDL — see lint-and-safety-levels.md.

Where to Put the Schema Directory

SchemaBot is location-agnostic: config discovery finds schemabot.yaml anywhere in the repository — the directory containing it is the schema directory. Namespaces come from directory names in either layout: with the config at the schema root, each subdirectory is a namespace (as below); with the config inside a single namespace directory next to its .sql files (flat layout), that directory's basename is the namespace. Nothing in SchemaBot forces a particular path, so placement is a repository-semantics choice. For greenfield repositories, a repository-root schema/ directory is the recommended layout:

myapp/
├── schema/
│   ├── schemabot.yaml
│   └── myapp/
│       ├── users.sql
│       └── orders.sql
├── service-a/
└── service-b/
  • Prefer to keep schema files out of build-tool resource directories (Maven/Gradle src/main/resources, embedded asset dirs). Those paths mean "packaged into the application artifact" — but the application never reads these files. Only SchemaBot does, from GitHub, at PR time. Placing them there ships dead weight in every build and implies runtime semantics the files don't have.
  • Keep visible distance from imperative tools during coexistence. If the repository still carries Flyway or Liquibase versioned change scripts while SchemaBot takes over, keep the two mechanisms in visibly different places. Those scripts live in resources/ because the app executes them from the classpath at startup; SchemaBot is the opposite model — declarative desired state, applied out-of-band. Side by side in one resources tree, "which file do I edit?" becomes a coin flip.
  • The schema is a repository-level concern. The database usually spans application modules (domain tables, framework tables such as Spring Batch's BATCH_*, history tables), so its desired state belongs at the top level, not nested inside one module.
  • Short stable paths keep operations simple. A server-side allowed_dirs: [schema] allowlist (matched against the directory containing schemabot.yaml and its descendants), CODEOWNERS scoping, and PR review path filters all stay trivial with a root path, and the directory-name-is-the-namespace contract stays visible at a glance.

A module-root directory outside src/ (e.g. my-data-module/schema/) also works and avoids the packaging problem — reasonable when one module clearly owns the database, but most schemas span modules.

For existing (brownfield) repositories, treat the root schema/ layout as a nice-to-have, not a requirement. SchemaBot behaves identically wherever the config lives, so adopting it does not require moving files that already have an established home. Relocating is worth it only if the points above start to bite — packaging dead weight, confusion next to imperative change scripts, or allowlist/CODEOWNERS churn — and can be done later as a follow-up.

$ENV Substitution in Namespace Names

Some infrastructure names schemas with an environment suffix: bikeshare_staging in staging, bikeshare_production in production. Rather than maintaining separate directories for each environment, you can use $ENV in the directory name. When an environment is specified (via -e), $ENV is replaced with the environment value.

Example

Directory structure:

myapp/schema/
├── schemabot.yaml
└── bikeshare_$ENV/
    ├── bikes.sql
    └── stations.sql

Running against different environments resolves the namespace accordingly:

# Namespace becomes "bikeshare_staging"
schemabot plan -s myapp/schema -e staging

# Namespace becomes "bikeshare_production"
schemabot plan -s myapp/schema -e production

Rules

  • $ENV is replaced with the environment value from -e (e.g., staging, production).
  • If no environment is specified, $ENV is left as-is (no substitution).
  • Works in both flat layout (directory name = namespace) and subdirectory layout (subdirectory names = namespaces).
  • You can mix $ENV directories with regular directories in the subdirectory layout.
  • When creating the directory from a shell, quote the name to prevent shell expansion: mkdir 'bikeshare_$ENV'

Ignoring Namespaces

Some schema roots contain a namespace directory that should never be reconciled against a live database. A common example is a Vitess keyspace that exists only in local test infrastructure: the repository carries its schema files so local tooling can create the keyspace, but no real environment has it (or the environment keeps it intentionally empty). Without an exclusion, every plan would propose creating those tables.

List such namespaces under ignore_namespaces in schemabot.yaml:

myapp/schema/
├── schemabot.yaml
├── commerce/
│   ├── orders.sql
│   └── vschema.json
└── commerce_test/          ← ignored: never planned or applied
    ├── fixtures.sql
    └── vschema.json
# schemabot.yaml
database: commerce
type: vitess
ignore_namespaces:
  - commerce_test

Rules

  • Entries are bare namespace names, not paths. An entry containing / or \ (e.g., schema/commerce_test) is rejected when the config is loaded.
  • Ignored namespaces are excluded from plans, applies, and merge-gate checks. This applies to both the GitHub PR flow and the CLI (schemabot plan / schemabot apply read the same schemabot.yaml).
  • $ENV substitution applies to entries the same way it applies to directory names: fixtures_$ENV ignores the fixtures_staging namespace when planning for staging.
  • Matching is exact and case-sensitive. An entry that matches no namespace directory excludes nothing; the plan proceeds and the unmatched entry is reported (a CLI warning, a server-side log) so a typo or stale entry is visible.
  • Ignoring every namespace in the schema root is an error: the plan fails rather than reconciling an empty desired state.
  • Ignoring a namespace does not exempt the directory from layout validation; a schema root mixing flat files and subdirectories is still rejected.

Exclusions are disclosed

Every plan that excluded namespaces says so: the PR plan comment renders an ℹ️ Namespaces excluded from this plan by ignore_namespaces: … line under the plan summary (also on "no changes" results, so a withheld namespace is distinguishable from an unchanged one), and the CLI prints the same disclosure for plan and apply. When reviewing a PR that introduces an ignore_namespaces entry, the disclosure plus the config diff is the review surface: the plan stops reconciling that namespace from this PR onward.

MySQL target DSN requirements

For MySQL targets, ignore_namespaces requires a namespace-free target DSN (one that does not name a database), where each namespace directory is diffed against its own database. A DSN that already names a database diffs the whole database as one unit: an ignored namespace's live tables would have no declaring files and the diff would plan them as DROP TABLE, the inverse of "ignore". SchemaBot refuses this combination, and the plan fails with an error asking for a namespace-free DSN or removal of ignore_namespaces.

Per-Target Schema Overrides

$ENV substitution handles physical schema names that vary by environment. When names vary by deployment within one environment — several regional clusters in the same environment naming the schema bikeshare_qa, bikeshare_eu_qa, and bikeshare_us_qa — one schema directory cannot express the variance, and copying the directory per region would triple the source of truth.

Instead, keep one canonical directory (bikeshare/) and map the canonical namespace to each deployment's physical schema on the data-plane target:

target_resolver:
  targets:
    eu-bikeshare-qa:
      type: mysql
      schema_overrides:
        bikeshare: bikeshare_eu_qa
      dsn_from:
        # namespace-free endpoint/credentials

The canonical namespace stays the name everywhere SchemaBot stores or shows it — requests, plans, tasks, drift comparison, pull responses. The physical name only enters the data plane where MySQL is actually addressed: the connection schema in the DSN and information_schema predicates.

Rules

  • MySQL only, and currently exactly one mapping per target.
  • The target DSN must be namespace-free; a DSN that already names a database is rejected at config load.
  • A non-empty map is a strict allowlist: a requested namespace without a mapping fails rather than falling back to the canonical name, so a misrouted request cannot land in the wrong physical schema.
  • An empty/omitted map preserves the default behavior: the requested namespace is the physical schema.
  • Schema names must be unquoted-identifier-safe ([a-zA-Z0-9_$], at most 64 characters).

Summary

Scenario schemabot.yaml Namespaces Example
MySQL, single schema 1 1 testapp/
MySQL, multiple schema names 1 many app_primary/, app_analytics/
MySQL, different databases 1 per database 1 each separate directories
Vitess, multiple keyspaces 1 many commerce/, commerce_sharded/
Environment-specific namespace 1 1 per env bikeshare_$ENV/
Repo-only namespace (never deployed) 1 all except ignored ignore_namespaces: [commerce_test]
Deployment-specific physical schema 1 1 canonical bikeshare/ + per-target schema_overrides

How Namespaces Flow Through the System

The namespace key is always the directory name. Each stage maps namespaces to progressively richer types:

1. Schema directory — directories on disk

schema/
├── schemabot.yaml
├── commerce/            ← namespace "commerce"
│   ├── orders_seq.sql
│   └── vschema.json
└── commerce_sharded/    ← namespace "commerce_sharded"
    ├── orders.sql
    └── vschema.json

2. SchemaFiles — parsed into map[string]*Namespace

"commerce"         → {Files: {"orders_seq.sql": "CREATE TABLE ...", "vschema.json": "{...}"}}
"commerce_sharded" → {Files: {"orders.sql": "CREATE TABLE ...", "vschema.json": "{...}"}}

3. PlanResult.Changes — engine outputs []SchemaChange

{Namespace: "commerce",         Tables: [{Table: "orders_seq", Operation: "alter", DDL: "..."}], Files: []}
{Namespace: "commerce_sharded", Tables: [{Table: "orders", Operation: "create", DDL: "..."}], Files: [{Name: "vschema.json", Diff: "..."}]}