schemadiff: describability refusals — partitions, FKs, unlogged, collations, sequence ownership

AGENTS.md

@@ -135,6 +135,14 @@ is a smoke tour of the built binary, not a second test suite.
 - Never reference internal company details (cluster names, hostnames, org names) in code,
   comments, commits, or PRs — this is a public repo.
 
+## Capability statement
+
+[docs/limitations.md](docs/limitations.md) and the README's "What pg-sprite does not do yet"
+section are the public capability statement. Any PR that adds, removes, or re-scopes a
+capability — a planner route, a refusal, a verdict outcome, a declarative-model boundary —
+updates both in the same PR, and every release sweeps them against the shipped behavior
+before tagging.
+
 ## Logging and observability
 
 - **stdout is the product's output; diagnostics go to stderr.** Command results (verdicts,

README.md

@@ -37,6 +37,28 @@ build plan live in [docs/](docs/) — start with
 [docs/README.md](docs/README.md); the vision — what pg-sprite is and is not —
 is [docs/vision.md](docs/vision.md).
 
+## What pg-sprite does not do yet
+
+So expectations are set before you point it at a database — the full list
+with per-item behavior is **[docs/limitations.md](docs/limitations.md)**;
+wherever an operation meets a boundary below, it fails closed with a typed
+refusal — never a silently wrong or incomplete result:
+
+- **Copy-and-swap** (genuine table rewrites) is not yet available — those
+  changes refuse rather than fall through to a blocking rewrite.
+- **Foreign keys** are out of the declarative model in either direction:
+  desired files cannot declare them, and export refuses both a table that
+  carries foreign keys and a table that other tables reference. FK DDL
+  still works through the statement front door (`NOT VALID` + `VALIDATE`).
+- **Partitioned tables** support in-place statement changes, but cannot be
+  expressed in or exported to desired files.
+- **Unlogged tables and explicit column collations** are outside the
+  declarative model: converging either is a table (or column) rewrite, so
+  export and diff refuse rather than plan one.
+- **Non-table objects** — views, standalone sequences, enums, domains,
+  extensions, functions, triggers — are outside the declarative model,
+  which covers one ordinary table plus its indexes per file.
+
 The codebase is partitioned into a small safety-critical core and a
 periphery — **[SAFETY.md](SAFETY.md)** says which packages are which and the
 rules that apply inside the core. Read it before changing anything under

docs/limitations.md

@@ -9,3 +9,19 @@ guarantees. These are current capability boundaries, not escape hatches:
 | `ADD CONSTRAINT ... USING INDEX` on a partitioned parent | PostgreSQL does not support adopting an existing index on a partitioned parent in any supported version. pg-sprite refuses before execution. |
 | `ADD FOREIGN KEY ... NOT VALID` on a partitioned parent | PostgreSQL does not support this before version 18, so pg-sprite refuses it on versions 14–17. It is supported on version 18 and later. |
 | Copy-and-swap | The copy-and-swap backend is not yet available. Statements that require it route to `refuse`; pg-sprite never falls through to a blocking rewrite. |
+
+## Declarative model boundaries
+
+The declarative front door (desired-state files, `diff`, and schema export)
+models **one ordinary table plus its indexes**. These are the boundaries of
+that model today. Wherever an operation meets a boundary, it fails closed
+with a typed refusal — never a silently wrong or incomplete result:
+
+| Not modeled | Current behavior |
+| --- | --- |
+| Foreign keys (either direction) | Unsupported in the declarative model, in both directions. A desired file cannot declare a `REFERENCES` clause (refused at parse), and export refuses both sides of a foreign-key relationship: a table whose definition carries foreign-key constraints surfaces the parse gate's typed error, and a table that other tables reference refuses with its own typed error — a single-table baseline cannot carry incoming foreign-key topology, so rendering one would silently drop the relationship. Foreign-key **DDL is still supported through the statement front door** (`ADD FOREIGN KEY` routes to the online `NOT VALID` + `VALIDATE` sequence). Because a desired file can never declare a foreign key, `diff` on a live table that carries one plans a **destructive** `DROP CONSTRAINT` for it — gated like every destructive change, never auto-executed — and incoming foreign keys are invisible to a single-table diff entirely, so tables participating in foreign-key relationships in either direction should not be managed declaratively yet. |
+| Partitioned tables | Partitioned parents and their partitions are introspectable, and the statement front door supports in-place changes on them (see the partitioned-parent rows above), but they cannot be expressed in or exported to a desired file: the model captures the partition key and attachment only to refuse — it does not carry partition bounds or the parent/partition topology. A partitioning mismatch between live and desired is a typed `diff` refusal, never a zero diff. |
+| Unlogged tables | Persistence is not managed: converging it (`SET LOGGED` / `SET UNLOGGED`) is a full table rewrite. Export refuses an unlogged table — a plain `CREATE TABLE` baseline would silently change its crash-safety and replication behavior — and a persistence mismatch between live and desired is a typed `diff` refusal, never a zero diff. |
+| Column collations | An explicit `COLLATE` on a column is not managed: converging a collation delta rewrites the column and its indexes. Export refuses a collated column — a baseline without the clause would silently change sort order and index semantics — and a collation delta (including on an added column) is a typed `diff` refusal. |
+| Non-table objects | Views, materialized views, standalone sequences, enums, domains, extensions, functions, and triggers are outside the model. A serial column's owned sequence is the one exception: it round-trips through the `serial` pseudo-types — and ownership is verified through the catalog (`pg_depend`), so a hand-written `nextval` default on a standalone sequence that merely carries the serial-style name refuses rather than exporting as `serial` and silently privatizing a shared sequence. A column may *use* an unmanaged type (an enum, a domain) — the type text round-trips — but the type's definition is not managed. |
+| Multiple tables per file | A desired file is single-table scoped: exactly one `CREATE TABLE` plus `CREATE INDEX` statements on it. Multi-table schemas are managed as one file per table. |

pkg/schemadiff/diff.go

@@ -10,8 +10,11 @@ import (
 )
 
 // ErrUnsupportedChange is returned when converging live onto desired would
-// need a change the engine does not derive (identity or generation changes
-// on an existing column). The caller surfaces it; nothing is guessed.
+// need a change the engine does not derive: identity, generation, or
+// collation changes on an existing column, a persistence (unlogged)
+// difference, or a partitioning difference (partition key or partition
+// attachment) — table identity that no ALTER converges. The caller
+// surfaces it; nothing is guessed.
 var ErrUnsupportedChange = errors.New("unsupported schema change")
 
 // ErrDifferentTables is returned when the two models describe different
@@ -99,6 +102,21 @@ func Diff(schema string, live, desired Model) ([]Change, error) {
 	if live.Table != desired.Table {
 		return nil, fmt.Errorf("%w: %q vs %q", ErrDifferentTables, live.Table, desired.Table)
 	}
+	// Partitioning is table identity, not an alterable attribute: no ALTER
+	// can add, remove, or change a partition key or a partition attachment
+	// in place, so a mismatch fails closed instead of diffing to silence.
+	if live.PartitionKey != desired.PartitionKey {
+		return nil, fmt.Errorf("partition key %q vs %q: %w", live.PartitionKey, desired.PartitionKey, ErrUnsupportedChange)
+	}
+	if live.IsPartition != desired.IsPartition {
+		return nil, fmt.Errorf("partition attachment differs between live and desired: %w", ErrUnsupportedChange)
+	}
+	// Persistence is convergeable only by a full table rewrite (SET LOGGED
+	// / SET UNLOGGED), which the engine does not derive — a mismatch fails
+	// closed instead of diffing to silence.
+	if live.Unlogged != desired.Unlogged {
+		return nil, fmt.Errorf("table persistence (unlogged) differs between live and desired: %w", ErrUnsupportedChange)
+	}
 	table := pgx.Identifier{schema, live.Table}.Sanitize()
 
 	liveCols := columnsByName(live.Columns)
@@ -157,6 +175,11 @@ func Diff(schema string, live, desired Model) ([]Change, error) {
 			if col.SequenceDefault {
 				return nil, fmt.Errorf("%w: column %q has a sequence-backed default (serial); the plan cannot create its sequence", ErrUnsupportedChange, col.Name)
 			}
+			// columnDef carries no COLLATE clause, so emitting the add
+			// would silently drop the collation — refuse instead.
+			if col.Collation != "" {
+				return nil, fmt.Errorf("%w: column %q has an explicit collation (%s); the plan cannot carry it", ErrUnsupportedChange, col.Name, col.Collation)
+			}
 			changes = append(changes, Change{
 				SQL:  "ALTER TABLE " + table + " ADD COLUMN " + columnDef(col),
 				Kind: ChangeAddColumn,
@@ -215,6 +238,12 @@ func alterColumnChanges(table string, live, desired Column) ([]Change, error) {
 	if live.Generated != desired.Generated {
 		return nil, fmt.Errorf("%w: column %q generated change", ErrUnsupportedChange, live.Name)
 	}
+	// A collation change rewrites the column and rebuilds its indexes —
+	// the engine does not derive it, so a delta fails closed instead of
+	// diffing to silence.
+	if live.Collation != desired.Collation {
+		return nil, fmt.Errorf("%w: column %q collation change (%q vs %q)", ErrUnsupportedChange, live.Name, live.Collation, desired.Collation)
+	}
 	if desired.Generated && (live.Type != desired.Type || live.Default != desired.Default) {
 		return nil, fmt.Errorf("%w: column %q generation expression or type change", ErrUnsupportedChange, live.Name)
 	}

pkg/schemadiff/diff_test.go

@@ -45,6 +45,52 @@ func TestDiffRefusesDifferentTables(t *testing.T) {
 	require.ErrorIs(t, err, ErrDifferentTables)
 }
 
+// Partitioning is table identity: no ALTER can change a partition key or a
+// partition attachment in place, so a mismatch is a typed refusal — never a
+// silent zero diff.
+func TestDiffRefusesPartitioningMismatch(t *testing.T) {
+	partitioned := base()
+	partitioned.PartitionKey = "RANGE (id)"
+	_, err := Diff("public", base(), partitioned)
+	require.ErrorIs(t, err, ErrUnsupportedChange)
+
+	child := base()
+	child.IsPartition = true
+	_, err = Diff("public", child, base())
+	require.ErrorIs(t, err, ErrUnsupportedChange)
+}
+
+// Persistence converges only by a full table rewrite (SET LOGGED / SET
+// UNLOGGED), which the engine does not derive — a mismatch is a typed
+// refusal, never a silent zero diff.
+func TestDiffRefusesPersistenceMismatch(t *testing.T) {
+	unlogged := base()
+	unlogged.Unlogged = true
+	_, err := Diff("public", unlogged, base())
+	require.ErrorIs(t, err, ErrUnsupportedChange)
+
+	_, err = Diff("public", base(), unlogged)
+	require.ErrorIs(t, err, ErrUnsupportedChange)
+}
+
+// A collation delta on an existing column rewrites the column, and an
+// added column's COLLATE clause is not carried by the emitted ADD COLUMN —
+// both are typed refusals, never a silent zero diff or a silently dropped
+// clause.
+func TestDiffRefusesCollationChanges(t *testing.T) {
+	collated := base()
+	collated.Columns[1].Collation = `"C"`
+	_, err := Diff("public", base(), collated)
+	require.ErrorIs(t, err, ErrUnsupportedChange)
+
+	added := base()
+	added.Columns = append(added.Columns, Column{
+		Name: "note", Type: "text", Collation: `"C"`,
+	})
+	_, err = Diff("public", base(), added)
+	require.ErrorIs(t, err, ErrUnsupportedChange)
+}
+
 func TestDiffAddColumn(t *testing.T) {
 	desired := base()
 	desired.Columns = append(desired.Columns, Column{

pkg/schemadiff/introspect.go

@@ -56,12 +56,16 @@ func introspectInTx(ctx context.Context, tx pgx.Tx, schema, table string) (Model
 	}
 
 	var oid uint32
-	var relkind string
+	var relkind, persistence string
+	var isPartition bool
+	var partitionKey string
 	err := tx.QueryRow(ctx, `
-		SELECT c.oid, c.relkind::text
+		SELECT c.oid, c.relkind::text, c.relpersistence::text, c.relispartition,
+		       COALESCE(pg_get_partkeydef(c.oid), '')
 		FROM pg_class c
 		JOIN pg_namespace n ON n.oid = c.relnamespace
-		WHERE n.nspname = $1 AND c.relname = $2`, schema, table).Scan(&oid, &relkind)
+		WHERE n.nspname = $1 AND c.relname = $2`, schema, table).
+		Scan(&oid, &relkind, &persistence, &isPartition, &partitionKey)
 	if errors.Is(err, pgx.ErrNoRows) {
 		return Model{}, fmt.Errorf("%s.%s: %w", schema, table, ErrTableNotFound)
 	}
@@ -72,7 +76,7 @@ func introspectInTx(ctx context.Context, tx pgx.Tx, schema, table string) (Model
 		return Model{}, fmt.Errorf("%s.%s has relkind %q: %w", schema, table, relkind, ErrNotTable)
 	}
 
-	m := Model{Table: table}
+	m := Model{Table: table, PartitionKey: partitionKey, IsPartition: isPartition, Unlogged: persistence == "u"}
 	if m.Columns, err = introspectColumns(ctx, tx, oid); err != nil {
 		return Model{}, fmt.Errorf("introspect columns of %s.%s: %w", schema, table, err)
 	}
@@ -82,11 +86,20 @@ func introspectInTx(ctx context.Context, tx pgx.Tx, schema, table string) (Model
 	if m.Indexes, err = introspectIndexes(ctx, tx, oid); err != nil {
 		return Model{}, fmt.Errorf("introspect indexes of %s.%s: %w", schema, table, err)
 	}
+	if m.ReferencedBy, err = introspectReferencedBy(ctx, tx, oid); err != nil {
+		return Model{}, fmt.Errorf("introspect incoming foreign keys of %s.%s: %w", schema, table, err)
+	}
 	return m, nil
 }
 
 // introspectColumns reads the canonical column list: server-formatted types
 // and server-decompiled default/generation expressions, in attribute order.
+// Two dependency facts ride along for each sequence-backed default: that
+// the default depends on a sequence at all (the pg_attrdef edge), and that
+// the sequence is owned by this exact column (the OWNED BY edge, deptype
+// 'a') — the discriminator between a serial column and a hand-written
+// nextval on a standalone or shared sequence. An explicit column collation
+// is read only when it differs from the type's default collation.
 func introspectColumns(ctx context.Context, tx pgx.Tx, oid uint32) ([]Column, error) {
 	rows, err := tx.Query(ctx, `
 		SELECT a.attname,
@@ -102,6 +115,29 @@ func introspectColumns(ctx context.Context, tx pgx.Tx, oid uint32) ([]Column, er
 		             AND dep.refclassid = 'pg_class'::regclass
 		           LIMIT 1
 		       ), false),
+		       COALESCE((
+		           SELECT true
+		           FROM pg_depend dep
+		           JOIN pg_class s ON s.oid = dep.refobjid AND s.relkind = 'S'
+		           JOIN pg_depend own ON own.classid = 'pg_class'::regclass
+		                             AND own.objid = s.oid
+		                             AND own.refclassid = 'pg_class'::regclass
+		                             AND own.refobjid = a.attrelid
+		                             AND own.refobjsubid = a.attnum
+		                             AND own.deptype = 'a'
+		           WHERE dep.classid = 'pg_attrdef'::regclass
+		             AND dep.objid = d.oid
+		             AND dep.refclassid = 'pg_class'::regclass
+		           LIMIT 1
+		       ), false),
+		       COALESCE((
+		           SELECT quote_ident(cn.nspname) || '.' || quote_ident(col.collname)
+		           FROM pg_collation col
+		           JOIN pg_namespace cn ON cn.oid = col.collnamespace
+		           JOIN pg_type t ON t.oid = a.atttypid
+		           WHERE col.oid = a.attcollation
+		             AND a.attcollation <> t.typcollation
+		       ), ''),
 		       a.attidentity::text,
 		       a.attgenerated::text
 		FROM pg_attribute a
@@ -116,7 +152,7 @@ func introspectColumns(ctx context.Context, tx pgx.Tx, oid uint32) ([]Column, er
 	for rows.Next() {
 		var c Column
 		var identity, generated string
-		if err := rows.Scan(&c.Name, &c.Type, &c.NotNull, &c.Default, &c.SequenceDefault, &identity, &generated); err != nil {
+		if err := rows.Scan(&c.Name, &c.Type, &c.NotNull, &c.Default, &c.SequenceDefault, &c.SequenceOwned, &c.Collation, &identity, &generated); err != nil {
 			return nil, fmt.Errorf("scan column: %w", err)
 		}
 		c.Identity = Identity(identity)
@@ -158,6 +194,41 @@ func introspectConstraints(ctx context.Context, tx pgx.Tx, oid uint32) ([]Constr
 	return cons, nil
 }
 
+// introspectReferencedBy reads the incoming foreign keys: constraints on
+// other tables whose referenced relation is this table, as
+// "table.constraint" strings. A self-referential foreign key is excluded —
+// it is already carried as one of the table's own constraints. Partition
+// clones are excluded too (conislocal, as in introspectConstraints): a
+// foreign key on a partitioned referencing table is mirrored onto every
+// partition, and those mirrors are the same relationship, not independent
+// ones. These are not part of this table's definition; the model carries
+// them only for the renderer to refuse on.
+func introspectReferencedBy(ctx context.Context, tx pgx.Tx, oid uint32) ([]string, error) {
+	rows, err := tx.Query(ctx, `
+		SELECT c.relname || '.' || con.conname
+		FROM pg_constraint con
+		JOIN pg_class c ON c.oid = con.conrelid
+		WHERE con.confrelid = $1 AND con.contype = 'f' AND con.conrelid <> $1
+		      AND con.conislocal
+		ORDER BY c.relname, con.conname`, oid)
+	if err != nil {
+		return nil, fmt.Errorf("query incoming foreign keys: %w", err)
+	}
+	defer rows.Close()
+	var refs []string
+	for rows.Next() {
+		var ref string
+		if err := rows.Scan(&ref); err != nil {
+			return nil, fmt.Errorf("scan incoming foreign key: %w", err)
+		}
+		refs = append(refs, ref)
+	}
+	if err := rows.Err(); err != nil {
+		return nil, fmt.Errorf("read incoming foreign keys: %w", err)
+	}
+	return refs, nil
+}
+
 // introspectIndexes reads the non-constraint indexes as server-decompiled
 // CREATE INDEX statements. Constraint-backed indexes (primary key, unique
 // constraint, exclusion) are represented by their constraint instead.