Skip to content

fix: bootstrap forward-references for v39-v41 schema replay#741

Open
lanceretter wants to merge 1 commit intogarrytan:masterfrom
lanceretter:fix/bootstrap-v39-v41-coverage
Open

fix: bootstrap forward-references for v39-v41 schema replay#741
lanceretter wants to merge 1 commit intogarrytan:masterfrom
lanceretter:fix/bootstrap-v39-v41-coverage

Conversation

@lanceretter
Copy link
Copy Markdown

@lanceretter lanceretter commented May 8, 2026

Summary

Three column-with-index forward references in the embedded schema blob were missing from applyForwardReferenceBootstrap, so any brain at config.version < 39 (Postgres) or < 41 (PGLite) wedges before the migration runner can advance.

Reproduced end-to-end on a PlanetScale Postgres brain stuck at config.version=34 trying to upgrade to v0.30.0:

ERROR: column "effective_date" does not exist
ERROR: column cc.modality does not exist

After hitting these errors, gbrain search and gbrain reindex-frontmatter both fail because the same columns are referenced by query-path code, but you can't get past the migration step to add them.

The schema-blob references that crash before migrations run

  • v39 (multimodal_dual_column_v0_27_1):
    CREATE INDEX idx_chunks_embedding_image
      ON content_chunks USING hnsw (embedding_image vector_cosine_ops)
      WHERE embedding_image IS NOT NULL;
  • v41 (pages_recency_columns):
    CREATE INDEX pages_coalesce_date_idx
      ON pages ((COALESCE(effective_date, updated_at)));

What's in this PR

  • Postgres engine (src/core/postgres-engine.ts): probe + branch for v39 (modality, embedding_image) — was entirely missing on Postgres, so Postgres brains pre-v39 hit the wedge that PGLite already protected against. PGLite's probe at line 273 had embedding_image_exists; Postgres did not.
  • Both engines: probe + branch for v40+v41. Bootstraps all five additive pages columns (emotional_weight, effective_date, effective_date_source, import_filename, salience_touched_at) gated on effective_date_exists as the proxy.
  • test/schema-bootstrap-coverage.test.ts: extends REQUIRED_BOOTSTRAP_COVERAGE with the six new columns AND the pre-test DROP block so both the per-target assertion test and the end-to-end "bootstrap + SCHEMA_SQL replay" test exercise the new coverage.

Bootstrap stays additive-columns-only. Indexes are left to schema replay / migrations as before — adding indexes in bootstrap duplicates responsibilities and worsens lock/surprise risk.

Test plan

  • bun test test/schema-bootstrap-coverage.test.ts — all 5 tests pass (was 5 before, still 5; expanded coverage caught no regressions in the existing assertions).
  • bun run typecheck — clean.
  • e2e Postgres bootstrap (test/e2e/postgres-bootstrap.test.ts) on a real Postgres — couldn't run locally without DATABASE_URL to a fresh test DB, but the same bootstrap function the PGLite test exercises is now mirrored on Postgres.

Companion to #740

#740 fixes three independent v0.30.0 upgrade-path bugs (deps, UNSAFE_TRANSACTION, missing connect). This PR is the structural follow-up flagged in #740's body — it closes the bootstrap gap so future brains crossing v0.27/v0.29.1 don't need the same manual ALTER TABLE ADD COLUMN IF NOT EXISTS workaround.

The two PRs are independent: either can land first.

🤖 Generated with Claude Code


View in Codesmith
Need help on this PR? Tag @codesmith with what you need.

  • Let Codesmith autofix CI failures and bot reviews

Three column-with-index forward references in the embedded schema blob were
missing from applyForwardReferenceBootstrap, so any brain at config.version
< 39 (Postgres) or < 41 (PGLite) wedges before the migration runner can
advance. Reproduced end-to-end on a PlanetScale Postgres brain stuck at
config.version=34 trying to upgrade to v0.30.0:

  ERROR: column "effective_date" does not exist
  ERROR: column cc.modality does not exist

(After upgrading, gbrain search and gbrain reindex-frontmatter both fail.)

The schema-blob references that crash before migrations run:

- v39 (multimodal_dual_column_v0_27_1):
    CREATE INDEX idx_chunks_embedding_image
      ON content_chunks USING hnsw (embedding_image vector_cosine_ops)
      WHERE embedding_image IS NOT NULL;
- v41 (pages_recency_columns):
    CREATE INDEX pages_coalesce_date_idx
      ON pages ((COALESCE(effective_date, updated_at)));

PGLite already covered v39 (lines 273+, 308+, 382-392). Postgres and PGLite
both lacked v40+v41 coverage. This commit adds:

- Postgres engine probe + branch for v39 (modality, embedding_image) — was
  entirely missing on Postgres, so Postgres brains < v39 hit the wedge that
  PGLite already protected against.
- Both engines: probe + branch for v40+v41. Bootstraps all five additive
  pages columns (emotional_weight, effective_date, effective_date_source,
  import_filename, salience_touched_at) gated on `effective_date_exists`
  as the proxy.
- test/schema-bootstrap-coverage.test.ts: extends REQUIRED_BOOTSTRAP_COVERAGE
  with the six new columns AND the pre-test DROP block so both the per-target
  assertion test and the end-to-end "bootstrap + SCHEMA_SQL replay" test
  exercise the new coverage.

All 5 tests in schema-bootstrap-coverage pass. typecheck clean.

Bootstrap stays additive-columns-only. Indexes are left to schema replay /
migrations as before.
garrytan added a commit that referenced this pull request May 9, 2026
Codex-mandated test gate (C3 from /codex review of v0.30.3 plan).

Pins the upgrade-path claim in the v0.30.3 release notes: brains stuck
at config.version < 39 (Postgres) or < 41 (PGLite) walk forward cleanly
through #741's bootstrap additions. Without this, the release note's
"old PGLite brains upgrade cleanly through v39-v41" was unproven.

Four cases:
  1. pre-v39 (missing modality + embedding_image)
  2. pre-v40 (missing emotional_weight + effective_date + effective_date_source)
  3. pre-v41 (missing import_filename + salience_touched_at)
  4. compounded pre-v34 wedge (v0.20 + v0.26.3 + v39-v41 all dropped at once)

Pattern follows test/e2e/v0_28_5-fix-wave.test.ts: build a fresh LATEST
brain, surgically rewind via DROP COLUMN CASCADE + UPDATE config.version,
then re-call initSchema and assert advancement to LATEST_VERSION with
the rewound columns restored. PGLite-only — Postgres-side bootstrap is
covered separately by test/e2e/postgres-bootstrap.test.ts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant