feat: sqlite drizzle support

[shadowfax92]

• feat: drizzle support with custom migration
• feat: migrate rater limiter to drizzle and checkin drizzle/ package
• feat: if migrate fails, nuke db and create fresh

[greptile-apps]

Greptile Summary

This PR migrates the database layer from raw Bun SQLite to Drizzle ORM, adding type-safe database operations and a custom embedded migration system for Bun compile compatibility.

Key changes:

• Replaced raw SQL queries with Drizzle ORM across RateLimiter and IdentityService
• Implemented custom migration system that embeds SQL in versions.ts instead of reading external files
• Added automatic database recovery mechanism that backs up and recreates corrupted databases
• Created new conversations and messages schemas for future conversation persistence
• Added db:sync script to convert drizzle-kit generated migrations into embedded format
• Comprehensive test coverage for migration system including constraints, cascade behavior, and idempotency

Issues found:

• The drizzle-generated SQL in drizzle/0000_messy_rafael_vega.sql is missing the CHECK (id = 1) constraint on the identity table that's critical for enforcing the singleton pattern. The constraint exists in versions.ts but not in the drizzle-generated files, creating a mismatch between the two migration sources.
• The database recovery path in db/index.ts could fail if db.close() throws an error on a corrupted connection, preventing the recovery from completing.
• Minor redundancy where _migrations table is created both in the migration runner and within the first migration itself.

The migration is well-designed with proper transaction handling, error recovery, and maintains backward compatibility with existing databases.

Confidence Score: 4/5

• This PR is safe to merge with minor issues that should be addressed
• The migration from raw SQL to Drizzle ORM is well-structured with proper testing, transaction handling, and recovery mechanisms. However, there's a critical schema mismatch where the drizzle-generated SQL doesn't include the CHECK constraint for the identity table singleton pattern, and a potential error handling issue in the database recovery path. These issues won't cause immediate failures but could lead to data integrity problems or recovery failures in edge cases.
• Pay close attention to apps/server/src/lib/db/schema/identity.ts (missing CHECK constraint) and apps/server/src/lib/db/index.ts (error handling in recovery path)

Important Files Changed

Filename	Overview
apps/server/src/lib/db/schema/identity.ts	Added Drizzle schema for identity table, but missing CHECK constraint for singleton pattern enforcement
apps/server/src/lib/db/index.ts	Implemented database initialization with migration recovery path, potential issue with db.close() error handling
apps/server/src/lib/db/migrations/index.ts	Custom migration system for embedded SQL migrations, properly handles transactions and version tracking
apps/server/src/lib/db/migrations/versions.ts	Embedded migration definitions with proper constraints, minor redundancy with _migrations table creation
apps/server/src/lib/rate-limiter/rate-limiter.ts	Migrated from raw SQL to Drizzle ORM, maintains same logic for rate limiting with proper deduplication
apps/server/src/lib/identity.ts	Migrated to Drizzle ORM, maintains same identity management logic with proper conflict handling
apps/server/src/lib/db/schema/conversations.ts	Added conversations and messages schemas with proper foreign key constraints, enum validation could be stronger at DB level

Sequence Diagram

sequenceDiagram
    participant App as Application
    participant Init as initializeDb()
    participant Open as openAndMigrate()
    participant Mig as runMigrations()
    participant Nuke as nukeAndRetry()
    participant DB as SQLite Database
    participant Drizzle as Drizzle Client

    App->>Init: initializeDb(dbPath)
    
    alt First initialization
        Init->>Open: openAndMigrate(dbPath)
        Open->>DB: new Database(dbPath)
        DB-->>Open: sqliteDb
        Open->>DB: PRAGMA journal_mode = WAL
        Open->>DB: PRAGMA foreign_keys = ON
        Open->>Mig: runMigrations(sqliteDb)
        
        Mig->>DB: CREATE TABLE IF NOT EXISTS _migrations
        Mig->>DB: SELECT version FROM _migrations
        DB-->>Mig: []
        
        loop For each pending migration
            Mig->>DB: BEGIN TRANSACTION
            Mig->>DB: Execute migration SQL (splitStatements)
            Mig->>DB: INSERT INTO _migrations
            Mig->>DB: COMMIT
        end
        
        Mig-->>Open: Success
        Open-->>Init: sqliteDb
        Init->>Drizzle: createDrizzleClient(sqliteDb)
        Drizzle-->>Init: drizzleDb
        Init-->>App: drizzleDb
    else Migration fails
        Open->>Mig: runMigrations(sqliteDb)
        Mig-->>Open: throws error
        Open-->>Init: throws error
        Init->>Init: catch error, close db
        Init->>Nuke: nukeAndRetry(dbPath)
        
        Nuke->>DB: rename db to .bak
        Nuke->>DB: delete WAL files
        Nuke->>Open: openAndMigrate(dbPath)
        Open->>DB: new Database (fresh)
        Open->>Mig: runMigrations(sqliteDb)
        Mig-->>Open: Success
        Open-->>Nuke: sqliteDb
        Nuke-->>Init: sqliteDb
        Init->>Drizzle: createDrizzleClient(sqliteDb)
        Drizzle-->>Init: drizzleDb
        Init-->>App: drizzleDb
    end
    
    App->>App: Use drizzleDb for queries

Loading

[greptile-apps]

_{20 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

logic: The .$default(() => 1) is a client-side default in Drizzle ORM, but the generated migration SQL in drizzle/0000_messy_rafael_vega.sql doesn't include the critical CHECK (id = 1) constraint that existed in the old schema. This constraint ensures the identity table only ever has a single row with id = 1, which is essential for the singleton identity pattern.

The drizzle-generated SQL shows:

CREATE TABLE `identity` (
  `id` integer PRIMARY KEY NOT NULL,
  ...
)

But the manually written migration in versions.ts correctly includes:

CREATE TABLE IF NOT EXISTS identity (
  id INTEGER PRIMARY KEY CHECK (id = 1),
  ...
)

This mismatch means the drizzle/ folder doesn't match versions.ts, which could cause confusion and issues during development or when syncing schemas.

Prompt To Fix With AI

This is a comment left during a code review.
Path: apps/server/src/lib/db/schema/identity.ts
Line: 9:12

Comment:
**logic:** The `.$default(() => 1)` is a client-side default in Drizzle ORM, but the generated migration SQL in `drizzle/0000_messy_rafael_vega.sql` doesn't include the critical `CHECK (id = 1)` constraint that existed in the old schema. This constraint ensures the identity table only ever has a single row with `id = 1`, which is essential for the singleton identity pattern.

The drizzle-generated SQL shows:
```sql
CREATE TABLE `identity` (
  `id` integer PRIMARY KEY NOT NULL,
  ...
)
```

But the manually written migration in `versions.ts` correctly includes:
```sql
CREATE TABLE IF NOT EXISTS identity (
  id INTEGER PRIMARY KEY CHECK (id = 1),
  ...
)
```

This mismatch means the `drizzle/` folder doesn't match `versions.ts`, which could cause confusion and issues during development or when syncing schemas.

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

logic: If db fails to open initially (line 57), it will be null here. However, if it partially opened (connection succeeded but migration failed), the db object may exist but be in a bad state. Calling db.close() on a partially-initialized or corrupted connection could throw an error. Wrap the .close() call in a try-catch to prevent the recovery path from failing.

Suggested change

	if (db) {
	db.close()
	db = null
	try {
	if (db) {
	db.close()
	}
	} catch {
	// Ignore close errors during recovery
	} finally {
	db = null
	}

Have you tested the recovery path when the database is corrupted mid-open to ensure close() doesn't throw?

Prompt To Fix With AI

This is a comment left during a code review.
Path: apps/server/src/lib/db/index.ts
Line: 64:66

Comment:
**logic:** If `db` fails to open initially (line 57), it will be `null` here. However, if it partially opened (connection succeeded but migration failed), the `db` object may exist but be in a bad state. Calling `db.close()` on a partially-initialized or corrupted connection could throw an error. Wrap the `.close()` call in a try-catch to prevent the recovery path from failing.

```suggestion
    try {
      if (db) {
        db.close()
      }
    } catch {
      // Ignore close errors during recovery
    } finally {
      db = null
    }
```

 Have you tested the recovery path when the database is corrupted mid-open to ensure close() doesn't throw?

How can I resolve this? If you propose a fix, please make it concise.