feat: Add MySQL/MariaDB Support as a Wren UI Persistence Backend #2040
Conversation
WalkthroughAdds MySQL/MariaDB support across wren-ui: environment/config entries, mysql2 dependency and knex/mysql client bootstrap, conditional migration changes (json vs jsonb and unsigned integers), and repository-level adjustments to handle MySQL's lack of RETURNING in inserts/updates. Changes
Sequence Diagram(s)sequenceDiagram
participant App as Application
participant Config as Config Loader
participant Knex as Knex Bootstrap
participant DB as Database
Note over Config,Knex: Startup DB selection
App->>Config: read env (DB_TYPE, MYSQL_*)
Config->>Knex: pass dbType + connection info
alt DB_TYPE == mysql
Knex->>DB: initialize mysql2 pool (host/port/user/db)
Note right of DB: JSON columns use `json`<br/>Integers may be `unsigned()`
else DB_TYPE == pg/sqlite
Knex->>DB: initialize pg/sqlite
Note right of DB: JSON columns use `jsonb` (pg) or type-specific
end
Note over App,DB: Data write flows
App->>DB: INSERT/UPDATE
alt mysql2 (no RETURNING)
DB-->>App: operation complete (no row)
App->>DB: SELECT inserted/updated rows by id
DB-->>App: return rows
else other client (supports RETURNING)
DB-->>App: return rows via RETURNING
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Areas to focus during review:
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (1)
🧰 Additional context used🧠 Learnings (1)📚 Learning: 2025-03-18T10:28:10.593ZApplied to files:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
wren-ui/migrations/20250102074255_create_dashboard_table.js (1)
27-37: Use the transaction object for the SELECT query.The
forUpdate()call must be executed within the transaction scope. Changeknex('project').forUpdate()totrx('project').forUpdate()to ensure the row lock is acquired within the same transaction as the INSERT operation.const projects = await trx('project').forUpdate();Without this change, the SELECT query executes outside the transaction context, defeating the purpose of row locking and risking race conditions during the migration.
wren-ui/src/apollo/server/repositories/baseRepository.ts (1)
110-161: MySQL inserts return 0 when PK is provided, breaking createOne/createManyOn mysql2,
.insert()returns[0]whenever the primary key is supplied (typical for UUID/string IDs). With the new branch,createOnethen querieswhere({ id: 0 }), getsundefined, andtransformFromDBDatathrows.createManysimilarly treatsfirstId === 0as numeric, probeswhereBetween('id', [0, …]), and returns no rows—so every bulk insert for tables with non auto-increment IDs comes back empty. The brand-newapi_historytable (idis a string primary key) will hit this immediately.We need to fall back to the caller-provided IDs whenever the driver can’t supply a positive auto-increment value, and ensure the bulk path always pushes rows.
@@ - const [id] = await query; - const inserted = await executer(this.tableName).where({ id }).first(); + const insertResult = await query; + const insertedId = Array.isArray(insertResult) + ? insertResult[0] + : insertResult; + const payloadId = + (this.transformToDBData(data) as Record<string, unknown>).id ?? + (data as Record<string, unknown>).id; + const lookupId = + typeof insertedId === 'number' && insertedId > 0 ? insertedId : payloadId; + if (lookupId === undefined) { + throw new Error( + `Unable to resolve primary key for ${this.tableName} insert on mysql2`, + ); + } + const inserted = await executer(this.tableName) + .where({ id: lookupId }) + .first(); @@ - const insertedIds = await query; // Returns the first ID in the sequence - const firstId = Array.isArray(insertedIds) - ? insertedIds[0] - : insertedIds; - - // Search for the range of inserted IDs (if the table uses autoincrement) - // ⚠️ This only works well if the 'id' field is auto increment. - if (typeof firstId === 'number') { - const lastId = firstId + batchValues.length - 1; - const rows = await executer(this.tableName) - .whereBetween('id', [firstId, lastId]) - .orderBy('id', 'asc'); - result.push(...rows.map(this.transformFromDBData)); - } else { - // Fallback without numeric ID — returns the raw inserted data - // Here batchValues are already in snake_case format; transformFromDBData converts to camelCase. - result.push(...batchValues.map(this.transformFromDBData)); - } + const insertResult = await query; + const firstInsertedId = Array.isArray(insertResult) + ? insertResult[0] + : insertResult; + const providedIds = batchValuesOriginal + .map((item) => (item as Record<string, unknown>).id) + .filter((id): id is string | number => id !== undefined && id !== null); + + let rows: unknown[] | null = null; + if ( + typeof firstInsertedId === 'number' && + Number.isFinite(firstInsertedId) && + providedIds.length === 0 + ) { + const lastId = firstInsertedId + batchValues.length - 1; + rows = await executer(this.tableName) + .whereBetween('id', [firstInsertedId, lastId]) + .orderBy('id', 'asc'); + } else if (providedIds.length > 0) { + rows = await executer(this.tableName) + .whereIn('id', providedIds) + .orderBy('id', 'asc'); + } + + const rowsToReturn = + rows && rows.length > 0 ? rows : batchValues; + result.push(...rowsToReturn.map(this.transformFromDBData));
🧹 Nitpick comments (2)
wren-ui/migrations/20250509000000_create_asking_task.js (1)
30-30: Fix formatting: Add spaces aroundelse.The
elsekeyword should have spaces for consistency with the project's coding style and the pattern used elsewhere in this file (line 12).Apply this diff:
- }else{ + } else {docker/.env.example (1)
55-60: Address formatting and ordering issues.The static analysis tool has identified valid concerns:
- Missing blank line at the end of the file
- Environment variables should be in alphabetical order (MYSQL_DB before MYSQL_HOST, MYSQL_PASSWORD before MYSQL_PORT)
Apply this diff to fix ordering and add trailing newline:
# MySQL Database Configuration +MYSQL_DB=wrenai_db MYSQL_HOST=mysql-database +MYSQL_PASSWORD=wrenai_pwd MYSQL_PORT=3306 MYSQL_USER=wrenai_user -MYSQL_PASSWORD=wrenai_pwd -MYSQL_DB=wrenai_db +
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
wren-ui/yarn.lockis excluded by!**/yarn.lock,!**/*.lock
📒 Files selected for processing (30)
docker/.env.example(1 hunks)wren-ui/README.md(1 hunks)wren-ui/knexfile.js(1 hunks)wren-ui/migrations/20240125070643_create_project_table.js(1 hunks)wren-ui/migrations/20240319083758_create_deploy_table.js(1 hunks)wren-ui/migrations/20240327030000_create_ask_table.js(2 hunks)wren-ui/migrations/20240419090558_add_foreign_key_to_model_column_and_metric_measure.js(1 hunks)wren-ui/migrations/20240446090560_update_relationship_table.js(1 hunks)wren-ui/migrations/20240530062133_update_project_table.js(1 hunks)wren-ui/migrations/20240530105955_drop_project_table_columns.js(2 hunks)wren-ui/migrations/20240610070534_create_schema_change_table.js(1 hunks)wren-ui/migrations/20240928165009_create_model_nested_column.js(1 hunks)wren-ui/migrations/20241106232204_update_project_table.js(2 hunks)wren-ui/migrations/20241107171828_update_thread_table.js(2 hunks)wren-ui/migrations/20241207000000_update_thread_response_for_answer.js(2 hunks)wren-ui/migrations/20241210072534_update_thread_response_table.js(1 hunks)wren-ui/migrations/20250102074255_create_dashboard_table.js(1 hunks)wren-ui/migrations/20250102074256_create_dashboard_item_table.js(1 hunks)wren-ui/migrations/20250102074256_create_sql_pair_table.js(1 hunks)wren-ui/migrations/20250311046282_create_instruction_table.js(1 hunks)wren-ui/migrations/20250423000000_create_dashboard_cache_refresh_table.js(1 hunks)wren-ui/migrations/20250509000000_create_asking_task.js(1 hunks)wren-ui/migrations/20250509000001_add_task_id_to_thread.js(1 hunks)wren-ui/migrations/20250510000000_add_adjustment_to_thread_response.js(1 hunks)wren-ui/migrations/20250511000000-create-api-history.js(2 hunks)wren-ui/package.json(1 hunks)wren-ui/src/apollo/server/config.ts(3 hunks)wren-ui/src/apollo/server/repositories/baseRepository.ts(3 hunks)wren-ui/src/apollo/server/repositories/threadResponseRepository.ts(1 hunks)wren-ui/src/apollo/server/utils/knex.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📚 Learning: 2025-01-15T03:51:14.432Z
Learnt from: andreashimin
Repo: Canner/WrenAI PR: 1117
File: wren-ui/src/apollo/server/repositories/threadResponseRepository.ts:37-37
Timestamp: 2025-01-15T03:51:14.432Z
Learning: In the ThreadResponseRepository, the `chartType` property is intentionally typed as string in the database layer, while the ChartType enum is used specifically for AI adaptor input/output in the application layer.
Applied to files:
wren-ui/migrations/20241210072534_update_thread_response_table.js
📚 Learning: 2025-01-08T07:36:36.665Z
Learnt from: andreashimin
Repo: Canner/WrenAI PR: 1090
File: wren-ui/src/apollo/client/graphql/__types__.ts:106-109
Timestamp: 2025-01-08T07:36:36.665Z
Learning: In the WrenAI system, there is a one-to-one relationship between projects and dashboards. Each project has exactly one dashboard, which is automatically created when the project is created. Therefore, `dashboardId` is not needed in dashboard-related operations as it can be derived from the project context.
Applied to files:
wren-ui/migrations/20250102074255_create_dashboard_table.js
📚 Learning: 2025-03-18T10:28:10.593Z
Learnt from: andreashimin
Repo: Canner/WrenAI PR: 1414
File: wren-ui/src/apollo/server/utils/error.ts:0-0
Timestamp: 2025-03-18T10:28:10.593Z
Learning: The typo in error code enum names ("IDENTIED_AS_GENERAL" and "IDENTIED_AS_MISLEADING_QUERY" instead of "IDENTIFIED_AS_GENERAL" and "IDENTIFIED_AS_MISLEADING_QUERY") in wren-ui/src/apollo/server/utils/error.ts was recognized but intentionally not fixed as it was considered out of scope for PR #1414.
Applied to files:
wren-ui/src/apollo/server/repositories/baseRepository.tswren-ui/src/apollo/server/utils/knex.tswren-ui/src/apollo/server/repositories/threadResponseRepository.ts
📚 Learning: 2025-02-12T22:05:37.109Z
Learnt from: cyyeh
Repo: Canner/WrenAI PR: 1293
File: wren-ai-service/pyproject.toml:38-38
Timestamp: 2025-02-12T22:05:37.109Z
Learning: The qdrant-client package version in wren-ai-service must match the Qdrant server version (1.11.0) to maintain compatibility.
Applied to files:
wren-ui/package.json
🧬 Code graph analysis (11)
wren-ui/migrations/20250509000001_add_task_id_to_thread.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20241106232204_update_project_table.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20241207000000_update_thread_response_for_answer.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20240319083758_create_deploy_table.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20240610070534_create_schema_change_table.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20240928165009_create_model_nested_column.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20250509000000_create_asking_task.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20240530062133_update_project_table.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20250510000000_add_adjustment_to_thread_response.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20250511000000-create-api-history.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
wren-ui/migrations/20250423000000_create_dashboard_cache_refresh_table.js (1)
wren-ui/tools/knex.js (1)
knex(30-30)
🪛 dotenv-linter (4.0.0)
docker/.env.example
[warning] 59-59: [UnorderedKey] The MYSQL_PASSWORD key should go before the MYSQL_PORT key
(UnorderedKey)
[warning] 60-60: [EndingBlankLine] No blank line at the end of the file
(EndingBlankLine)
[warning] 60-60: [UnorderedKey] The MYSQL_DB key should go before the MYSQL_HOST key
(UnorderedKey)
🔇 Additional comments (28)
wren-ui/migrations/20241210072534_update_thread_response_table.js (1)
7-11: LGTM! Dialect-aware JSON column type selection.The conditional logic correctly selects
jsonfor MySQL andjsonbfor PostgreSQL/others, aligning with the repository-wide pattern for MySQL/MariaDB support.wren-ui/migrations/20240319083758_create_deploy_table.js (1)
11-15: LGTM! Consistent dialect-aware JSON type handling.The conditional JSON type selection follows the established pattern across the codebase for MySQL compatibility.
wren-ui/migrations/20250102074255_create_dashboard_table.js (1)
8-19: LGTM! Appropriate use of unsigned() for MySQL foreign keys.The conditional
unsigned()modifier is correctly applied for MySQL, which is necessary when referencing auto-increment primary keys.wren-ui/README.md (1)
32-56: LGTM! Comprehensive MySQL/MariaDB setup documentation.The documentation clearly explains the MySQL configuration with separate examples for Windows and Unix/macOS, includes helpful notes about privileges and compatibility, and maintains consistency with the existing PostgreSQL documentation style.
wren-ui/migrations/20240928165009_create_model_nested_column.js (1)
9-13: LGTM! Correct unsigned() usage for MySQL foreign key.The conditional
unsigned()modifier appropriately handles MySQL's requirement for foreign keys referencing auto-increment primary keys.wren-ui/migrations/20250311046282_create_instruction_table.js (2)
8-19: LGTM! Appropriate MySQL-specific column handling.The conditional
unsigned()modifier correctly handles MySQL foreign key requirements.
21-25: LGTM! Consistent JSON type handling.The conditional JSON type selection (json vs jsonb) follows the established dialect-aware pattern.
wren-ui/migrations/20250509000001_add_task_id_to_thread.js (1)
7-22: LGTM! Correct foreign key handling for MySQL.The conditional
unsigned()modifier appropriately handles MySQL's requirements for foreign keys, with proper nullable and onDelete constraints.wren-ui/migrations/20241207000000_update_thread_response_for_answer.js (3)
44-54: LGTM! Consistent dialect-aware JSON type in up migration.The conditional JSON type selection correctly uses
jsonfor MySQL andjsonbfor PostgreSQL/others.
116-120: LGTM! Consistent dialect handling in down migration.The down migration correctly applies the same dialect-aware JSON type pattern, ensuring proper rollback support.
61-99: LGTM! Data migration logic handles existing records.The migration correctly transforms existing thread_response records, with proper error handling for JSON parsing and null handling.
wren-ui/migrations/20250509000000_create_asking_task.js (2)
10-14: LGTM! Consistent dialect-aware JSON type handling.The conditional json/jsonb column definition correctly handles MySQL's lack of native jsonb support while preserving PostgreSQL's more efficient jsonb type.
16-42: LGTM! Proper unsigned handling for MySQL foreign keys.The migration correctly applies the
unsigned()modifier to foreign key columns when using MySQL, which is necessary for proper constraint creation. The else branch ensures non-MySQL databases retain their original behavior.wren-ui/migrations/20240419090558_add_foreign_key_to_model_column_and_metric_measure.js (1)
5-19: LGTM! Correct unsigned handling for MySQL foreign key constraints.The migration properly ensures that foreign key columns are marked as unsigned in MySQL before establishing constraints, which prevents type mismatch errors with auto-incrementing primary keys.
wren-ui/migrations/20250510000000_add_adjustment_to_thread_response.js (1)
7-21: LGTM! Consistent dialect-aware column addition.The conditional json/jsonb handling follows the established pattern across the codebase and properly preserves all column attributes (nullable, comment) in both branches.
wren-ui/migrations/20240327030000_create_ask_table.js (2)
18-22: LGTM! Proper MySQL unsigned handling for foreign key column.The migration correctly applies unsigned to the thread_id foreign key column only in MySQL, ensuring compatibility with auto-increment primary keys.
31-37: LGTM! Appropriate JSON type selection per database.The conditional correctly uses json for MySQL and jsonb for PostgreSQL/others, accounting for database-specific JSON storage capabilities.
wren-ui/migrations/20240530105955_drop_project_table_columns.js (1)
26-65: LGTM! Proper rollback with dialect-aware column restoration.The down migration correctly restores dropped columns with appropriate json/jsonb types based on the database client, ensuring clean rollback capability across all supported databases.
wren-ui/migrations/20241107171828_update_thread_table.js (2)
7-17: LGTM! Consistent dialect-aware column addition.The questions column correctly uses json for MySQL and jsonb for other databases, maintaining the established pattern.
26-36: LGTM! Proper JSON type handling for questions_error.The questions_error column follows the same dialect-aware pattern, ensuring consistency across the migration.
wren-ui/package.json (1)
32-32: mysql2 version ^3.15.3 is current and secure.The latest version is 3.15.3, which matches the codebase. All reported security advisories (including 2 CRITICAL and 1 HIGH severity) affect versions below 3.9.8, and the current version is well above all patch thresholds, leaving no known vulnerabilities.
wren-ui/knexfile.js (1)
12-24: MySQL branch config looks solid.Connection credentials and pool settings align with mysql2 expectations, so this branch should bootstrap cleanly.
wren-ui/migrations/20241106232204_update_project_table.js (1)
7-36: Good call on dialect-specific JSON columns.Switching to
jsonfor mysql2 avoids unsupportedjsonbtypes while preserving comments and nullability.wren-ui/migrations/20250102074256_create_sql_pair_table.js (1)
7-24: Unsigned guard keeps FK types aligned.Applying
unsigned()under mysql2 matches the auto-incrementingproject.id, avoiding foreign-key mismatch errors.wren-ui/migrations/20240530062133_update_project_table.js (1)
8-19: Nice consistency on connection_info column.Using
jsonfor mysql2 keeps MySQL happy while matching the existingjsonbbehavior elsewhere.wren-ui/src/apollo/server/config.ts (3)
14-19: MySQL knobs correctly join the config contract. The optional properties mirror how pg/sqlite settings are declared, so downstream consumers can adopt them without breaking existing builds.
71-75: Defaults keep MySQL opt‑in but ready. Providing localhost/3306 scaffolding matches the sqlite/pg patterns and ensures a clean fallback when env vars are absent.
105-112: Runtime env hydration looks solid. The env parsing cleanly lifts MYSQL_* values while leaving defaults intact when unset, preserving the prior behavior.
1 participant
This PR adds first-class support for using MySQL or MariaDB as the persistence layer for Wren UI, alongside the existing SQLite and PostgreSQL options. It introduces new configuration environment variables and updates the server bootstrap logic so the application can run against a MySQL/MariaDB database with minimal setup changes.
Motivation
Teams deploying Wren UI in production may prefer MySQL/MariaDB due to:
Providing this option increases adoption flexibility while keeping backward compatibility with current defaults (SQLite for local dev, PostgreSQL already supported).
Key Changes
config.tsto read MySQL/MariaDB connection parameters:DB_TYPE=mysqlMYSQL_HOST,MYSQL_PORT,MYSQL_USER,MYSQL_PASSWORD,MYSQL_DBbootstrapKnexinsrc/apollo/server/utils/knex.tsto initialize amysql2Knex client whendbType === 'mysql'.DB_TYPEis not set (or set to the prior defaults), and retains PostgreSQL support (DB_TYPE=pg).min: 2, max: 10) similar to other drivers.Configuration
To run Wren UI with MySQL/MariaDB, set:
Summary by CodeRabbit
New Features
Bug Fixes / Compatibility
Documentation
Chores