Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 43 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,6 @@

A comprehensive monorepo for PostgreSQL Abstract Syntax Tree (AST) parsing, manipulation, and code generation. This collection of packages provides everything you need to work with PostgreSQL at the AST level, from parsing SQL queries to generating type-safe TypeScript definitions.

## 📦 Packages

| Package | Description | Key Features |
|---------|-------------|--------------|
| [**pgsql-parser**](./packages/parser) | The real PostgreSQL parser for Node.js | • Uses actual PostgreSQL C parser via WebAssembly<br>• Symmetric parsing and deparsing<br>• Battle-tested with 23,000+ SQL statements |
| [**pgsql-deparser**](./packages/deparser) | Lightning-fast SQL generation from AST | • Pure TypeScript, zero runtime dependencies<br>• No WebAssembly overhead<br>• Perfect for AST-to-SQL conversion only |
| [**@pgsql/cli**](./packages/pgsql-cli) | Unified CLI for all PostgreSQL AST operations | • Parse SQL to AST<br>• Deparse AST to SQL<br>• Generate TypeScript from protobuf<br>• Single tool for all operations |
| [**@pgsql/utils**](./packages/utils) | Type-safe AST node creation utilities | • Programmatic AST construction<br>• Runtime Schema<br>• Seamless integration with types |
| [**pg-proto-parser**](./packages/proto-parser) | PostgreSQL protobuf parser and code generator | • Generate TypeScript interfaces from protobuf<br>• Create enum mappings and utilities<br>• AST helper generation |

## 🚀 Quick Start

### Installation
Expand Down Expand Up @@ -70,6 +60,41 @@ const sql = await deparse(ast);
console.log(sql); // SELECT * FROM users WHERE id = 1
```

#### Build AST with Types
```typescript
import { deparse } from 'pgsql-deparser';
import { SelectStmt } from '@pgsql/types';

const stmt: { SelectStmt: SelectStmt } = {
SelectStmt: {
targetList: [
{
ResTarget: {
val: {
ColumnRef: {
fields: [{ A_Star: {} }]
}
}
}
}
],
fromClause: [
{
RangeVar: {
relname: 'some_table',
inh: true,
relpersistence: 'p'
}
}
],
limitOption: 'LIMIT_OPTION_DEFAULT',
op: 'SETOP_NONE'
}
};

await deparse(stmt);
```

#### Build AST Programmatically
```typescript
import * as t from '@pgsql/utils';
Expand Down Expand Up @@ -98,19 +123,16 @@ const stmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({
await deparse(stmt);
```

#### Use the CLI
```bash
npm install -g @pgsql/cli

# Parse SQL file
pgsql parse query.sql
## 📦 Packages

# Convert AST to SQL
pgsql deparse ast.json
| Package | Description | Key Features |
|---------|-------------|--------------|
| [**pgsql-parser**](./packages/parser) | The real PostgreSQL parser for Node.js | • Uses actual PostgreSQL C parser via WebAssembly<br>• Symmetric parsing and deparsing<br>• Battle-tested with 23,000+ SQL statements |
| [**pgsql-deparser**](./packages/deparser) | Lightning-fast SQL generation from AST | • Pure TypeScript, zero runtime dependencies<br>• No WebAssembly overhead<br>• Perfect for AST-to-SQL conversion only |
| [**@pgsql/cli**](./packages/pgsql-cli) | Unified CLI for all PostgreSQL AST operations | • Parse SQL to AST<br>• Deparse AST to SQL<br>• Generate TypeScript from protobuf<br>• Single tool for all operations |
| [**@pgsql/utils**](./packages/utils) | Type-safe AST node creation utilities | • Programmatic AST construction<br>• Runtime Schema<br>• Seamless integration with types |
| [**pg-proto-parser**](./packages/proto-parser) | PostgreSQL protobuf parser and code generator | • Generate TypeScript interfaces from protobuf<br>• Create enum mappings and utilities<br>• AST helper generation |

# Generate TypeScript from protobuf
pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums
```

## 🛠️ Development

Expand Down
30 changes: 30 additions & 0 deletions __fixtures__/generated/generated.json
Original file line number Diff line number Diff line change
Expand Up @@ -21181,6 +21181,15 @@
"original/alter/alter-95.sql": "ALTER TABLE mytable ADD COLUMN height_in numeric GENERATED ALWAYS AS (height_cm / 2.54) STORED",
"original/alter/alter-96.sql": "ALTER SCHEMA schemaname RENAME TO newname",
"original/alter/alter-97.sql": "ALTER SCHEMA schemaname OWNER TO newowner",
"original/alter/alter-table-column-1.sql": "ALTER TABLE public.table1 ALTER COLUMN id ADD GENERATED ALWAYS AS IDENTITY (\n SEQUENCE NAME public.table1\n START WITH 1\n INCREMENT BY 1\n NO MINVALUE\n NO MAXVALUE\n CACHE 1\n)",
"original/alter/alter-table-column-2.sql": "ALTER TABLE public.sales\nADD COLUMN total_price NUMERIC GENERATED ALWAYS AS (quantity * unit_price) STORED",
"original/alter/alter-table-column-3.sql": "ALTER TABLE public.comments\nADD COLUMN post_id INTEGER NOT NULL REFERENCES public.posts(id) ON DELETE CASCADE",
"original/alter/alter-table-column-4.sql": "ALTER TABLE public.devices\nADD COLUMN device_token UUID UNIQUE DEFAULT gen_random_uuid()",
"original/alter/alter-table-column-5.sql": "ALTER TABLE public.products\nADD COLUMN product_id BIGINT GENERATED BY DEFAULT AS IDENTITY (\n START WITH 5000\n INCREMENT BY 10\n)",
"original/alter/alter-table-column-6.sql": "ALTER TABLE public.users\nADD COLUMN name TEXT COLLATE \"fr_FR\"",
"original/alter/alter-table-column-7.sql": "ALTER TABLE public.books\nADD COLUMN tags TEXT[] DEFAULT '{}'",
"original/alter/alter-table-column-8.sql": "CREATE TYPE mood AS ENUM ('happy', 'sad', 'neutral')",
"original/alter/alter-table-column-9.sql": "ALTER TABLE public.profiles\nADD COLUMN current_mood mood DEFAULT 'neutral'",
"misc/quotes_etc-1.sql": "CREATE USER MAPPING FOR local_user SERVER \"foreign_server\" OPTIONS (user 'remote_user', password 'secret123')",
"misc/quotes_etc-2.sql": "CREATE USER MAPPING FOR local_user SERVER foreign_server OPTIONS (user 'remote_user', password 'secret123')",
"misc/quotes_etc-3.sql": "SELECT E'Line 1\\nLine 2'",
Expand Down Expand Up @@ -21211,6 +21220,23 @@
"misc/quotes_etc-28.sql": "DO $$\nBEGIN\n RAISE NOTICE 'Line one\\nLine two';\nEND;\n$$ LANGUAGE plpgsql",
"misc/quotes_etc-29.sql": "CREATE USER MAPPING FOR local_user SERVER \"foreign_server\" OPTIONS (user 'remote_user', password 'secret123')",
"misc/quotes_etc-30.sql": "CREATE USER MAPPING FOR local_user SERVER foreign_server OPTIONS (user 'remote_user', password 'secret123')",
"misc/pg_catalog-1.sql": "SELECT json_object('{}')",
"misc/pg_catalog-2.sql": "SELECT * FROM generate_series(1, 5)",
"misc/pg_catalog-3.sql": "SELECT get_byte(E'\\\\xDEADBEEF'::bytea, 1)",
"misc/pg_catalog-4.sql": "SELECT now()",
"misc/pg_catalog-5.sql": "SELECT clock_timestamp()",
"misc/pg_catalog-6.sql": "SELECT to_char(now(), 'YYYY-MM-DD HH24:MI:SS')",
"misc/pg_catalog-7.sql": "SELECT json_build_object('name', 'Alice', 'age', 30)",
"misc/pg_catalog-8.sql": "SELECT pg_typeof(42), pg_typeof('hello'), pg_typeof(now())",
"misc/pg_catalog-9.sql": "SELECT substring('abcdefg' FROM 2 FOR 3)",
"misc/pg_catalog-10.sql": "SELECT replace('hello world', 'l', 'L')",
"misc/pg_catalog-11.sql": "SELECT length('yolo')",
"misc/pg_catalog-12.sql": "SELECT position('G' IN 'ChatGPT')",
"misc/pg_catalog-13.sql": "SELECT trim(' padded text ')",
"misc/pg_catalog-14.sql": "SELECT ltrim('---abc', '-')",
"misc/pg_catalog-15.sql": "SELECT array_agg(id) FROM (VALUES (1), (2), (3)) AS t(id)",
"misc/pg_catalog-16.sql": "SELECT string_agg(name, ', ') FROM (VALUES ('Alice'), ('Bob'), ('Carol')) AS t(name)",
"misc/pg_catalog-17.sql": "SELECT json_agg(name) FROM (VALUES ('A'), ('B')) AS t(name)",
"misc/launchql-ext-types-1.sql": "CREATE DOMAIN attachment AS jsonb CHECK ( value ?& ARRAY['url', 'mime'] AND (value->>'url') ~ '^(https?)://[^\\s/$.?#].[^\\s]*$' )",
"misc/launchql-ext-types-2.sql": "COMMENT ON DOMAIN attachment IS E'@name launchqlInternalTypeAttachment'",
"misc/launchql-ext-types-3.sql": "CREATE DOMAIN email AS citext CHECK ( value ~ '^[a-zA-Z0-9.!#$%&''*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$' )",
Expand Down Expand Up @@ -21889,6 +21915,10 @@
"latest/postgres/create_table-39.sql": "DROP FUNCTION plusone(INT)",
"latest/postgres/create_table-40.sql": "DROP TYPE comp_type",
"latest/postgres/create_table-41.sql": "DROP DOMAIN posint",
"latest/postgres/create_table-42.sql": "CREATE TABLE generated_cols (\n a INT,\n b INT GENERATED ALWAYS AS (a * 2) STORED\n)",
"latest/postgres/create_table-43.sql": "CREATE TYPE comp_type AS (x INT, y TEXT)",
"latest/postgres/create_table-44.sql": "CREATE TABLE uses_comp (\n id INT,\n data comp_type\n)",
"latest/postgres/create_table-45.sql": "CREATE TABLE public.users (\n user_id INTEGER GENERATED ALWAYS AS IDENTITY (\n START WITH 1000\n INCREMENT BY 5\n CACHE 10\n ),\n username TEXT NOT NULL,\n created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP\n)",
"latest/postgres/create_schema-1.sql": "CREATE ROLE regress_create_schema_role SUPERUSER",
"latest/postgres/create_schema-2.sql": "CREATE SCHEMA AUTHORIZATION regress_create_schema_role\n CREATE SEQUENCE schema_not_existing.seq",
"latest/postgres/create_schema-3.sql": "CREATE SCHEMA AUTHORIZATION regress_create_schema_role\n CREATE TABLE schema_not_existing.tab (id int)",
Expand Down
25 changes: 25 additions & 0 deletions __fixtures__/kitchen-sink/latest/postgres/create_table.sql
Original file line number Diff line number Diff line change
Expand Up @@ -126,3 +126,28 @@ CREATE TABLE mlvl_leaf PARTITION OF mlvl_sub FOR VALUES FROM (1) TO (10);
DROP FUNCTION plusone(INT);
DROP TYPE comp_type;
DROP DOMAIN posint;

-- generated columns
CREATE TABLE generated_cols (
a INT,
b INT GENERATED ALWAYS AS (a * 2) STORED
);

-- composite types
CREATE TYPE comp_type AS (x INT, y TEXT);
CREATE TABLE uses_comp (
id INT,
data comp_type
);

-- generated columns
CREATE TABLE public.users (
user_id INTEGER GENERATED ALWAYS AS IDENTITY (
START WITH 1000
INCREMENT BY 5
CACHE 10
),
username TEXT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

34 changes: 34 additions & 0 deletions __fixtures__/kitchen-sink/original/alter/alter-table-column.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
ALTER TABLE public.table1 ALTER COLUMN id ADD GENERATED ALWAYS AS IDENTITY (
SEQUENCE NAME public.table1
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1
);

ALTER TABLE public.sales
ADD COLUMN total_price NUMERIC GENERATED ALWAYS AS (quantity * unit_price) STORED;

ALTER TABLE public.comments
ADD COLUMN post_id INTEGER NOT NULL REFERENCES public.posts(id) ON DELETE CASCADE;

ALTER TABLE public.devices
ADD COLUMN device_token UUID UNIQUE DEFAULT gen_random_uuid();

ALTER TABLE public.products
ADD COLUMN product_id BIGINT GENERATED BY DEFAULT AS IDENTITY (
START WITH 5000
INCREMENT BY 10
);

ALTER TABLE public.users
ADD COLUMN name TEXT COLLATE "fr_FR";

ALTER TABLE public.books
ADD COLUMN tags TEXT[] DEFAULT '{}';

CREATE TYPE mood AS ENUM ('happy', 'sad', 'neutral');

ALTER TABLE public.profiles
ADD COLUMN current_mood mood DEFAULT 'neutral';
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ it('latest-postgres-create_table', async () => {
"latest/postgres/create_table-38.sql",
"latest/postgres/create_table-39.sql",
"latest/postgres/create_table-40.sql",
"latest/postgres/create_table-41.sql"
"latest/postgres/create_table-41.sql",
"latest/postgres/create_table-42.sql",
"latest/postgres/create_table-43.sql",
"latest/postgres/create_table-44.sql",
"latest/postgres/create_table-45.sql"
]);
});
25 changes: 25 additions & 0 deletions packages/deparser/__tests__/kitchen-sink/misc-pg_catalog.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@

import { FixtureTestUtils } from '../../test-utils';
const fixtures = new FixtureTestUtils();

it('misc-pg_catalog', async () => {
await fixtures.runFixtureTests([
"misc/pg_catalog-1.sql",
"misc/pg_catalog-2.sql",
"misc/pg_catalog-3.sql",
"misc/pg_catalog-4.sql",
"misc/pg_catalog-5.sql",
"misc/pg_catalog-6.sql",
"misc/pg_catalog-7.sql",
"misc/pg_catalog-8.sql",
"misc/pg_catalog-9.sql",
"misc/pg_catalog-10.sql",
"misc/pg_catalog-11.sql",
"misc/pg_catalog-12.sql",
"misc/pg_catalog-13.sql",
"misc/pg_catalog-14.sql",
"misc/pg_catalog-15.sql",
"misc/pg_catalog-16.sql",
"misc/pg_catalog-17.sql"
]);
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@

import { FixtureTestUtils } from '../../test-utils';
const fixtures = new FixtureTestUtils();

it('original-alter-alter-table-column', async () => {
await fixtures.runFixtureTests([
"original/alter/alter-table-column-1.sql",
"original/alter/alter-table-column-2.sql",
"original/alter/alter-table-column-3.sql",
"original/alter/alter-table-column-4.sql",
"original/alter/alter-table-column-5.sql",
"original/alter/alter-table-column-6.sql",
"original/alter/alter-table-column-7.sql",
"original/alter/alter-table-column-8.sql",
"original/alter/alter-table-column-9.sql"
]);
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`non-pretty: original/alter/alter-table-column-1.sql 1`] = `"ALTER TABLE public.table1 ALTER COLUMN id ADD GENERATED ALWAYS AS IDENTITY (SEQUENCE NAME public.table1 START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1)"`;

exports[`non-pretty: original/alter/alter-table-column-2.sql 1`] = `"ALTER TABLE public.sales ADD COLUMN total_price numeric GENERATED ALWAYS AS (quantity * unit_price) STORED"`;

exports[`non-pretty: original/alter/alter-table-column-3.sql 1`] = `"ALTER TABLE public.comments ADD COLUMN post_id int NOT NULL REFERENCES public.posts (id) ON DELETE CASCADE"`;

exports[`non-pretty: original/alter/alter-table-column-4.sql 1`] = `"ALTER TABLE public.devices ADD COLUMN device_token uuid UNIQUE DEFAULT gen_random_uuid()"`;

exports[`non-pretty: original/alter/alter-table-column-5.sql 1`] = `"ALTER TABLE public.products ADD COLUMN product_id bigint GENERATED BY DEFAULT AS IDENTITY (START WITH 5000 INCREMENT BY 10)"`;

exports[`non-pretty: original/alter/alter-table-column-6.sql 1`] = `"ALTER TABLE public.users ADD COLUMN name text COLLATE "fr_FR""`;

exports[`non-pretty: original/alter/alter-table-column-7.sql 1`] = `"ALTER TABLE public.books ADD COLUMN tags text[] DEFAULT '{}'"`;

exports[`non-pretty: original/alter/alter-table-column-8.sql 1`] = `"CREATE TYPE mood AS ENUM ('happy', 'sad', 'neutral')"`;

exports[`non-pretty: original/alter/alter-table-column-9.sql 1`] = `"ALTER TABLE public.profiles ADD COLUMN current_mood mood DEFAULT 'neutral'"`;

exports[`pretty: original/alter/alter-table-column-1.sql 1`] = `
"ALTER TABLE public.table1
ALTER COLUMN id ADD GENERATED ALWAYS AS IDENTITY (
SEQUENCE NAME public.table1
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1
)"
`;

exports[`pretty: original/alter/alter-table-column-2.sql 1`] = `
"ALTER TABLE public.sales
ADD COLUMN total_price numeric
GENERATED ALWAYS AS (quantity * unit_price) STORED"
`;

exports[`pretty: original/alter/alter-table-column-3.sql 1`] = `
"ALTER TABLE public.comments
ADD COLUMN post_id int
NOT NULL
REFERENCES public.posts (id)
ON DELETE CASCADE"
`;

exports[`pretty: original/alter/alter-table-column-4.sql 1`] = `
"ALTER TABLE public.devices
ADD COLUMN device_token uuid
UNIQUE
DEFAULT gen_random_uuid()"
`;

exports[`pretty: original/alter/alter-table-column-5.sql 1`] = `
"ALTER TABLE public.products
ADD COLUMN product_id bigint
GENERATED BY DEFAULT AS IDENTITY (
START WITH 5000
INCREMENT BY 10
)"
`;

exports[`pretty: original/alter/alter-table-column-6.sql 1`] = `
"ALTER TABLE public.users
ADD COLUMN name text
COLLATE "fr_FR""
`;

exports[`pretty: original/alter/alter-table-column-7.sql 1`] = `
"ALTER TABLE public.books
ADD COLUMN tags text[]
DEFAULT '{}'"
`;

exports[`pretty: original/alter/alter-table-column-8.sql 1`] = `"CREATE TYPE mood AS ENUM ('happy', 'sad', 'neutral')"`;

exports[`pretty: original/alter/alter-table-column-9.sql 1`] = `
"ALTER TABLE public.profiles
ADD COLUMN current_mood mood
DEFAULT 'neutral'"
`;
14 changes: 14 additions & 0 deletions packages/deparser/__tests__/pretty/alter-table-column.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { PrettyTest } from '../../test-utils/PrettyTest';
const prettyTest = new PrettyTest([
"original/alter/alter-table-column-1.sql",
"original/alter/alter-table-column-2.sql",
"original/alter/alter-table-column-3.sql",
"original/alter/alter-table-column-4.sql",
"original/alter/alter-table-column-5.sql",
"original/alter/alter-table-column-6.sql",
"original/alter/alter-table-column-7.sql",
"original/alter/alter-table-column-8.sql",
"original/alter/alter-table-column-9.sql"
]);

prettyTest.generateTests();
Loading