Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
486dc48
feat: add smoke test script for sql-assessment-service
simonmysun Apr 15, 2026
e92ced8
docs: update README
simonmysun Apr 15, 2026
cbec7f1
feat: implement CLI functionality for sql-assessment-service
simonmysun Apr 9, 2026
061094f
feat: implement CLI functionality for josnpath-mapper-service
simonmysun Apr 9, 2026
8243d8b
feat: reduce Docker image size by using multi-stage build and excludi…
simonmysun Apr 9, 2026
bcf5fe6
chore: switch to esbuild and minimize docker image size
simonmysun Apr 9, 2026
1b3bd40
fix: update exposed port in Dockerfile from 3333 to 3000
simonmysun Apr 9, 2026
54c5f8d
fix: add missing dev dependency ts-node
simonmysun Apr 9, 2026
7caba67
chore: switch to esbuild and minimize docker image size
simonmysun Apr 10, 2026
c8aeb97
refactor: separate neo4j into its own service and remove supervisord
simonmysun Apr 10, 2026
976c22c
feat: implement CLI for graph rewriting service
simonmysun Apr 10, 2026
c68d7d2
chore: switch to esbuild for faster builds and smaller runtime image
simonmysun Apr 10, 2026
ef98e83
fix: use esbuild to copy @fastify/swagger-ui static files for runtime…
simonmysun Apr 10, 2026
a0b1fca
feat: enhance build process to inject package version and description…
simonmysun Apr 10, 2026
bb475b2
Apply suggestions from code review
simonmysun Apr 10, 2026
193292b
feat: add tests for DatabaseController and QueryExecutionController w…
simonmysun Apr 15, 2026
d80fb5a
feat: integrate PGlite support for database analysis and query execution
simonmysun Apr 15, 2026
40d89e2
feat: add in-memory graph database backend
simonmysun Apr 20, 2026
13a177e
feat: add smoke test script and update Makefile and README
simonmysun Apr 20, 2026
0a987a2
refactor(api): introduce implicit database initialization for depende…
simonmysun Apr 21, 2026
43f841d
feat(cli): add CLI tests for Neo4j backend functionality
simonmysun Apr 21, 2026
d2e84f6
feat(cli): include cli tests in smoke test
simonmysun Apr 21, 2026
ab44151
fix: fixes linting and formatting issues in cli.test.ts in graph-rewr…
simonmysun Apr 22, 2026
b1ced22
fix(tests): warm up PGlite WASM to reduce cold-start costs and extend…
simonmysun Apr 22, 2026
9df06de
refactor: update dbGraphService type to IGraphDB in memory and neo4j …
simonmysun Apr 22, 2026
5f91da7
feat(graph): integrate @grafeo-db/js for enhanced graph database func…
simonmysun Apr 22, 2026
04dd404
fix: remove eslint-disable comments from generated schema files
simonmysun Apr 22, 2026
d14f47f
fix: update createNode method to use createNodeUuid for internalId ge…
simonmysun Apr 22, 2026
1051d15
fix: update error message for missing connection info to use internat…
simonmysun Apr 22, 2026
20406be
fix: update error message for missing database initialization script …
simonmysun Apr 22, 2026
3bb0d8a
fix: update DatabaseService to use fs/promises for asynchronous file …
simonmysun Apr 22, 2026
e929786
fix: update README to correct environment variable name for OpenAI AP…
simonmysun Apr 22, 2026
e159a71
fix: update CLI command in README for sql generation to reflect new p…
simonmysun Apr 22, 2026
fed4621
fix: remove eslint-disable comments for explicit any type in database…
simonmysun Apr 22, 2026
b847620
fix: remove unnecessary whitespace
simonmysun Apr 22, 2026
358e8d2
fix: validate PostgreSQL connection info before generating database key
simonmysun Apr 22, 2026
376968e
fix: remove unnecessary blank lines in DatabaseService methods
simonmysun Apr 22, 2026
0921e13
fix: update Dockerfile to use node:22-slim for compatibility and modi…
simonmysun Apr 22, 2026
3242c49
fix: remove unnecessary blank line
simonmysun Apr 22, 2026
9343501
fix: correct database key format in generateDatabaseKey function
simonmysun Apr 22, 2026
d5d7180
fix: remove unnecessary blank line
simonmysun Apr 22, 2026
cd16c25
fix: enhance createEdge method to validate endpoints and return error…
simonmysun Apr 23, 2026
84063d6
fix: pass aliasMap to analyzePGlite method for improved schema extrac…
simonmysun Apr 23, 2026
234c18d
fix: remove unnecessary blank line
simonmysun Apr 23, 2026
c7ca1ab
fix(sql-assessment): correctly handle composite UNIQUE constraints fo…
simonmysun Apr 23, 2026
dde0e4e
fix(postgres): remove unused Dockerfile for PostgreSQL 13
simonmysun Apr 23, 2026
7fc6a34
fix: share single PGlite instance for init-SQL-backed databases
simonmysun Apr 23, 2026
714916d
fix(cli): exit with error when --init-sql-file is missing its path ar…
simonmysun Apr 23, 2026
526dc28
fix: load dotenv before createControllers() in REST API entrypoint
simonmysun Apr 23, 2026
99c9857
fix(sql-assessment): populate indices from pg_catalog in PGlite schem…
simonmysun Apr 23, 2026
9ad2a7a
fix: copy swagger-ui-express/dist into runtime image and mark as exte…
simonmysun Apr 24, 2026
4ed3954
fix: pin uv version to 0.11.7 in Dockerfile build stage
simonmysun Apr 24, 2026
bf023ab
fix(test): format expect statement for clarity in resolveInitSqlFile …
simonmysun Apr 24, 2026
5b4fa88
fix: enable PGlite in task-generation by moving validateConnectionInf…
simonmysun Apr 24, 2026
f3f480e
fix(grading): move validateConnectionInfo after auto-analyze; reject …
simonmysun Apr 24, 2026
855e9e6
fix(database-service): coalesce concurrent init-SQL PGlite initialisa…
simonmysun Apr 24, 2026
ee0bc18
refactor(cli): replace Math.max flag lookup with explicit resolveFile…
simonmysun Apr 24, 2026
3834dd3
fix(docker): bundle swagger-ui-express to resolve missing express at …
simonmysun Apr 24, 2026
c65af06
fix(query): forward unhandled executeQuery rejections to next() in ro…
simonmysun Apr 24, 2026
3a0c098
refactor(query-execution-controller): format code for better readability
simonmysun Apr 24, 2026
fb00df9
feat(graph-service): add deleteEdge method to return undefined for mi…
simonmysun Apr 24, 2026
1b0e3db
fix(database-service): treat explicit empty sqlContent as invalid, no…
simonmysun Apr 24, 2026
ac7a14f
fix(test): derive DB_KEY via generateDatabaseKey to match localhost:5…
simonmysun Apr 24, 2026
9d24ef1
test(query): fix EMPTY_QUERY test to actually invoke the controller a…
simonmysun Apr 25, 2026
d150f44
fix(cli): scan past flags to find positional JSON body; error on -f/-…
simonmysun Apr 25, 2026
c4c410a
refactor(test): simplify assertion for flags-only arguments in resolv…
simonmysun Apr 25, 2026
ec9d74a
fix(docker): remove default NEO4J_PASSWORD to prevent weak-credential…
simonmysun Apr 26, 2026
4dced91
fix(memory): update onRequest hook to ignore unused reply parameter
simonmysun Apr 26, 2026
042edaf
refactor(database): type pgliteInstances as Map<string, PGlite>
simonmysun Apr 26, 2026
a7952fc
refactor(database): change sqlContent variable declaration to const
simonmysun Apr 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions services/fermentaladin-service/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
test/
http-tests/
*.md
.gitignore
.demo.env
Makefile
__pycache__/
.ruff_cache/
Comment thread
simonmysun marked this conversation as resolved.
.git/
.venv/
32 changes: 15 additions & 17 deletions services/fermentaladin-service/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,26 +1,24 @@
FROM python:3.12-slim-trixie

# The installer requires curl (and certificates) to download the release archive
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# ---- build stage ----
FROM python:3.12-slim-trixie AS builder

# Download the latest installer
ADD https://astral.sh/uv/install.sh /uv-installer.sh
COPY --from=ghcr.io/astral-sh/uv:0.11.7 /uv /uvx /bin/

# Run the installer then remove it
RUN sh /uv-installer.sh && rm /uv-installer.sh
COPY . /app
WORKDIR /app

# Ensure the installed binary is on the `PATH`
ENV PATH="/root/.local/bin/:$PATH"
# Sync into the project virtual environment; exclude dev dependencies
RUN uv sync --locked --no-dev

# Copy the project into the image
COPY . /app
# ---- runtime stage ----
FROM python:3.12-slim-trixie

# Sync the project into a new environment, asserting the lockfile is up to date
# --no-dev: exclude dev dependencies (pytest, ruff) from the production image
# Copy only the application code and the ready-made virtual environment
COPY --from=builder /app /app
WORKDIR /app
RUN uv sync --locked --no-dev

# Place the virtual-env Python first on PATH so we don't need `uv run`
ENV PATH="/app/.venv/bin:$PATH"

EXPOSE 8000

CMD ["uv", "run", "uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000", "--app-dir", "src"]
CMD ["uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000", "--app-dir", "src"]
18 changes: 18 additions & 0 deletions services/fermentaladin-service/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,24 @@ uv run src/main.py -i excel -p ./test/Bioreactor_forPy.xlsx -o df -f ausgabe.jso

This reads the model parameters from file ./test/Bioreactor_forPy.xlsx and creates a file ausgabe.json in a Dataframe format

### Usage inside Docker

The Docker image uses a multi-stage build and does not include `uv`. Instead, the virtual environment is already activated via `PATH`, so you run `python` directly instead of `uv run`:

```sh
# local
uv run src/main.py -i excel -p input.xlsx -o df -f ausgabe.json

# inside Docker
python src/main.py -i excel -p input.xlsx -o df -f ausgabe.json
```

Example with `docker run`:

```sh
docker run --rm fermentaladin-service python src/main.py -i file -p ./test/test_data/user_input.json -o df -f ausgabe.json
```

## Architecture

### Input
Expand Down
15 changes: 10 additions & 5 deletions services/graph-rewriting-service/.env.example
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
# App Environment. Can be one of: development, testing, production. Defaults to "development".
APP_ENV=development
# Application environment. Can be one of: development, production. Defaults to "production".
APP_ENV=production

# Connection data for neo4j docker container
NEO4J_URI=bolt://localhost:7687
# Database backend: "memory" (default, no external DB needed) or "neo4j"
DB_BACKEND=memory

# Neo4j connection configuration (only needed when DB_BACKEND=neo4j)
# In docker-compose, the host is 'neo4j' (the service name).
# When running locally, typically 'localhost'.
NEO4J_URI=bolt://neo4j:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=
NEO4J_PASSWORD=your-secure-password-here
67 changes: 31 additions & 36 deletions services/graph-rewriting-service/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# ── Stage 1: builder ─────────────────────────────────────────────────────────
# Compiles TypeScript to JavaScript and installs production-only dependencies.
# Bundles TypeScript to JavaScript via esbuild.
FROM node:22-slim AS builder

WORKDIR /build
Expand All @@ -8,52 +8,47 @@ WORKDIR /build
COPY package.json package-lock.json ./
RUN npm ci

# Copy source and compile
# Copy source and build
COPY tsconfig.json ./
COPY scripts/ ./scripts/
COPY src/ ./src/
RUN npm run build

# Prune to production deps only
RUN npm ci --omit=dev

# ── Stage 2: runtime ──────────────────────────────────────────────────────────
# Uses the official Neo4j image as base so Neo4j is available out of the box.
# Node 22 is installed via NodeSource.
FROM neo4j:5.25.1

# Install Node.js 22 (NodeSource) and supervisord
RUN apt-get update && \
apt-get install -y --no-install-recommends curl ca-certificates supervisor && \
curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
apt-get install -y --no-install-recommends nodejs && \
apt-get clean && rm -rf /var/lib/apt/lists/*

# Copy compiled app and production node_modules from builder
# @grafeo-db/js ships a native Rust addon compiled against glibc (gnu).
# Using node:22-slim (Debian/glibc) keeps the same libc as the builder so the
# native binding works. node:22-alpine (musl) would require the -musl variant.
FROM node:22-slim

WORKDIR /app
COPY --from=builder /build/dist ./dist
COPY --from=builder /build/node_modules ./node_modules

# Copy process manager config and entrypoint
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY scripts/entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
# Copy bundled app output from builder
COPY --from=builder /build/dist ./dist

# Create log directory for supervisord
RUN mkdir -p /var/log/supervisor
# @grafeo-db/js ships a native Rust addon (.node file) that cannot be bundled
# by esbuild and is therefore externalized. Copy the packages so both the
# in-memory and neo4j backends are available at runtime.
COPY --from=builder /build/node_modules/@grafeo-db ./node_modules/@grafeo-db

# ── Environment defaults ───────────────────────────────────────────────────────
# NEO4J_AUTH is constructed in entrypoint.sh from USERNAME + PASSWORD.
# NEO4J_PASSWORD has a default, but should be supplied at runtime.
# DB_BACKEND defaults to "memory" (no external DB needed).
# Set DB_BACKEND=neo4j to use Neo4j and supply all three NEO4J_* vars;
# NEO4J_PASSWORD has no default and must be provided explicitly.
ENV APP_ENV=production \
NEO4J_URI=bolt://localhost:7687 \
NEO4J_USERNAME=neo4j \
NEO4J_PASSWORD=graph_rewriting
NODE_ENV=production \
NEO4J_URI=bolt://neo4j:7687 \
NEO4J_USERNAME=neo4j


# ── Ports ──────────────────────────────────────────────────────────────────────
# ── Port ────────────────────────────────────────────────────────────────────────
# 8080 – Fastify HTTP API
# 7474 – Neo4j Browser UI
# 7687 – Neo4j Bolt
EXPOSE 8080 7474 7687
EXPOSE 8080

# Health check (optional)
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD node -e "require('http').get('http://localhost:8080/health', (r) => {if (r.statusCode !== 200) throw new Error(r.statusCode)})"
Comment thread
simonmysun marked this conversation as resolved.

# CLI is also available inside the container:
# docker run --rm -e NEO4J_URI=... graph-rewriting-service node dist/cli.js transform /data/request.json

ENTRYPOINT ["/bin/bash", "/entrypoint.sh"]
# Start API server by default
CMD ["node", "dist/index.js"]
5 changes: 4 additions & 1 deletion services/graph-rewriting-service/Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: build test lint start clean compile-schemas generate-openapi docker-build
.PHONY: build test lint start clean compile-schemas generate-openapi docker-build smoke-test

prep:
npm i
Expand Down Expand Up @@ -28,3 +28,6 @@ generate-openapi:

docker-build:
docker build -t graph-rewriting-service .

smoke-test:
./scripts/smoke-test.sh
124 changes: 120 additions & 4 deletions services/graph-rewriting-service/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

## Prerequisites

This repository contains all files necessary to run the application in a local development environment.
It does depend on Docker and Node.js being available on your system.
This repository contains all files necessary to run the application in local development and production environments.
It requires Docker and Node.js 22+ on your system.

When cloning this repository and opening it through VSCode, it will ask you to install all recommended VSCode extensions.

Expand All @@ -15,12 +15,86 @@ In order to install and run this application in your local development environme

Next an .env file should be created by copying the .env.example and setting the appropriate values.

## Running the application
### Development Environment

You may start the project by running the tasks `docker compose up` and `npm run dev` in your terminal.
The application supports two database backends:

#### In-Memory (default, no Neo4j needed)

Simply start the application — no external database required:

```bash
npm run dev
```

#### With Neo4j

To run with Neo4j as the database backend:

1. Start Neo4j container:
```bash
docker compose -f docker-compose.dev.yml up
```

2. In another terminal, start the application with neo4j backend:
```bash
DB_BACKEND=neo4j npm run dev
```

If using VSCode you can instead run the VSCode-Task `Start dev environment` as a shortcut.

### Production Environment (Docker + Compose)

To run the application in production using Docker containers:

```bash
# Set required environment variables
export NEO4J_PASSWORD=your-secure-password

# Start both Neo4j and the application service
docker compose -f docker-compose.prod.yml up -d
```

The application will be available at `http://localhost:8080`.
Neo4j Browser is accessible at `http://localhost:7474`.

**Note:** As of this version, the Docker image is split into two services:
- `graph-rewriting-service`: Node.js API server (port 8080)
- `neo4j`: Graph database (ports 7687 for Bolt, 7474 for Browser)

This separation reduces image size and enables independent scaling.

## Architecture

### Build Pipeline

- `npm run build` now bundles the service with esbuild.
- `npm run typecheck` runs TypeScript checks without emitting files.
- The build emits bundled artifacts to `dist/index.js` and `dist/cli.js`.
- Static assets from `src/static` are copied to `dist/static` during the build.

### Docker Images

- **graph-rewriting-service**: Lightweight Node.js container based on `node:22-alpine`
- Runs the Fastify HTTP API
- Connects to Neo4j via environment variables
- Contains bundled runtime artifacts only (no `node_modules` in the final image)
- No embedded database or process manager

- **neo4j** (from official image): Graph database service
- Provides pattern matching and graph transformation operations
- Can be deployed separately or in the same compose stack

### Environment Variables

| Variable | Default | Required | Description |
|----------|---------|----------|-------------|
| `APP_ENV` | `production` | No | Application environment: `development`, `production` |
| `DB_BACKEND` | `memory` | No | Database backend: `memory` (in-memory, no external DB) or `neo4j` |
| `NEO4J_URI` | `bolt://neo4j:7687` | No | Neo4j Bolt connection URI (only when `DB_BACKEND=neo4j`) |
| `NEO4J_USERNAME` | `neo4j` | No | Neo4j username (only when `DB_BACKEND=neo4j`) |
| `NEO4J_PASSWORD` | (none) | When neo4j | Neo4j password (must be set when using neo4j backend) |

## Documentation

For documentation of the rewrite and instantiation rules, please refer to the [Wiki](https://github.com/sonjaka/graph-rewriting-as-a-service/wiki).
Expand Down Expand Up @@ -52,6 +126,48 @@ A very simple TicTacToe game againt a computer player powered by graph transform
This testcase consists of a very basic web app built on the Vue.js Framework.
You can install the project by first running `npm install`, then `npm run dev`.

## Testing

### Unit Tests

```bash
npm test
```

### Smoke Tests

End-to-end smoke tests using `curl` against a running service.

#### Memory backend only (no external dependencies)

```bash
make smoke-test
```

#### Memory + Neo4j backend

Start a Neo4j container first, then run the smoke test with Neo4j credentials:

```bash
# Start Neo4j
NEO4J_USERNAME=neo4j NEO4J_PASSWORD=<your-password> \
docker compose -f docker-compose.dev.yml up -d neo4j

# Run smoke tests for both backends
NEO4J_URI=bolt://localhost:7687 \
NEO4J_USERNAME=neo4j \
NEO4J_PASSWORD=<your-password> \
make smoke-test
```

When `NEO4J_URI` is not set, the Neo4j portion of the smoke test is skipped automatically.

#### Teardown

```bash
docker compose -f docker-compose.dev.yml down
```

## SwaggerUI / OpenAPI

When the server is running, you can access the SwaggerUI / OpenAPI documentation via the following url:
Expand Down
Loading
Loading