Crystal Structure Search Application

Website: crystallography.io

Architecture

Microservice / event-driven architecture for crystal structure search and analysis. Data sourced from COD (Crystallography Open Database).

Stack: TypeScript, Node.js 22, Express, React 18, Vite, Zustand, MongoDB, Redis, RabbitMQ, Docker Swarm, Traefik

Containers

Package	Description
@chemistry/c14-api	REST API for COD+ database access
@chemistry/c14-web	Web UI with server-side rendering
@chemistry/cod-to-disk	Sync CIF files from COD via rsync
@chemistry/disk-to-db	Parse CIF files and store in MongoDB
@chemistry/structure-to-index	Build search indexes (authors, names, formulas, fragments)
@chemistry/maintenance	Scheduled tasks (catalog, sitemap generation)
@chemistry/searchrouter	Substructure search API (BullMQ + Socket.IO)
@chemistry/searchworker	Substructure search worker (molecule matching)

Libraries

Package	Description
@chemistry/cif-2-json	CIF to JSON parser

Data Pipeline

COD (rsync) → cod-to-disk → disk-to-db → MongoDB
                                ↓
                        structure-to-index → authors, names, formulas, fragments
                                ↓
                           maintenance → catalog, sitemap

Development

npm install          # Install all workspaces
npm run build        # TypeScript composite build
npm run lint         # ESLint check
npm run test         # Vitest
npm run verify       # build + lint + test
npm run dev          # Start c14-web with Vite dev server

Deployment

Fully automated via GitHub Actions — no manual SSH required.

Workflow	Trigger	Purpose
ci.yml	Every push	Build + lint + test
pr-validation.yml	PR to master	CI + Docker build validation
deploy.yml	Push to master / manual	Build images → deploy to Docker Swarm
sync-cod.yml	Manual / weekly cron	Rsync CIF files from COD
seed-import.yml	Manual	Queue CIF files for initial data import
server-status.yml	Manual	Check services and resources

Deploy to a new server

1. Set GitHub secrets: DEPLOY_HOST, DEPLOY_SSH_KEY, MONGO_PASSWORD, REDIS_PASSWORD, SENTRY_DSN
2. Trigger deploy.yml — stack is up
3. Trigger sync-cod.yml — CIF files synced (~532K files, ~111 GB)
4. Trigger seed-import.yml — data pipeline populates MongoDB
5. Point DNS → server, Traefik auto-generates SSL

Library Release

git tag l0.0.18 && git push --tags

Triggers release.yml → npm publish for @chemistry/cif-2-json.

License

This project is licensed under the MIT license, Copyright (c) 2020 Volodymyr Vreshch. For more information see LICENSE.