Skip to content

Migrate database layer from TigerGraph to PostgreSQL for local development convenience #7

Description

@Ishwarpatra

Description

The project currently relies on TigerGraph Cloud (using the pyTigerGraph library) to persist and query city graph data. While TigerGraph is a powerful graph database, relying on it for local development creates several friction points for contributors:

  • It requires credentials/tokens for TigerGraph Cloud (the hardcoded token in config.py is set to expire).
  • Local testing is dependent on external internet access and server uptime of the TigerGraph Cloud instance.
  • Setup is complex and prone to permission/network issues on corporate or institutional networks.

Replacing the database layer (or offering a parallel backend) using a lightweight SQL database like PostgreSQL (with optional PostGIS for location querying) would make the project much easier to set up, run offline, and containerize.

Affected Files

  • backend_python/config.py — connection settings and db flags
  • backend_python/custom_db/tigergraph_client.py— client wrapper class and helper functions
  • backend_python/custom_db/__init__.py — database module imports
  • backend_python/requirements.txt — dependencies list (remove pyTigerGraph, add psycopg2-binary or asyncpg)
    -data/load_to_tg.py & data/tigergraph_loader.py — initial mock data ingestion scripts

Expected Behavior

  1. Support for a relational database backend (like PostgreSQL) to store intersection nodes, edges/segments, and incidents.
  2. A database configuration flag (e.g., USE_POSTGRES or DB_TYPE="postgres") in config.py.
  3. Relational equivalents for all high-level database helper functions exposed in custom_db/__init__.py:
    • get_client()
    • upsert_incident_to_graph(payload: dict)
    • get_zone_data(zone_id: str)
    • update_zone_danger_score(zone_id: str, danger_score: float)
  4. A data migration script (e.g., data/load_to_postgres.py) to seed the PostgreSQL database from local raw JSON graph data.

Suggested Implementation

  • Design a relational schema that mimics the graph's nodes (vertices) and edges as tables (e.g., intersections and road_segments with fields for danger_score, risk_score, coordinates, and connected nodes).
  • Implement a PostgresClient that inherits from or matches the interface of TigerGraphClient, and use factory pattern or conditional imports in custom_db/__init__.py depending on the active environment config.
  • Add setup steps and schema migrations to the README.

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions