Skip to content

[Feat] : Add moss-connector-postgres #168

@yatharthk2

Description

@yatharthk2

Summary

This connector reads the result of a SQL query from a Postgres database and turns each row into a document in a Moss search index.

It uses psycopg (version 3) for the connection, so it works against any Postgres-compatible database — Postgres itself, Neon, Supabase (via direct URL), CockroachDB, Amazon RDS, Timescale, and pgvector.

To get started

  1. Read _template/README.md.
  2. Copy the template:
    cd packages/moss-data-connector
    cp -r _template moss-connector-postgres
    cd moss-connector-postgres
  3. Rename TemplateConnectorPostgresConnector in src/connector.py, update src/__init__.py to re-export it, and fill in pyproject.toml.
  4. Implement __iter__ using psycopg. Set row_factory=dict_row so rows come back as dicts instead of tuples.

Deliverables

The caller should be able to write:

from moss_connector_postgres import PostgresConnector, ingest
from moss import DocumentInfo

source = PostgresConnector(
    dsn="postgresql://user:pass@host:5432/dbname",
    query="SELECT id, title, body FROM articles",
    mapper=lambda row: DocumentInfo(
        id=str(row["id"]),
        text=row["body"],
        metadata={"title": row["title"]},
    ),
)

await ingest(source, project_id="...", project_key="...", index_name="articles")

Files:

  • src/connector.pyPostgresConnector class
  • src/__init__.py — re-exports PostgresConnector and ingest
  • src/ingest.py — copy from _template/src/ingest.py
  • pyproject.toml — lists psycopg[binary]>=3.1
  • tests/test_postgres.py — unit test with mocked psycopg.connect and MossClient
  • tests/test_integration_postgres_moss.py — live test, skips without POSTGRES_DSN and Moss creds
  • README.md
  • Row added to packages/moss-data-connector/README.md

Required for review / PR acceptance

  • Loom video (≈1 min): show the Postgres DB, run the live test, show Moss queried.
  • CLA signed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions