Skip to content

[Feat] : Add moss-connector-snowflake; Snowflake source connector #170

@yatharthk2

Description

@yatharthk2

Summary

This connector reads the result of a SQL query from a Snowflake warehouse and turns each row into a document in a Moss search index.

It uses the official snowflake-connector-python driver, so it works against any Snowflake account — standard, enterprise, or Snowflake-on-Azure.

To get started

  1. Read _template/README.md.
  2. Copy the template:
    cd packages/moss-data-connector
    cp -r _template moss-connector-snowflake
    cd moss-connector-snowflake
  3. Rename TemplateConnectorSnowflakeConnector in src/connector.py, update src/__init__.py to re-export it, and fill in pyproject.toml.
  4. Implement __iter__ using snowflake.connector. Use DictCursor (from snowflake.connector import DictCursor) so rows come back as dicts instead of tuples.

Deliverables

The caller should be able to write:

from moss_connector_snowflake import SnowflakeConnector, ingest
from moss import DocumentInfo

source = SnowflakeConnector(
    account="xy12345.us-east-1",
    user="your_user",
    password="your_password",
    warehouse="COMPUTE_WH",
    database="MY_DB",
    schema="PUBLIC",
    query="SELECT id, title, body FROM articles",
    mapper=lambda row: DocumentInfo(
        id=str(row["ID"]),
        text=row["BODY"],
        metadata={"title": row["TITLE"]},
    ),
)

await ingest(source, project_id="...", project_key="...", index_name="articles")

Files:

  • src/connector.pySnowflakeConnector class
  • src/__init__.py — re-exports SnowflakeConnector and ingest
  • src/ingest.py — copy from _template/src/ingest.py
  • pyproject.toml — lists snowflake-connector-python>=3.0
  • tests/test_snowflake.py — unit test with mocked snowflake.connector.connect and MossClient
  • tests/test_integration_snowflake_moss.py — live test, skips without SNOWFLAKE_ACCOUNT, SNOWFLAKE_USER, SNOWFLAKE_PASSWORD, and Moss creds
  • README.md
  • Row added to packages/moss-data-connector/README.md

Required for review / PR acceptance

  • Loom video (≈1 min): show the Snowflake warehouse, run the live test, show Moss queried.
  • CLA signed.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions