feat: add moss-connector-dynamodb (DynamoDB source connector)#180
feat: add moss-connector-dynamodb (DynamoDB source connector)#180Abhijitam01 wants to merge 5 commits intousemoss:mainfrom
Conversation
Adds a Python Semantic Kernel plugin that exposes Moss retrieval as a @kernel_function, letting any SK agent query a Moss index with 3 lines of code. Mirrors the strands-agents-moss integration pattern. Includes plugin implementation, 14 unit/integration tests, example, README with Quick Start and configuration docs, and TODOS.md for follow-up work (CI/CD pipeline, .NET design doc).
feat: add Semantic Kernel plugin for Moss semantic search
Aligns with the rest of the repo — all other integration packages (strands-agents-moss, elevenlabs-moss, pipecat-moss, moss-cli) depend on moss and import from moss, not inferedge-moss.
Add a new DynamoDB source connector package that reads every item from a DynamoDB table via boto3 scan with automatic pagination, maps each item to a DocumentInfo via a user-supplied mapper, and ingests them into a Moss search index. Key design decisions: - scan_kwargs dict for all DynamoDB scan options (FilterExpression, ProjectionExpression, ExpressionAttributeNames, etc.) - endpoint_url param for DynamoDB Local / localstack dev/testing - Standard boto3 credential chain (no AWS credential params) - Immutable scan_kwargs copy in __iter__ to prevent mutation Also pulls in the existing moss-data-connector packages (template, SQLite, MongoDB) from upstream/main, updates the parent README with the DynamoDB row, and adds a streaming/batched ingest TODO. Closes usemoss#171 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Abhijitam01
requested review from
CoderOMaster,
HarshaNalluru,
r4ghu and
yatharthk2
as code owners
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| # Import your driver *inside* this method, not at module top, so | ||
| # importing the package never fails just because the driver isn't | ||
| # installed. |
There was a problem hiding this comment.
🟡 Template connector comment contradicts template README import rule
The template connector.py comment on lines 30-32 says to import the driver inside __iter__, but the template README.md at packages/moss-data-connector/_template/README.md:30 explicitly states the opposite rule: "Declare your driver as a main dependency in pyproject.toml and import it at the top of the module." All three actual connectors (SQLite, MongoDB, DynamoDB) follow the README rule and import at the top. A new contributor copying this template would see contradictory advice, and if they follow the code comment, their connector would diverge from the established pattern and the authoritative rules.
| # Import your driver *inside* this method, not at module top, so | |
| # importing the package never fails just because the driver isn't | |
| # installed. | |
| # Import your driver at the top of the module (see README rules). | |
| # Declare it as a main dependency in pyproject.toml so pip installs it. | |
| # |
Was this helpful? React with 👍 or 👎 to provide feedback.
| project_id: Moss project ID. Falls back to ``MOSS_PROJECT_ID`` env var. | ||
| project_key: Moss project key. Falls back to ``MOSS_PROJECT_KEY`` env var. | ||
| index_name: Name of the Moss index to query. | ||
| top_k: Number of results to retrieve per query. | ||
| alpha: Blend between semantic (1.0) and keyword (0.0) scoring. | ||
| result_prefix: Prefix prepended to formatted search results. | ||
| """ | ||
| self._client = MossClient(project_id=project_id, project_key=project_key) |
There was a problem hiding this comment.
🚩 MossPlugin documents env-var fallback that isn't implemented in MossClient
The MossPlugin.__init__ docstring at packages/semantic-kernel-moss/src/semantic_kernel_moss/moss_plugin.py:64-65 claims project_id and project_key fall back to MOSS_PROJECT_ID / MOSS_PROJECT_KEY env vars. However, MossClient.__init__ at sdks/python/sdk/src/moss/client/moss_client.py:65 takes project_id: str, project_key: str with no env-var fallback visible in the Python layer. The pre-existing strands-agents-moss package follows the identical pattern (strands_agents_moss/moss_search_tool.py:72), so this appears to be an intentional convention — the fallback likely happens in the Rust ManageClient/IndexManager bindings. Could not verify without access to the Rust source. If the Rust layer does NOT handle None, users relying on env-var fallback would get a runtime error.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
can you please provide with loom demo, does not have to be more than 1 min |
2 participants
Summary
moss-connector-dynamodbpackage that scans a DynamoDB table, maps items toDocumentInfovia user-supplied mapper, and ingests into a Moss search index_template/scaffold)moss-data-connector/packages (template, SQLite, MongoDB) from upstreamKey design decisions
scan_kwargsdict — single passthrough dict for all DynamoDB scan options (FilterExpression,ProjectionExpression,ExpressionAttributeNames, etc.) instead of separate paramsendpoint_urlparam — enables DynamoDB Local / localstack for dev/testingscan_kwargscopy in__iter__to prevent mutation across iterationsCloses #171
Test plan
DYNAMODB_TABLE,AWS_REGION,MOSS_PROJECT_ID,MOSS_PROJECT_KEY)pip install -e ".[dev]"andpython -c "from moss_connector_dynamodb import DynamoDBConnector, ingest"workGenerated with Claude Code