Skip to content

Commit e237760

Browse files
Initial implementation of HTTP connector
1 parent 85e8e88 commit e237760

17 files changed

+502
-63
lines changed
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
singer_sdk.connectors.BaseConnector
2+
===================================
3+
4+
.. currentmodule:: singer_sdk.connectors
5+
6+
.. autoclass:: BaseConnector
7+
:members:
8+
:special-members: __init__, __call__

docs/guides/custom-connector.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Using a custom connector class
2+
3+
The Singer SDK has a few built-in connector classes that are designed to work with a variety of sources:
4+
5+
* [`SQLConnector`](../../classes/singer_sdk.SQLConnector) for SQL databases
6+
7+
If you need to connect to a source that is not supported by one of these built-in connectors, you can create your own connector class. This guide will walk you through the process of creating a custom connector class.
8+
9+
## Subclass `BaseConnector`
10+
11+
The first step is to create a subclass of [`BaseConnector`](../../classes/singer_sdk.connectors.BaseConnector). This class is responsible for creating streams and handling the connection to the source.
12+
13+
```python
14+
from singer_sdk.connectors import BaseConnector
15+
16+
17+
class MyConnector(BaseConnector):
18+
pass
19+
```
20+
21+
## Implement `get_connection`
22+
23+
The [`get_connection`](http://127.0.0.1:5500/build/classes/singer_sdk.connectors.BaseConnector.html#singer_sdk.connectors.BaseConnector.get_connection) method is responsible for creating a connection to the source. It should return an object that implements the [context manager protocol](https://docs.python.org/3/reference/datamodel.html#with-statement-context-managers), e.g. it has `__enter__` and `__exit__` methods.
24+
25+
```python
26+
from singer_sdk.connectors import BaseConnector
27+
28+
29+
class MyConnector(BaseConnector):
30+
def get_connection(self):
31+
return MyConnection()
32+
```

docs/guides/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,5 @@ The following pages contain useful information for developers building on top of
77
88
porting
99
pagination-classes
10+
custom-connector
1011
```

docs/reference.rst

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,3 +140,12 @@ Batch
140140

141141
batch.BaseBatcher
142142
batch.JSONLinesBatcher
143+
144+
Abstract Connector Classes
145+
--------------------------
146+
147+
.. autosummary::
148+
:toctree: classes
149+
:template: class.rst
150+
151+
connectors.BaseConnector

noxfile.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,9 @@
3939
test_dependencies = [
4040
"coverage[toml]",
4141
"pytest",
42-
"pytest-snapshot",
4342
"pytest-durations",
43+
"pytest-httpserver",
44+
"pytest-snapshot",
4445
"freezegun",
4546
"pandas",
4647
"pyarrow",

poetry.lock

Lines changed: 68 additions & 35 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,7 @@ types-simplejson = "^3.18.0"
115115
types-PyYAML = "^6.0.12"
116116
coverage = {extras = ["toml"], version = "^7.2"}
117117
pyarrow = ">=11,<13"
118+
pytest-httpserver = "^1.0.6"
118119
pytest-snapshot = "^0.9.0"
119120

120121
# Cookiecutter tests

0 commit comments

Comments
 (0)