Skip to content

Commit

Permalink
First doc pass.
Browse files Browse the repository at this point in the history
  • Loading branch information
Fuyukai committed Jul 4, 2021
1 parent b8dc876 commit c8744f7
Show file tree
Hide file tree
Showing 14 changed files with 910 additions and 637 deletions.
14 changes: 14 additions & 0 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# .readthedocs.yml

version: 2

sphinx:
configuration: docs/conf.py

python:
version: "3.8"
install:
- method: pip
path: .
extra_requirements:
- docs
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
46 changes: 46 additions & 0 deletions docs/_static/css/custom.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
@import url('https://fonts.googleapis.com/css?family=Overpass+Mono|Cabin|Quicksand&display=swap');


div.highlight pre, span.pre {
font-family: "JetBrains Mono", "Overpass Mono", monospace !important;
font-size: 14px !important;
}

.rst-content code.literal {
font-family: "JetBrains Mono", "Overpass Mono", monospace !important;
}


h1, h2, h3, h4, h5, h6 {
font-family: "Cabin";
}

p, li, strong, em, a {
font-family: "Cabin" !important;
}

/* get rid of annoying list margins */
.rst-content .section ol li > *, .rst-content .section ul li > * {
margin-top: 0 !important;
}


/* Borrowed from the trio RTD */

.wy-side-nav-search {
background-color: #75E6DE !important;
}

.wy-side-nav-search > a {
color: #000000 !important;
}

.wy-side-nav-search > a.logo {
display: block !important;
padding-bottom: 0.809em !important;
}

.wy-side-nav-search > a img.logo {
display: inline !important;
padding: 0 !important;
}
6 changes: 6 additions & 0 deletions docs/api-highlevel.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.. _highlevel:

High-Level API
==============

Currently a work in progress. Check back soon!
235 changes: 235 additions & 0 deletions docs/api-lowlevel.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
Low-Level API
=============

The low-level API is a sans-IO_, in-memory PostgreSQL client. It implements creation and parsing of
PostgreSQL network protocol messages, and maintains internal state relating to the client
connection.

.. warning::

Despite this class primarily being laid out similar to the PostgreSQL protocol itself, this
is *not* a protocol state machine, but a client with its own semantics (e.g. you can never send
a Parse message by itself, it must always be accompanied with a Describe).

Creation
--------

The primary class for the low-level client is :class:`~.pg_purepy.protocol.SansIOClient`.

.. autoclass:: pg_purepy.SansIOClient
:members: __init__

Doing... Stuff
--------------

The client by itself doesn't really do anything. Instead, there's a process in order to make it do
something.

1. Call the ``do_*`` method for the thing you want to do, e.g. send a query. This updates internal
states and returns the bytes for the command.
2. Send the bytes returned by the ``do_*`` function to the server.
3. Enter the Ready Loop.

The Ready Loop
--------------

The ready loop is the key construct for working with the low-level API. It consists of a simple loop
that does several steps.

First, you need to check if the protocol is already ready. If you don't do this, you'll likely get
a deadlock because the server will have nothing to send and you'll wait forever on reading.

.. autoattribute:: pg_purepy.SansIOClient.ready

Second, you need to check if there are any incoming messages from the server. This is done by
calling :meth:`.SansIOClient.next_event` in an inner ``while True`` loop until it returns the
special ``NO_DATA`` constant sentinel value. You should also do error handling wrt to
:class:`.ErrorResponse` messages, if so desired. If this inner loop returns :class:`.ReadyForQuery`,
then you have finished the whole loop and should break out of both the inner and outer loops.

.. code-block:: python3
while True:
event = protocol.next_event()
if event == NO_DATA:
break
# optional, you can just yield it or whatever else
if isinstance(event, ErrorResponse): ...
do_something_with_event(event)
if isinstance(event, ReadyForQuery):
return
.. automethod:: pg_purepy.SansIOClient.next_event

Next, you need to get any synchronisation data that the client needs to send to the server. This
may occur during authentication for example.

.. code-block:: python3
to_send = protocol.get_needed_synchronisation()
send_data(to_send)
.. automethod:: pg_purepy.SansIOClient.get_needed_synchronisation

Finally, you need to read incoming data from the server, and feed it into the protocol machine for
later processing. This data won't actually be processed until you call
:meth:`~SansIOClient.next_event` in the next iteration of the loop.

.. code-block:: python3
received = read_data()
protocol.receive_bytes(received)
.. automethod:: pg_purepy.SansIOClient.receive_bytes

The ready loop in the mid-level API looks like this, for reference:

.. code-block:: python3
while not self._protocol.ready:
while True:
next_event = self._protocol.next_event()
if next_event is NEED_DATA:
break
if isinstance(next_event, ErrorOrNoticeResponse) and next_event.notice:
if next_event.severity == "WARNING":
err = wrap_error(next_event)
warnings.warn(str(err))
yield next_event
if isinstance(next_event, ReadyForQuery):
await anyio.sleep(0) # checkpoint()
return
to_send = self._protocol.get_needed_synchronisation()
if to_send:
await self._stream.send(to_send)
received = await self._stream.receive(65536)
self._protocol.receive_bytes(received)
.. note::

The order of this may look odd, but it helps prevent deadlocks. Draining the events first means
that any events that may have been sent AFTER the ReadyForQuery, for example a second query
that was queued by the client before the server sent the ReadyForQuery message.

The ready loop should be performed after every single action to completion, in order to drain data
from the server that is no longer relevant. If that's not possible, then methods should call the
ready loop before performing any other actions, for the same reason.

Sending Commands
----------------

Each ``do_*`` command performs a specific action or set of actions on the server. Each command will
cause the server to issue a specific set of responses in return, which are constructed into their
own message classes.

Additionally, the server may issue certain responses at any point in the connection. These will
be captured by the client and returned from :meth:`.next_event` automatically. These are:

- :class:`.ParameterStatus`, when a runtime parameter is set
- :class:`.ErrorOrNoticeResponse`, for server-sent errors or notices

.. warning::

:class:`.ErrorOrNoticeResponse` instances will only be captured if they are either notices, or
recoverable. Non-recoverable errors will be turned into protocol errors.

Once the the server has sent all messages, a :class:`.ReadyForQuery` event will be returned.

Startup
~~~~~~~

The first message you must send is the startup message. This contains the username and database
information.

.. automethod:: pg_purepy.SansIOClient.do_startup

Any additional messages may be sent during the synchronisation stage.

This will yield several messages, usually in this order:

1. The :class:`.AuthenticationRequest` describing how the client should authenticate, if needed.
2. Several :class:`.ParameterStatus` message, setting runtime parameters.
3. A :class:`.BackendKeyData` message, used for potential later cancellation.
4. A :class:`.AuthenticationCompleted` message, signifying that the client can now send commands.

Simple Queries
~~~~~~~~~~~~~~

Simple queries are static queries with no parameters that combine parsing and execution into one
step. These queries should be used by a higher-level client when no parameters are needed, for
network performance reasons.

To perform a simple query, you should call :meth:`.do_simple_query`.

.. automethod:: pg_purepy.SansIOClient.do_simple_query

This will yield several messages, usually in this order:

1. A :class:`.RowDescription`, if this query returns data, or a :class:`.ErrorOrNoticeResponse` if
the query was invalid in some way.
2. Zero to N :class:`.DataRow` instances.
3. One :class:`.CommandComplete` instance.


Extended Queries
~~~~~~~~~~~~~~~~

Extended queries are queries using prepared statements and SQL injection invulnerable parameter
injection.

To perform an extended query, you need to create a prepared statement. This will issue a
Parse+Describe combo of messages to the server.

.. automethod:: pg_purepy.SansIOClient.do_create_prepared_statement

This will yield several messages, usually in this order:

1. A :class:`.ParseComplete`, or a :class:`.ErrorOrNoticeResponse` if the query was invalid in some
way.
2. A :class:`.ParameterDescription`, if the query has parameters to be filled.
3. A :class:`.PreparedStatementInfo` wrapping the relevant info about the prepared query.

.. warning::

Internally, the server actually returns either a :class:`.RowDescription` or a
``NoData`` message, but both are wrapped into the :class:`.PreparedStatementInfo`.

Once you have a prepared statement, you can then run the query.

.. automethod:: pg_purepy.SansIOClient.do_bind_execute

This will yield a similar sequence of messages as the simple query:

1. (Optional) A :class:`.ErrorOrNoticeResponse` if the parameters were invalid in some way.
2. A :class:`.BindComplete`.
3. 0 to N :class:`.DataRow` instances for the actual data of the query.

Termination
~~~~~~~~~~~

When you are done with a connection, you should send a Terminate message. This closes the client
and no further actions will work.

.. automethod:: pg_purepy.SansIOClient.do_terminate

Internal State
~~~~~~~~~~~~~~

The client object exposes a number of useful read-only properties.

.. autoattribute:: pg_purepy.SansIOClient.encoding

.. autoattribute:: pg_purepy.SansIOClient.state

.. autoclass:: pg_purepy.protocol.ProtocolState
:members:
:undoc-members:

.. _sans-IO: https://sans-io.readthedocs.io/
Loading

0 comments on commit c8744f7

Please sign in to comment.