Skip to content

Commit 1fa1c61

Browse files
committed
Added support for sparse sequences
1 parent 51387f3 commit 1fa1c61

File tree

3 files changed

+488
-87
lines changed

3 files changed

+488
-87
lines changed

README.md

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,40 @@ await db.doTn(tn => {
6161
6262
> **Note:** Running multiple operations in a single transaction is not _necessarily_ faster than running them in separate transactions. Internally, operations on a single transaction would be serialized to ensure correctness. In the future, a more sophisticated locking might mitigate the need to serialize operations.
6363
64+
## Sparse sequences
65+
66+
It is recommended to enable _sparse sequences_ mode if possible.
67+
68+
Mutations in PouchDB databases are organized into sequences. By default, the adapter keeps these sequences strictly sequential (ie without gaps), just like LevelDB and other adapters. In a distributed database like FoundationDB, this would quickly become a bottleneck by making write concurrency impossible. Concurrent writes would conflict and need to be retried or canceled. Instead, the adapter can make use of FoundationDB commit versions. Versions are strictly monotonically increasing (like sequences) but not sequential. This should be, on its own, okay for most applications. The problem is that versions are big - 12 bytes each - so they don't fit into typical numeric data types. Instead, they are exposed as 24-character _hex strings_.
69+
70+
**Caveats:**
71+
72+
- Sparse sequences are not strictly sequential. This can cause issues when, for example, expecting `seq - 1` to exist for each reported `seq`.
73+
- Sparse sequences are too large to fit into basic numeric data types. Instead, a 24-character hex strings are used. This can cause issues with PouchDB ecosystem and existing application code.
74+
- Commit versions are, by definition, not finalized until the transaction is committed. When using multi-operation transactions, the sequences reported by the adapter are **non-final**. A transaction read version is used as a placeholder to ensure some level of robustness within the transaction. If the final sequences are needed outside the transaction, it's up to the userland code to replace the first 10 bytes (20 hex characters) with the actual commit version.
75+
76+
> **Changing the mode for an existing database is not supported.** But effort has been made to make it possible. If this is an important feature for you, please raise an issue.
77+
78+
Sparse sequences can be enabled by passing `sparseSeq: true` to PouchDB constructor:
79+
80+
```
81+
const pouch = new PouchDB({
82+
name: 'foo',
83+
adapter: 'foundationdb',
84+
db,
85+
sparseSeq: true
86+
});
87+
88+
db.changes({
89+
since: 0n,
90+
include_docs: true
91+
})
92+
```
93+
94+
> **Note:** It is generally safe to create multiple PouchDB instances with the same transaction, but make sure that all instances with the same `name` use the same version of this adapter.
95+
96+
> **Note:** Running multiple operations in a single transaction is not _necessarily_ faster than running them in separate transactions. Internally, operations on a single transaction would be serialized to ensure correctness. In the future, a more sophisticated locking might mitigate the need to serialize operations.
97+
6498
## Limitations
6599

66100
The adapter is subject to the fundamental [limitations](https://apple.github.io/foundationdb/known-limitations.html) of FoundationDB. In particular:
@@ -69,10 +103,6 @@ The adapter is subject to the fundamental [limitations](https://apple.github.io/
69103
- Transactions can last at most 5 seconds from the first read.
70104
- Write transactions have overall size limitations.
71105

72-
Also, there are non-FoundationDB-specific limitations:
73-
74-
- All write operations update "last update seq" and "doc count" keys, so concurrent write transactions are effectively guaranteed to conflict and wouldn't benefit from concurrency. This can also affect read operations, eg if they need to update views.
75-
76106
Some of these limitations are solvable. If these limitations are a significant problem for you, please create an issue.
77107

78108
## Testing

0 commit comments

Comments
 (0)