Skip to content

Commit 6f3124e

Browse files
authored
feat(array): array support and protocol for doubles (#55)
* array column support * protocol version option, binary protocol for doubles
1 parent 2eca1ef commit 6f3124e

25 files changed

+2612
-166
lines changed

.github/workflows/build.yml

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,8 @@ jobs:
2424
- name: Run vet
2525
run: go vet ./...
2626

27-
- name: Run staticcheck
28-
uses: dominikh/[email protected]
29-
with:
30-
version: "2023.1.2"
31-
install-go: false
32-
cache-key: ${{ matrix.go-version }}
27+
- name: Run Staticcheck
28+
run: go run honnef.co/go/tools/cmd/[email protected] ./...
3329

3430
- name: Run tests
3531
run: go test -v ./...

README.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Features:
1717

1818
New in v3:
1919
* Supports ILP over HTTP using the same client semantics
20+
* Supports n-dimensional arrays of doubles for QuestDB servers 9.0.0 and up
2021

2122
Documentation is available [here](https://pkg.go.dev/github.com/questdb/go-questdb-client/v3).
2223

@@ -99,6 +100,95 @@ HTTP is the recommended transport to use. To connect via TCP, set the configurat
99100
// ...
100101
```
101102

103+
## N-dimensional arrays
104+
105+
QuestDB server version 9.0.0 and newer supports n-dimensional arrays of double precision floating point numbers.
106+
The Go client provides several methods to send arrays to QuestDB:
107+
108+
### 1D Arrays
109+
110+
```go
111+
// Send a 1D array of doubles
112+
values1D := []float64{1.1, 2.2, 3.3, 4.4}
113+
err = sender.
114+
Table("measurements").
115+
Symbol("sensor", "temp_probe_1").
116+
Float64Array1DColumn("readings", values1D).
117+
AtNow(ctx)
118+
```
119+
120+
### 2D Arrays
121+
122+
```go
123+
// Send a 2D array of doubles (must be rectangular)
124+
values2D := [][]float64{
125+
{1.1, 2.2, 3.3},
126+
{4.4, 5.5, 6.6},
127+
{7.7, 8.8, 9.9},
128+
}
129+
err = sender.
130+
Table("matrix_data").
131+
Symbol("experiment", "test_001").
132+
Float64Array2DColumn("matrix", values2D).
133+
AtNow(ctx)
134+
```
135+
136+
### 3D Arrays
137+
138+
```go
139+
// Send a 3D array of doubles (must be regular cuboid shape)
140+
values3D := [][][]float64{
141+
{{1.0, 2.0}, {3.0, 4.0}},
142+
{{5.0, 6.0}, {7.0, 8.0}},
143+
}
144+
err = sender.
145+
Table("tensor_data").
146+
Symbol("model", "neural_net_v1").
147+
Float64Array3DColumn("weights", values3D).
148+
AtNow(ctx)
149+
```
150+
151+
### N-dimensional Arrays
152+
153+
For higher dimensions, use the `NewNDArray` function:
154+
155+
```go
156+
// Create a 2x3x4 array
157+
arr, err := qdb.NewNDArray[float64](2, 3, 4)
158+
if err != nil {
159+
log.Fatal(err)
160+
}
161+
162+
// Fill with values
163+
arr.Fill(1.5)
164+
165+
// Or set individual values
166+
arr.Set([]uint{0, 1, 2}, 42.0)
167+
168+
err = sender.
169+
Table("ndarray_data").
170+
Symbol("dataset", "training_batch_1").
171+
Float64ArrayNDColumn("features", arr).
172+
AtNow(ctx)
173+
```
174+
175+
The array data is sent over a new protocol version (2) that is auto-negotiated
176+
when using HTTP(s), or can be specified explicitly via the ``protocol_version=2``
177+
parameter when using TCP(s).
178+
179+
We recommend using HTTP(s), but here is an TCP example, should you need it:
180+
181+
```go
182+
sender, err := qdb.NewLineSender(ctx,
183+
qdb.WithTcp(),
184+
qdb.WithProtocolVersion(qdb.ProtocolVersion2))
185+
```
186+
187+
When using ``protocol_version=2`` (with either TCP(s) or HTTP(s)), the sender
188+
will now also serialize ``float64`` (double-precision) columns as binary.
189+
You might see a performance uplift if this is a dominant data type in your
190+
ingestion workload.
191+
102192
## Pooled Line Senders
103193

104194
**Warning: Experimental feature designed for use with HTTP senders ONLY**

0 commit comments

Comments
 (0)