Skip to content

Commit e929d41

Browse files
evilmonkeyincevilmonkeyinc
authored
Merge pull request #14 from evilmonkeyinc/chore/benchmarks
chore: add benchmark tests and notes
2 parents 43a03aa + 61c8977 commit e929d41

13 files changed

+437
-2
lines changed

‎README.md

+4-2
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,9 @@ The library supports scripts and filters using a [standard script engine](script
203203
Additionally, a custom script engine can be created and passed as an additional option when compiling the JSONPath selector
204204

205205
```golang
206-
206+
...
207+
compiled, err := jsonpath.Compile(selector, jsonpath.ScriptEngine(customScriptEngine))
208+
...
207209
```
208210

209211
## History
@@ -212,6 +214,6 @@ The [original specification for JSONPath](https://goessner.net/articles/JsonPath
212214

213215
## Hows does this compare to...
214216

215-
There are many [implementations](https://cburgmer.github.io/json-path-comparison/) in multiple languages so I will not claim that this library is better in any way but I believe that it is true to the original specification and was an enjoyable challenge.
217+
There are many [implementations](https://cburgmer.github.io/json-path-comparison/) in multiple languages. Some sample benchmarks against other implementations are detailed [here](benchmark/README.md). This implementation has merit but it not the quickest golang implementation available but could be useful for those not wanting to use json marshaling.
216218

217219
Sample queries and the expected response for this implementation compared to the community consensus are available [here](test/README.md)

‎benchmark/README.md

+186
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
# Benchmarks
2+
3+
Benchmarks of this library compared to other golang implementations.
4+
5+
The example selectors from the original specification along with the sample data have been used to create this data.
6+
7+
If any library is not mentioned for a selector, that means that implementation returned an error of some kind
8+
9+
Test of accuracy are based off of the expected response based on the original specification and the consensus from the [json-path-comparison](https://cburgmer.github.io/json-path-comparison/)
10+
11+
## Command
12+
13+
```bash
14+
go test -bench=. -cpu=1 -benchmem -count=1 -benchtime=100x
15+
```
16+
17+
## Libraries
18+
19+
- `github.com/PaesslerAG/jsonpath v0.1.1`
20+
- `github.com/bhmj/jsonslice v1.1.2`
21+
- `github.com/evilmonkeyinc/jsonpath v0.7.0`
22+
- `github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852`
23+
- `github.com/spyzhov/ajson v0.7.0`
24+
25+
## TL;DR
26+
27+
This implementation is slower than others, but is only one of two that has a non-error response to all sample selectors, the other being the [spyzhov/ajson](https://github.com/spyzhov/ajson) implementation which is on average twice as fast but relies on its own json marshaller (which is impressive in it's own right)
28+
29+
Generally the accuracy of the implementations that could run are the same, with a minor deviation with how array ranges are handled with one, one implementation ran but did not return a response I suspect the testing method is flawed but without adequate documentation I could not confirm this.
30+
31+
## Selectors
32+
33+
### `$.store.book[*].author`
34+
35+
Expected Response: `["Nigel Rees","Evelyn Waugh","Herman Melville","J. R. R. Tolkien"]`
36+
37+
|library|ns/op|B/op|allocs/op|accurate|
38+
|-|-|-|-|-|
39+
|evilmonkeyinc|43551 ns/op|6496 B/op|188 allocs/op|true|
40+
|paesslerAG|25549 ns/op|6417 B/op|131 allocs/op|false|
41+
|bhmj|6188 ns/op|1188 B/op|14 allocs/op|true|
42+
|spyzhov|17612 ns/op|6608 B/op|127 allocs/op|true|
43+
44+
45+
### `$..author`
46+
47+
Expected Response: `["Nigel Rees","Evelyn Waugh","Herman Melville","J. R. R. Tolkien"]`
48+
49+
|library|ns/op|B/op|allocs/op|accurate|
50+
|-|-|-|-|-|
51+
|evilmonkeyinc|198323 ns/op|16689 B/op|458 allocs/op|true|
52+
|paesslerAG|16293 ns/op|6361 B/op|122 allocs/op|false|
53+
|bhmj|16665 ns/op|1554 B/op|27 allocs/op|true|
54+
|spyzhov|20614 ns/op|7912 B/op|159 allocs/op|true|
55+
56+
57+
### `$.store.*`
58+
59+
Expected Response: `[too large]`
60+
> the expected response is an array with two components, the bike object and and array containing the book
61+
62+
|library|ns/op|B/op|allocs/op|accurate|
63+
|-|-|-|-|-|
64+
|evilmonkeyinc|25933 ns/op|4928 B/op|130 allocs/op|true|
65+
|paesslerAG|16568 ns/op|6233 B/op|120 allocs/op|false|
66+
|bhmj|8429 ns/op|3708 B/op|9 allocs/op|true|
67+
|spyzhov|13288 ns/op|6376 B/op|117 allocs/op|true|
68+
69+
### `$.store..price`
70+
71+
Expected Response; `[19.95,8.95,12.99,8.99,22.99]`
72+
73+
|library|ns/op|B/op|allocs/op|accurate|
74+
|-|-|-|-|-|
75+
|evilmonkeyinc|104648 ns/op|15673 B/op|443 allocs/op|true|
76+
|paesslerAG|99737 ns/op|6297 B/op|125 allocs/op|false|
77+
|bhmj|90572 ns/op|1195 B/op|28 allocs/op|true|
78+
|spyzhov|23793 ns/op|7816 B/op|158 allocs/op|true|
79+
80+
### `$..book[2]`
81+
82+
Expected Response: `[{"author":"Herman Melville","category":"fiction","isbn":"0-553-21311-3","price":8.99,"title":"Moby Dick"}]`
83+
84+
|library|ns/op|B/op|allocs/op|accurate|
85+
|-|-|-|-|-|
86+
|evilmonkeyinc|192611 ns/op|16961 B/op|471 allocs/op|true|
87+
|paesslerAG|25408 ns/op|6545 B/op|130 allocs/op|false|
88+
|bhmj|13719 ns/op|1260 B/op|16 allocs/op|true|
89+
|spyzhov|130744 ns/op|7904 B/op|160 allocs/op|true|
90+
91+
### `$..book[(@.length-1)]`
92+
93+
Expected Response: `[{"author":"J. R. R. Tolkien","category":"fiction","isbn":"0-395-19395-8","price":22.99,"title":"The Lord of the Rings"}]`
94+
95+
|library|ns/op|B/op|allocs/op|accurate|
96+
|-|-|-|-|-|
97+
|evilmonkeyinc|138309 ns/op|18001 B/op|542 allocs/op|true|
98+
|spyzhov|47062 ns/op|8840 B/op|197 allocs/op|true|
99+
100+
### `$..book[-1:]`
101+
102+
Expected Response" `[{"author":"J. R. R. Tolkien","category":"fiction","isbn":"0-395-19395-8","price":22.99,"title":"The Lord of the Rings"}]`
103+
104+
|library|ns/op|B/op|allocs/op|accurate|
105+
|-|-|-|-|-|
106+
|evilmonkeyinc|198634 ns/op|17201 B/op|486 allocs/op|true|
107+
|paesslerAG|64934 ns/op|6801 B/op|137 allocs/op|false|
108+
|bhmj|16392 ns/op|1709 B/op|22 allocs/op|note1|
109+
|spyzhov|17658 ns/op|7968 B/op|164 allocs/op|true|
110+
111+
> note1: returned an array containing the expected response, an array in an array, but the correct object
112+
113+
### `$..book[0,1]`
114+
115+
Expected Response: `[{"author":"Nigel Rees","category":"reference","price":8.95,"title":"Sayings of the Century"},{"author":"Evelyn Waugh","category":"fiction","price":12.99,"title":"Sword of Honour"}]`
116+
117+
|library|ns/op|B/op|allocs/op|accurate|
118+
|-|-|-|-|-|
119+
|evilmonkeyinc|111628 ns/op|17297 B/op|489 allocs/op|true|
120+
|paesslerAG|54361 ns/op|6817 B/op|136 allocs/op|false|
121+
|bhmj|23537 ns/op|2285 B/op|23 allocs/op|note1|
122+
|spyzhov|49976 ns/op|8048 B/op|165 allocs/op|true|
123+
124+
> note1: returned an array containing the expected response, an array in an array, but the correct object
125+
126+
### `$..book[:2]`
127+
128+
Expected Response: `[{"author":"Nigel Rees","category":"reference","price":8.95,"title":"Sayings of the Century"},{"author":"Evelyn Waugh","category":"fiction","price":12.99,"title":"Sword of Honour"}]`
129+
130+
|library|ns/op|B/op|allocs/op|accurate|
131+
|-|-|-|-|-|
132+
|evilmonkeyinc|138072 ns/op|17281 B/op|483 allocs/op|true|
133+
|paesslerAG|28601 ns/op|6801 B/op|137 allocs/op|false|
134+
|bhmj|21478 ns/op|2349 B/op|24 allocs/op|note1|
135+
|spyzhov|77671 ns/op|7984 B/op|164 allocs/op|true|
136+
137+
> note1: returned an array containing the expected response, an array in an array, but the correct object
138+
139+
### `$..book[?(@.isbn)]`
140+
141+
Expected Response: ` [{"author":"Herman Melville","category":"fiction","isbn":"0-553-21311-3","price":8.99,"title":"Moby Dick"},{"author":"J. R. R. Tolkien","category":"fiction","isbn":"0-395-19395-8","price":22.99,"title":"The Lord of the Rings"}]`
142+
143+
|library|ns/op|B/op|allocs/op|accurate|
144+
|-|-|-|-|-|
145+
|evilmonkeyinc|211344 ns/op|20265 B/op|556 allocs/op|true|
146+
|paesslerAG|138063 ns/op|6937 B/op|143 allocs/op|false|
147+
|bhmj|78538 ns/op|2731 B/op|30 allocs/op|note1|
148+
|spyzhov|71054 ns/op|8864 B/op|217 allocs/op|true|
149+
150+
> note1: returned an array containing the expected response, an array in an array, but the correct object
151+
152+
### `$..book[?(@.price<10)]`
153+
154+
Expected Response: `{"author":"Nigel Rees","category":"reference","price":8.95,"title":"Sayings of the Century"},{"author":"Herman Melville","category":"fiction","isbn":"0-553-21311-3","price":8.99,"title":"Moby Dick"}]`
155+
156+
|library|ns/op|B/op|allocs/op|accurate|
157+
|-|-|-|-|-|
158+
|evilmonkeyinc|282446 ns/op|20153 B/op|564 allocs/op|true|
159+
|bhmj|79741 ns/op|2899 B/op|43 allocs/op|note1|
160+
|spyzhov|79312 ns/op|10160 B/op|263 allocs/op|true|
161+
162+
> note1: returned an array containing the expected response, an array in an array, but the correct object
163+
164+
### `$..book[?(@.price<$.expensive)]`
165+
166+
Expected Response: `{"author":"Nigel Rees","category":"reference","price":8.95,"title":"Sayings of the Century"},{"author":"Herman Melville","category":"fiction","isbn":"0-553-21311-3","price":8.99,"title":"Moby Dick"}]`
167+
168+
|library|ns/op|B/op|allocs/op|accurate|
169+
|-|-|-|-|-|
170+
|evilmonkeyinc|305200 ns/op|21449 B/op|628 allocs/op|true|
171+
|bhmj|147911 ns/op|2995 B/op|46 allocs/op|note1|
172+
|spyzhov|232748 ns/op|10088 B/op|285 allocs/op|true|
173+
174+
> note1: returned an array containing the expected response, an array in an array, but the correct object
175+
176+
### `$..*`
177+
178+
Expected Response: `[too large]`
179+
> the expected response is an array that contains every value from the sample data, this will include an array, objects, and then each individual element of those collections
180+
181+
|library|ns/op|B/op|allocs/op|accurate|
182+
|-|-|-|-|-|
183+
|evilmonkeyinc|144373 ns/op|20193 B/op|546 allocs/op|true|
184+
|paesslerAG|32120 ns/op|6216 B/op|117 allocs/op|false|
185+
|bhmj|78242 ns/op|31209 B/op|69 allocs/op|true|
186+
|spyzhov|71936 ns/op|9288 B/op|187 allocs/op|true|

‎benchmark/benchmark_test.go

+141
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
package benchmark
2+
3+
import (
4+
"encoding/json"
5+
"testing"
6+
7+
paesslerAG "github.com/PaesslerAG/jsonpath"
8+
bhmj "github.com/bhmj/jsonslice"
9+
emi "github.com/evilmonkeyinc/jsonpath"
10+
oliveagle "github.com/oliveagle/jsonpath"
11+
spyzhov "github.com/spyzhov/ajson"
12+
)
13+
14+
var selectors = []string{
15+
"$.store.book[*].author",
16+
"$..author",
17+
"$.store.*",
18+
"$.store..price",
19+
"$..book[2]",
20+
"$..book[(@.length-1)]",
21+
"$..book[-1:]",
22+
"$..book[0,1]",
23+
"$..book[:2]",
24+
"$..book[?(@.isbn)]",
25+
"$..book[?(@.price<10)]",
26+
"$..book[?(@.price<$.expensive)]",
27+
"$..*",
28+
}
29+
30+
var sampleDataString string = `{ "store": { "book": [{ "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 }, { "category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99 }, { "category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99 } ], "bicycle": { "color": "red", "price": 19.95 } }, "expensive": 10 }`
31+
32+
func Benchmark_Comparison(b *testing.B) {
33+
34+
for _, selector := range selectors {
35+
b.Run(selector, func(b *testing.B) {
36+
b.Run("evilmonkeyinc", func(b *testing.B) {
37+
var err error
38+
for i := 0; i < b.N; i++ {
39+
_, err = emi.QueryString(selector, sampleDataString)
40+
}
41+
if err != nil {
42+
b.SkipNow()
43+
}
44+
})
45+
b.Run("paesslerAG", func(b *testing.B) {
46+
var err error
47+
for i := 0; i < b.N; i++ {
48+
value := make(map[string]interface{})
49+
sampleData := json.Unmarshal([]byte(sampleDataString), &value)
50+
_, err = paesslerAG.Get(selector, sampleData)
51+
}
52+
if err != nil {
53+
b.SkipNow()
54+
}
55+
})
56+
b.Run("bhmj", func(b *testing.B) {
57+
var err error
58+
for i := 0; i < b.N; i++ {
59+
_, err = bhmj.Get([]byte(sampleDataString), selector)
60+
}
61+
if err != nil {
62+
b.SkipNow()
63+
}
64+
})
65+
b.Run("oliveagle", func(b *testing.B) {
66+
var err error
67+
for i := 0; i < b.N; i++ {
68+
value := make(map[string]interface{})
69+
sampleData := json.Unmarshal([]byte(sampleDataString), &value)
70+
71+
var compiled *oliveagle.Compiled
72+
compiled, err = oliveagle.Compile(selector)
73+
if err == nil {
74+
_, err = compiled.Lookup(sampleData)
75+
}
76+
}
77+
if err != nil {
78+
b.SkipNow()
79+
}
80+
})
81+
b.Run("spyzhov", func(b *testing.B) {
82+
var err error
83+
for i := 0; i < b.N; i++ {
84+
root, _ := spyzhov.Unmarshal([]byte(sampleDataString))
85+
_, err = root.JSONPath(selector)
86+
}
87+
if err != nil {
88+
b.SkipNow()
89+
}
90+
})
91+
})
92+
}
93+
}
94+
95+
/**
96+
func Test_Comparison(t *testing.T) {
97+
98+
for _, selector := range selectors {
99+
t.Run(selector, func(t *testing.T) {
100+
101+
var response interface{}
102+
103+
// evilmonkeyinc
104+
obj, _ := emi.QueryString(selector, sampleDataString)
105+
bytes, _ := json.Marshal(obj)
106+
response = string(bytes)
107+
fmt.Printf("%s %s %v\n", selector, "evilmonkeyinc", response)
108+
109+
// paesslerAG
110+
value := interface{}(nil)
111+
sampleData := json.Unmarshal([]byte(sampleDataString), &value)
112+
response, _ = paesslerAG.Get(selector, sampleData)
113+
fmt.Printf("%s %s %v\n", selector, "paesslerAG", response)
114+
115+
// bhmj
116+
bytes, _ = bhmj.Get([]byte(sampleDataString), selector)
117+
response = string(bytes)
118+
fmt.Printf("%s %s %v\n", selector, "bhmj", response)
119+
120+
121+
// oliveagle
122+
value = make(map[string]interface{})
123+
sampleData = json.Unmarshal([]byte(sampleDataString), &value)
124+
125+
compiled, err := oliveagle.Compile(selector)
126+
if err == nil {
127+
response, _ = compiled.Lookup(sampleData)
128+
fmt.Printf("%s %s %v\n", selector, "oliveagle", response)
129+
} else {
130+
fmt.Printf("%s %s %v\n", selector, "oliveagle", "failed to compile")
131+
}
132+
133+
134+
// spyzhov
135+
root, _ := spyzhov.Unmarshal([]byte(sampleDataString))
136+
response, _ = root.JSONPath(selector)
137+
fmt.Printf("%s %s %v\n", selector, "spyzhov", response)
138+
})
139+
}
140+
}
141+
**/

‎benchmark/go.mod

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
module github.com/evilmonkeyinc/jsonpath/benchmark
2+
3+
go 1.17
4+
5+
require (
6+
github.com/PaesslerAG/jsonpath v0.1.1
7+
github.com/bhmj/jsonslice v1.1.2
8+
github.com/evilmonkeyinc/jsonpath v0.7.0
9+
github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852
10+
github.com/spyzhov/ajson v0.7.0
11+
)
12+
13+
require (
14+
github.com/PaesslerAG/gval v1.0.0 // indirect
15+
github.com/bhmj/xpression v0.9.1 // indirect
16+
)

‎benchmark/go.sum

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
github.com/PaesslerAG/gval v1.0.0 h1:GEKnRwkWDdf9dOmKcNrar9EA1bz1z9DqPIO1+iLzhd8=
2+
github.com/PaesslerAG/gval v1.0.0/go.mod h1:y/nm5yEyTeX6av0OfKJNp9rBNj2XrGhAf5+v24IBN1I=
3+
github.com/PaesslerAG/jsonpath v0.1.0/go.mod h1:4BzmtoM/PI8fPO4aQGIusjGxGir2BzcV0grWtFzq1Y8=
4+
github.com/PaesslerAG/jsonpath v0.1.1 h1:c1/AToHQMVsduPAa4Vh6xp2U0evy4t8SWp8imEsylIk=
5+
github.com/PaesslerAG/jsonpath v0.1.1/go.mod h1:lVboNxFGal/VwW6d9JzIy56bUsYAP6tH/x80vjnCseY=
6+
github.com/bhmj/jsonslice v1.1.2 h1:Lzen2S9iG3HsESpiIAnTM7Obs1QiTz83ZXa5YrpTTWI=
7+
github.com/bhmj/jsonslice v1.1.2/go.mod h1:O3ZoA0zdEefdbk1dkU5aWPOA36zQhhS/HV6RQFLTlnU=
8+
github.com/bhmj/xpression v0.9.1 h1:N7bX/nWx9oFi/zsiMTx2ehoRApTDAWdQadq/5o2wMGk=
9+
github.com/bhmj/xpression v0.9.1/go.mod h1:j9oYmEXJjeL9mrgW1+ZDBKJXnbupsCPGhlO9J5YhS1Q=
10+
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
11+
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
12+
github.com/evilmonkeyinc/jsonpath v0.7.0 h1:jHar8KiQNNhobwKpftn5SzqaVGbGbNb+pWm+BaRQiQw=
13+
github.com/evilmonkeyinc/jsonpath v0.7.0/go.mod h1:exI6Yme6vucFk2m/1iVtxPaHGI17Dy9uWfsqbJ4iGHM=
14+
github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 h1:Yl0tPBa8QPjGmesFh1D0rDy+q1Twx6FyU7VWHi8wZbI=
15+
github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852/go.mod h1:eqOVx5Vwu4gd2mmMZvVZsgIqNSaW3xxRThUJ0k/TPk4=
16+
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
17+
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
18+
github.com/spyzhov/ajson v0.7.0 h1:iDvpu9sYuWS2QvWjne/hJM7kdlgA6xyJCH7/y+kEbMI=
19+
github.com/spyzhov/ajson v0.7.0/go.mod h1:63V+CGM6f1Bu/p4nLIN8885ojBdt88TbLoSFzyqMuVA=
20+
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
21+
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
22+
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
23+
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
24+
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
25+
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

0 commit comments

Comments
 (0)