Slow write speeds

[petermakeswebsites]

Is it normal to need ~40-70ms for one write on an SSD and a decent processor?

type PostType struct {
	Name       string   `json:"name"`
	PluralName string   `json:"pluralName"`
	Slug       string   `json:"slug" boltholdUnique:"Slug"`
	PostTypeID uint64   `json:"postTypeID" boltholdKey:"PostTypeID"`
	Fields     FieldMap `json:"fields"`
}

type FieldMap map[uint64]Field

type Field struct {
	Name    string
	Kind    string
	Slug    string
	Options string
}

...

pt := PostType{......}

x := time.Now()
err := d.Insert(bolthold.NextSequence(), &pt)
fmt.Println("Creation time: ", time.Since(x))

Creation time usually hangs out around 40-70ms

[timshannon]

40-70 ms is an eternity. To start troubleshooting the issue the first thing I'd try is removing bolthold from the equation entirely. Just do inserts directly into boltdb, and see if it's still taking that long.

[petermakeswebsites]

I actually switched to the sister version - badgerhold - and it went down to 20ms... Still crazy long IMO. How much time is considered normal in your experience for a single write? Just so I have a vague target for my debugging. I'll try doing it directly to Bolt instead and see what happens

[timshannon]

I'll throw together some quick timings on my machine, which has spinning metal and ssd's.

BadgerDB's storage format is designed specifically to take advantage of SSD's where as Bolt doesn't.

https://dgraph.io/blog/post/badger/

Looking at dgraph's benchmarks you should see inserts at around 9ms depending on the size of the insert.

And if there are index inserts, then you should double that number 9ms for the main insert, and another 9 ms for the index insert.

[petermakeswebsites]

So I tried to access bolt directly via boltholdstore.db.Bolt()...

All I can say is wow. I wrote my own pretty straightforward put transaction with a bucket NextSequence. The tx.commit part of the tx alone takes ~60ms. That's just crazy. I was working on my own database previously and was getting about 70 us (0.07ms) per write with Badger. Granted, badger is faster than Bolt... but... a thousand times faster is definitely fishy. No clue what could be slowing it down but I'll continue to poke around.

It seems at this point that this isn't a bolthold thing at least. Though, it's still possible that some init setting is causing this slow write speed. Not sure.

I'll do another test setting up my own bolt db instead of accessing it through bolthold and see if it makes a difference. Can't imagine it will, but then at least then I can totally rule out bolthold.

[timshannon]

Thanks for relaying the info. I know boltdb and badger have very different goals, where as bolt aims for simplicity and (https://github.com/boltdb/bolt#leveldb-rocksdb) and badger is designed for with a file format catered towards SSDs.

[HeavyHorst]

I was working on my own database previously and was getting about 70 us (0.07ms) per write with Badger.

BadgerDB doesn't sync writes with the default config as far as i know.

  package main                                                                                                                                                                                                       
                                                                                                                                                                                                                     
  import (                                                                                                                                                                                                           
      "fmt"                                                                                                                                                                                                          
      "time"                                                                                                                                                                                                         
                                                                                                                                                                                                                     
      "github.com/timshannon/badgerhold"                                                                                                                                                                             
  )                                                                                                                                                                                                                  
                                                                                                                                                                                                                     
  type PostType struct {                                                                                                                                                                                             
      Name       string   `json:"name"`                                                                                                                                                                              
      PluralName string   `json:"pluralName"`                                                                                                                                                                        
      PostTypeID uint64   `json:"postTypeID" boltholdKey:"PostTypeID"`                                                                                                                                               
      Fields     FieldMap `json:"fields"`                                                                                                                                                                            
  }                                                                                                                                                                                                                  
                                                                                                                                                                                                                     
  type FieldMap map[uint64]Field                                                                                                                                                                                     
                                                                                                                                                                                                                     
  type Field struct {                                                                                                                                                                                                
      Name    string                                                                                                                                                                                                 
      Kind    string                                                                                                                                                                                                 
      Slug    string                                                                                                                                                                                                 
      Options string                                                                                                                                                                                                 
  }                                                                                                                                                                                                                  
                                                                                                                                                                                                                     
  func main() {                                                                                                                                                                                                      
      options := badgerhold.DefaultOptions                                                                                                                                                                           
      options.Dir = "data"                                                                                                                                                                                           
      options.ValueDir = "data"                                                                                                                                                                                      
      options.SyncWrites = false                                                                                                                                                                                     
      d, err := badgerhold.Open(options)                                                                                                                                                                             
      if err != nil {                                                                                                                                                                                                
          panic(err)                                                                                                                                                                                                 
      }                                                                                                                                                                                                              
                                                                                                                                                                                                                     
      pt := PostType{}                                                                                                                                                                                               
                                                                                                                                                                                                                     
      for i := 0; i < 100; i++ {                                                                                                                                                                                     
          x := time.Now()                                                                                                                                                                                            
          err = d.Insert(badgerhold.NextSequence(), &pt)                                                                                                                                                             
          fmt.Println(err)                                                                                                                                                                                           
          fmt.Println("Creation time: ", time.Since(x))                                                                                                                                                              
      }                                                                                                                                                                                                              
  }

With this config i get (badgerhold):
Creation time: 154.381µs
Creation time: 108.543µs
Creation time: 140.977µs
Creation time: 120.703µs
...

With activated sync writes and badgerhold (saver):
Creation time: 13.996905ms
Creation time: 24.53465ms
Creation time: 10.498898ms
Creation time: 18.433725ms
....

With Bolthold:
Creation time: 32.025662ms
Creation time: 46.942041ms
Creation time: 45.16791ms
Creation time: 37.354526ms
...

With bolthold and NoSync flag=true:
Creation time: 151.732µs
Creation time: 147.138µs
Creation time: 114.893µs
Creation time: 132.968µs
...

This is on a relatively slow pc:
Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
Samsung SSD 840 Series SATA 120GB SSD

[HeavyHorst]

Same test on my Notebook:
Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
SanDisk SD8SN8U512G1002 512GB M2 SSD

With this config i get (badgerhold):
Creation time: 39.432µs
Creation time: 38.369µs
Creation time: 45.024µs
Creation time: 54.537µs

With activated sync writes and badgerhold (saver):
Creation time: 1.762754ms
Creation time: 1.832461ms
Creation time: 1.720504ms
Creation time: 1.809905ms

With Bolthold:
Creation time: 1.632288ms
Creation time: 1.702195ms
Creation time: 1.65075ms
Creation time: 1.685826ms

With bolthold and NoSync flag=true:
Creation time: 70.892µs
Creation time: 67.832µs
Creation time: 76.844µs
Creation time: 96.243µs