1
1
<img src =" ./.github/logo_200x200.png " alt =" sno logo " title =" sno " align =" left " height =" 200 " />
2
2
3
- A spec for ** unique IDs in distributed systems** based on the Snowflake design, e.g . a coordination-based ID variant.
3
+ A spec for ** unique IDs in distributed systems** based on the Snowflake design, i.e . a coordination-based ID variant.
4
4
It aims to be friendly to both machines and humans, compact, * versatile* and fast.
5
5
6
6
This repository contains a ** Go** package for generating such IDs.
@@ -29,10 +29,10 @@ go get -u github.com/muyo/sno
29
29
(256 combinations) per tick-tock (1 bit adjustment for clock drifts ➜ [ Time and sequence] ( #time-and-sequence ) ).
30
30
** 549,755,813,888,000** is the global pool ** per second** when the metabyte, partition and tick-tock are
31
31
taken into account.
32
- - Data layout straightforward to inspect or encode/decode (➜ [ Layout] ( #layout ) )
32
+ - ** Simple data layout** - straightforward to inspect or encode/decode (➜ [ Layout] ( #layout ) )
33
33
- Configuration and coordination optional (➜ [ Usage] ( #usage ) )
34
- - Fast, wait-free, safe for concurrent use (➜ [ Benchmarks] ( #benchmarks ) )
35
- - Fills a niche (➜ [ Alternatives ] ( #alternatives ) )
34
+ - ** Fast** , wait-free, safe for concurrent use (➜ [ Benchmarks] ( #benchmarks ) )
35
+ < br />Clocks in at just over 500 LoC, has no external dependencies and minimal dependencies on std.
36
36
37
37
### Non-features / cons
38
38
@@ -46,7 +46,7 @@ unpredictability of IDs is a must. They still, however, meet the common requirem
46
46
47
47
<br />
48
48
49
- ## Usage
49
+ ## Usage (➜ [ API reference ] ( https://pkg.go.dev/github.com/muyo/sno?tab=doc ) )
50
50
51
51
** sno** comes with a package-level generator on top of letting you configure your own generators.
52
52
@@ -57,15 +57,14 @@ id := sno.New(0)
57
57
```
58
58
59
59
Where ` 0 ` is the ➜ [ Metabyte] ( #metabyte ) .<br />
60
- Vide [ godoc] ( https://pkg.go.dev/github.com/muyo/sno?tab=doc ) for the entire API.
61
60
62
61
The global generator is immutable and private. It's therefore also not possible to restore it using a Snapshot.
63
- It uses 2 random bytes as its partition, meaning its partition changes across restarts.
62
+ It uses 2 pseudo random bytes as its partition, meaning its partition changes across restarts.
64
63
65
64
* As soon as you run more than 1 generator, you ** should** start coordinating* the creation of generators to
66
65
actually * guarantee* a collision-free ride. This applies to all ID specs of the Snowflake variant.
67
66
68
- ### Partitions
67
+ ### Partitions (➜ [ doc ] ( https://pkg.go.dev/github.com/muyo/sno?tab=doc#Partition ) )
69
68
70
69
Partitions are one of several friends you have to get you those guarantees. A partition is 2 bytes.
71
70
What they mean and how you define them is up to you.
@@ -79,7 +78,7 @@ generator, err := sno.NewGenerator(&sno.GeneratorSnapshot{
79
78
Multiple generators can share a partition by dividing the sequence pool between
80
79
them (➜ [ Sequence sharding] ( #sequence-sharding ) ).
81
80
82
- ### Snapshots
81
+ ### Snapshots (➜ [ doc ] ( https://pkg.go.dev/github.com/muyo/sno?tab=doc#GeneratorSnapshot ) )
83
82
84
83
Snapshots happen to serve both as configuration and a means of saving and restoring generator data. They are
85
84
optional - simply pass ` nil ` to ` NewGenerator() ` , and what you get will be configured like the package-level generator,
@@ -91,7 +90,7 @@ Snapshots can be taken at runtime:
91
90
s := generator.Snapshot ()
92
91
```
93
92
94
- This exposes most of a generator's internal bookkeeping data. In an ideal world where programmers are not lazy even
93
+ This exposes most of a generator's internal bookkeeping data. In an ideal world where programmers are not lazy
95
94
until their system runs into an edge case - you'd persist that snapshot across restarts and restore generators
96
95
instead of just creating them from scratch each time. This will keep you safe both if a large clock drift happens
97
96
during the restart -- or before, and you just happen to come back online again "in the past", relative to IDs that
@@ -202,7 +201,7 @@ generators.
202
201
203
202
It is safe for a range previously used by another generator to be assigned to a different generator under the
204
203
following conditions:
205
- - it happens in a different timeframe * in the future* , e.g . no sooner than after 4msec have passed (no orchestrator
204
+ - it happens in a different timeframe * in the future* , i.e . no sooner than after 4msec have passed (no orchestrator
206
205
is fast enough to get a new container online to replace a dead one for this to be a worry);
207
206
- if you can guarantee the new Generator won't regress into a time the previous Generator was running in.
208
207
@@ -322,7 +321,7 @@ without requiring slew mode nor having to worry about even large drifts.
322
321
** sno** attempts to eliminate the issue * entirely* - both despite and because of its small pool of bits to work with.
323
322
324
323
The approach it takes is simple - each generator keeps track of the highest wall clock time it got from the OS\* ,
325
- each time it generates a new timestamp. If we get a time that is lower than the one we recorded, e.g . the clock
324
+ each time it generates a new timestamp. If we get a time that is lower than the one we recorded, i.e . the clock
326
325
drifted backwards and we'd risk generating colliding IDs, we toggle a bit - stored from here on out in
327
326
each ** sno** generated * until the next regression* . Rinse, repeat - tick, tock.
328
327
@@ -344,7 +343,7 @@ So this is a **sno-go**.
344
343
(I will show myself out...)
345
344
346
345
The simplistic approach of tick-tocking * entirely eliminates* that collision chance - but with a rigorous assumption:
347
- regressions happen at most once into a specific period, e.g . from the highest recorded time into the past
346
+ regressions happen at most once into a specific period, i.e . from the highest recorded time into the past
348
347
and never back into that particular timeframe (let alone even further into the past).
349
348
350
349
This * generally* is exactly the case but oddities as far as time synchronization, bad clocks and NTP client
@@ -412,7 +411,7 @@ And simple constants tend to do the trick.
412
411
</summary >
413
412
<p >
414
413
415
- Untyped integers can pass as ` uint8 ` (e.g . ` byte ` ) in Go, so the following would work and keep things tidy:
414
+ Untyped integers can pass as ` uint8 ` (i.e . ` byte ` ) in Go, so the following would work and keep things tidy:
416
415
417
416
``` go
418
417
const (
@@ -605,15 +604,15 @@ unexpected in the parallel test (branch prediction?) but remained consistent.
605
604
606
605
** Snowflakes**
607
606
608
- What does ` Unbounded ` mean? [ xid] , for example, is unbounded, e.g . it does not prevent you from generating more IDs
607
+ What does ` Unbounded ` mean? [ xid] , for example, is unbounded, i.e . it does not prevent you from generating more IDs
609
608
than it has a pool for (nor does it account for time). In other words - at high enough throughput you simply and
610
609
silently start overwriting already generated IDs. * Realistically* you are not going to fill its pool of
611
610
a 16,777,216 capacity. But it does reflect in synthetic benchmarks. [ Sandflake] does not bind nor handle clock
612
611
drifts either. In both cases their results are ` WYSIWYG ` .
613
612
614
613
The implementations that do bind, approach this issue (and clock drifts) differently. [ Sonyflake] goes to sleep,
615
614
[ Snowflake] spins aggressively to get the OS time. ** sno** , when about to overflow, starts a single timer and
616
- locks all overflowing requests on a condition, waking them up when the sequence resets, e.g . time changes.
615
+ locks all overflowing requests on a condition, waking them up when the sequence resets, i.e . time changes.
617
616
618
617
Both of the above are edge cases, * realistically* - Go's benchmarks happen to saturate the capacities and hit
619
618
those cases. ** Most of the time what you get is the unbounded overhead** . Expect said overhead of the
0 commit comments