A teaching lab for the SWIM membership and failure-detection protocol — in the style of the MIT 6.5840 labs. You implement the protocol; a deterministic test suite grades it. Ships with a reference implementation.
SWIM libraries are everywhere — hashicorp/memberlist, ScaleCube, Corrosion. A hands-on lab with a test suite — the thing that turns "I read the SWIM paper" into "I built a failure detector that stays quiet under packet loss" — did not exist. This is that lab.
It is the reference implementation promised by §4 of the Distributed Systems Engineer Roadmap, and its headline test is that roadmap's §4.1 milestone made executable.
You're done when your SWIM-based membership converges on a 20-node cluster, detects a real death within the window, and produces zero false positives under 10% packet loss — i.e.
go test ./...is green.
That last clause is the whole point. Detecting a node that is actually dead is easy. Not declaring a live node dead when the network is dropping one packet in ten is what separates SWIM from a naïve heartbeat — and it is exactly what this suite checks.
Ping each node in turn; if an ack doesn't come back, call it dead. That works perfectly — until packets drop. Then one lost ack becomes a phantom death. This repo ships that naïve detector too, behind a build tag, so you can watch it happen:
$ go test -tags naive ./...
--- PASS: TestDetectDeath # a real death, perfect network — fine
--- FAIL: TestNoFalsePositivesUnderLoss
seed 1: 380 false positive(s) — a live node was wrongly declared dead under 10% loss
--- FAIL: TestDetectDeathUnderLoss
seed 1: 340 false positive(s) on the survivors
--- FAIL: TestRefutation
Three things close that gap, and they are the three things you implement:
- Indirect probing — when a direct ping goes unanswered, ask k other nodes to probe the target too. One dropped packet no longer looks like a death.
- Suspicion — a missed node is suspected, not killed. It gets a grace period to prove itself.
- Refutation via incarnation numbers — a live node that hears it is suspected bumps its incarnation and floods "I'm alive," overriding the rumor.
The network is simulated and deterministic — no wall-clock timers, no goroutine races. Time advances one logical tick at a time; packet loss comes from a seeded RNG; the harness can kill or isolate nodes. The same seed always produces the same history, so a failing test always reproduces. (This is itself milestone §8.2 of the roadmap — deterministic simulation testing — applied to the lab that teaches §4.)
| File | What it is |
|---|---|
api.go |
Shared types (given — don't edit) |
sim.go |
The deterministic network + the assertions the tests build on (given) |
swim_test.go |
The grader (given) |
swim.go |
The reference implementation (default build) |
swim_starter.go |
A skeleton with TODOs — your starting point (-tags starter) |
swim_naive.go |
The naïve detector, built to fail one test on purpose (-tags naive) |
Reimplement the node in a copy of swim.go (or fill in swim_starter.go) until the suite is green.
go test ./... # grade your implementation (default build = the reference)
go test -tags starter ./... # the empty skeleton — everything red, your starting point
go test -tags naive ./... # the naïve detector — green on a perfect net, red under loss
go test -run TestDetectDeathUnderLoss -v ./... # the flagship 20-node milestone
Each test is a checkpoint. Implement toward them top to bottom.
TestJoinConvergence— You're done when a 10-node cluster started from a single seed agrees on the full membership. (Gossip works.)TestDetectDeath— You're done when a killed node is agreed dead by everyone on a perfect network, and no live node is ever wrongly declared dead.TestRefutation— You're done when a node told it is suspected bumps its incarnation and is restored to Alive across the cluster.TestNoFalsePositivesUnderLoss— You're done when 20 nodes run for 120 periods under 10% loss with nobody dead, and zero live nodes are ever declared dead.TestDetectDeathUnderLoss— the flagship: You're done when a 20-node cluster under 10% loss detects a real death within the window and produces zero false positives on the survivors.
You have finished the lab — and cleared roadmap milestone §4.1 — when go test ./... passes.
MIT — see LICENSE.