diff --git a/benchmarking/routercap/RESULTS.md b/benchmarking/routercap/RESULTS.md new file mode 100644 index 000000000..34caa2dae --- /dev/null +++ b/benchmarking/routercap/RESULTS.md @@ -0,0 +1,252 @@ +# atenet-router capacity + +> ***Conclusion:*** scale `atenet-router` horizontally past 10,000 QPS. Keep +> router pods at 4 to 8 cores and add pods, not cores. For example, to serve +> 50,000 QPS, run five 8-core routers rather than one large one. + +## Test environment + +Run `runs/v130-w200`. Charts and every measured window are in +that run's `report.html`. + +* **atenet-router:** one pod (Envoy v1.30 + the Go ext_proc sidecar) alone on + a tainted `c3-standard-88` node. Envoy's CPU limit is the variable under + test; the sidecar's limit is 8 cores in every arm. +* **Workers:** 200 pods on 4 tainted `c3-standard-88` nodes, one pre-warmed + actor per pod. +* **Generator:** one pod alone on its own `c3-standard-88` node, dialing the + router pod IP directly (no DNS or kube-proxy in the measured path). +* **Load:** each arm walks a ladder that rises 1,000 QPS per rung, 45 s per + rung. The first 10 s of every rung are discarded so that no window blends + the ramp to the new rate with its steady state. +* **Arms:** 2, 4, 8 and 16 cores with matched thread counts, three + thread-count variants on the 8-core limit (2, 4 and 16 threads), and one + two-replica arm (2 × 4 cores). + +## Summary + +Sustained QPS is the highest rung where completed and successful requests +both stayed within 1% of offered and no window's median latency exceeded +100 ms. Latency columns show that rung's worst window; the last column shows +the next rung, where the arm has already failed the criterion. + +| Topology | Sustained QPS | p50 | p95 | Envoy CPU | % of limit | Next rung, p50 | +|---|---|---|---|---|---|---| +| 1 × 2c | 5,000 | 6.7 ms | 251.1 ms | 1.86 | 93% | 6,000 @ 1,539 ms | +| 1 × 4c | 9,000 | 14.2 ms | 47.1 ms | 3.47 | 87% | 10,000 @ 550 ms | +| 1 × 8c | 11,000 | 11.8 ms | 71.5 ms | 5.19 | 65% | 12,000 @ 23 ms | +| 1 × 16c | 13,000 | 62.4 ms | 252.3 ms | 8.49 | 53% | 14,000 @ 232 ms | +| 2 × 4c | 16,000 | 21.6 ms | 77.4 ms | 3.53 per pod | 88% | 17,000 @ 138 ms | + +What the table says: + +* The return on cores falls as they are added: each doubling buys half the + previous increment (+4,000, +2,000, +1,000). That is the signature of a + shared cost that grows with core count, not of any single exhausted + resource. +* Only the small arms are CPU-bound. The 2- and 4-core arms die at 93% and + 87% of their limit; the 8- and 16-core arms die at 65% and 53%, with cores + to spare. +* Splitting a pod beats growing it. Two 4-core replicas on one node sustained + 16,000 against 11,000 for one 8-core pod on the same silicon. Each replica + carried 8,000 QPS at the pair's wall — 89% of the 9,000 a 4-core pod + sustains running alone, so almost nothing is lost to sharing the node. +* Do not move to Envoy v1.39 until its regression is understood: it sustained + 20% to 38% less on every arm (measured below). + +## Bottlenecks in vertical scaling + +Why one Envoy stops absorbing load as it grows. + +### Every added worker thread taxes every request + +The same load costs 19% more CPU on 8 threads than on 2, and 38% more on 16, +because each extra event loop wakes more often for less work per wake, and +every wake has a fixed kernel cost. + +**Context:** Envoy parallelises with worker threads. `--concurrency` sets how +many event loops it runs, every connection is served by one of them, and each +loop sleeps in the kernel until work arrives. To separate the thread count +from the cores, diagnostic arms pinned the CPU limit at 8 cores and varied +only `--concurrency`. + +Envoy CPU, in cores, every column on the same 8-core limit (mean over the +rung's windows): + +| Offered QPS | 2 threads | 4 threads | 8 threads | 16 threads | +|---|---|---|---|---| +| 3,000 | 1.29 | 1.20 | 1.29 | 1.43 | +| 4,000 | 1.60 | 1.68 | 1.62 | 1.95 | +| 5,000 | 1.86 | 2.10 | 2.21 | 2.57 | + +Each rung holds 45 seconds, so a cell averages two or three windows; at low +rates the differences are smaller than window-to-window wobble (that is how +4 threads reads cheaper than 2 at 3,000 QPS). The 5,000 row is stable and is +the one to quote. + +A perf profile taken mid-ladder (30 s of stack samples at 99 Hz while the +2-thread and the 8-thread configurations each held 4,000 QPS) shows where the +extra CPU goes: kernel scheduling. Context-switch time nearly doubles its +share and `epoll_wait` triples, while per-syscall batch work falls. The event +stream is fixed by the request rate, so more loops means each wakes more +often for fewer events per wake. Lock contention is not the cause: mutex +tracing showed ~50 ms of blocked-acquisition wait per 15-second window across +all threads, which is noise. + +### Skewed threads + +One overloaded thread can fail the pod while container CPU reads 29% +utilized, because container CPU is a sum across threads and a sum cannot show +the spread. + +**Context:** Envoy assigns each downstream connection to one worker thread +for life, so threads do not share load. One thread drowning plus seven +napping reads the same as eight threads quarter-busy. + +The 8-core arm's collapse windows measured the first case directly: the eight +workers read 0.91, 0.72, 0.31, 0.26, 0.06, 0.02, 0.00, 0.00 cores in the +worst window. The thread holding the busiest connections was near saturation +with its requests queueing, two threads did nothing, and the container total +showed capacity to spare. The collapse then compounds: waiting requests hold +their connections, the generator dials fresh ones, and each TLS handshake is +real work for whichever thread it lands on. Monitor per-thread CPU, not the +container average. The "hottest worker" vs "mean worker" series on the +per-thread CPU panel in [report.html](runs/v130-w200/report.html) plots +exactly this; the lines hugging is balanced load, a gap opening is skew. + +### Router port ceiling + +The router node's ephemeral port range caps concurrent upstream connections +at 28,232; it is the operating-system limit that everything else is ordered +under. + +**Context:** Envoy holds each request open end to end, so every in-flight +request owns one connection, and one source port, toward the workers. The +28,232 is the Linux default `net.ipv4.ip_local_port_range` of 32768-60999 +(60999 − 32768 + 1 ports), measured from the live pod rather than assumed. + +Nothing in this study reached the ceiling, and that is by design: the circuit +breakers below sit under it so that overload surfaces as a counted Envoy +rejection rather than the kernel's opaque connect failure. + +### Envoy limits that must be raised + +Two circuit breakers default far below these capacities; left at their +defaults they would have opened well below every wall and been misread as +router capacity. + +* The ext_proc cluster's `max_requests` defaults to twice the sidecar's + parking lot, 2,048. Every in-flight request holds one ext_proc stream, so + this breaker opens first of everything; the sweep raised it to 20,000 with + `--extproc-max-requests` (the shipped default is unchanged — raise the + flag for pods expected to carry more in flight). +* The actor cluster configured no breakers at all, leaving Envoy's default of + 1,024 against arms that reach 13,000 to 26,000 in flight. Set to 20,000 in + `xds.go`, below the 28,232-port ceiling so the breaker trips before the + kernel does. + +Did the raised limits trip during the runs? Below every wall, never — no +overflow counter moved in any measured rung, so no sustained figure is +breaker-shaped. During post-wall collapse spirals the ext_proc cluster's +pending-request queue, which sat at Envoy's 1,024 default during these runs +(the flag's ceiling covers it too since then), overflowed and +shed up to ~30,000 requests in a window; the actor cluster's raised breakers +never opened at all. + +### Envoy v1.39 regresses capacity + +Upgrading the proxy from v1.30 to v1.39 cost 20% to 38% of sustained QPS on +every arm, and the loss grows with thread count, which points at per-thread +overhead rather than uniform slowdown. + +**Context:** v1.39 was tested for its worker-pinning flag (next section), so +the version moved first, alone, flag off, same everything else. Full run: +[runs/v139-w200/report.html](runs/v139-w200/report.html). + +| Arm | v1.30 | v1.39 | Change | +|---|---|---|---| +| 2c | 5,000 | 4,000 | −20% | +| 4c | 9,000 | 6,000 | −33% | +| 8c/2t | 5,000 | 4,000 | −20% | +| 8c/4t | 9,000 | 6,000 | −33% | +| 8c/8t | 11,000 | 7,000 | −36% | +| 16c | 13,000 | 8,000 | −38% | + +v1.39 costs ~25% more CPU per request at every rate, and its event loops +stretch an order of magnitude earlier (worst per-worker iteration at 3,000 +QPS: 1,108 µs against v1.30's 89 µs). Both structural findings replicate on +v1.39: the quota stays a bystander and returns on threads still fall. Worth +reporting upstream; the cluster and manifests stay on v1.30. + +### CPU pinning regresses capacity further, but improves latency + +On the v1.39 16-core arm, turning on `enable_worker_cpu_affinity` dropped +sustained QPS one more rung, from 8,000 (v1.39, flag off) to 7,000 — while +below the wall it delivered visibly better latency: p50 3.4 ms against 6.3 ms +at 5,000 QPS, and steadier event loops. A trade of peak capacity for +smoothness, and on this platform a bad one. + +**Context:** the flag (new in v1.39) pins worker i to CPU i, and its +documented rationale is bare metal, where pinned cores are otherwise quiet. +On a GKE VM they are not: network interrupt work lands on the same +low-numbered cores, and an unpinned worker escapes to any of ~70 idle vCPUs +while a pinned one works buried. Pinning was verified live (workers 0-15 each +locked to their own physical core; on this machine type, vCPU i and i+44 are +hyperthread siblings, so the first 16 vCPUs are 16 distinct cores). The +pinned arm is `arm-16c-pinned` in the v1.39 report linked above. + +### Finding: the thread count is the arm + +Capacity follows the thread count, not the CPU limit — N threads can use at +most ~N cores, so the limit only matters once the threads can spend it — and +adding threads past the cores buys one more rung by hiding the wake-up stall. + +**Context:** an arm normally raises the CPU limit and `--concurrency` +together, so which of the two sets capacity is invisible in the main sweep. +The decoupled arms hold the limit at 8 cores, vary only the thread count, and +run each ladder to its wall. + +| Threads on 8 cores | Sustained QPS | Envoy cores at the wall | How it died | +|---|---|---|---| +| 2 | 5,000 | 1.87 | thread-bound — the 2-core arm's numbers exactly | +| 4 | 9,000 | 3.45 | thread-bound — the 4-core arm's numbers exactly | +| 8 | 11,000 | 5.19 (65%) | idle-stalled | +| 16 | 12,000 | 7.32 (92%) | CPU spent: collapsed at 14,000 at 7.94 of 8 | + +Six spare cores of quota changed nothing for the 2- and 4-thread arms: for +capacity, the thread count *is* the arm. In the other direction, 16 threads +on the same limit sustain one rung more than the matched 8, because while one +thread waits for the kernel to wake it another has work in hand — and it is +the only configuration in the study that died with its CPU actually spent. +The price is the thread tax in full (2.57 cores at 5,000 QPS against the +8-thread arm's 2.21) and a wall p50 of 38 ms against 5.8 ms. Capacity tracks +threads with falling returns, and 16 threads on 16 cores reach only 13,000: +the wake-up stall, not the quota and not the tax, is what sets the wall. +Treat `--concurrency` at twice the cores as a lever for a pod that cannot be +split, not as a strategy. + +## Harness ceilings (configurable, not bottlenecks) + +The rig has its own limits. Guards watch them and abort an arm when the rig, +not the router, runs out; in this run every fatal trip landed past the +sustainable rung, so the trips are a footnote to valid numbers, not a warning +about them. (One of those trips is visible in the summary table: the 4-core +arm's next rung reads 550 ms from its ramp-up window, after which the +keep-alive guard ended the arm.) + +### Generator port ceiling + +The generator's source ports cap how many connections it can hold to the +router. The Job spec widens the range to 64,511 ports (`ip_local_port_range` +1025-65535), the binary reads its own range at startup, and the guard trips +at 80% of whatever it measured, so the ceiling follows the spec without +recompiling. + +### Worker count + +Each actor cliffs near 120 requests per second: the measured worker hop rises +from 2.8 ms at 110/s to 62 ms at 120/s. Size the pool at or above target QPS +divided by 120, or the workers become the wall before the router does. The +200-pod pool kept this study router-bound; doubling from 100 to 200 workers +moved the walls by at most one rung, which is what demoted the workers from +cause to co-limiter. diff --git a/benchmarking/routercap/runs/v130-w200/report.html b/benchmarking/routercap/runs/v130-w200/report.html new file mode 100644 index 000000000..2fa543e1a --- /dev/null +++ b/benchmarking/routercap/runs/v130-w200/report.html @@ -0,0 +1,8363 @@ + + + +atenet-router capacity + +
+
+

atenet-router capacity

+
Generated by running python3 benchmarking/routercap/charts.py benchmarking/routercap/runs/v130-w200
+
+ +

Headline

+
Sustainable is the highest rung whose measured windows, summed, kept both completed and +successful requests within 1% of what was offered, with no window's median above 100 ms.
+
2c
5,000 qps sustainable
held through rung 4 of 5 · 2,500 qps per core
4c
9,000 qps sustainable
held through rung 8 of 9 · 2,250 qps per core
2×4c
16,000 qps sustainable
held through rung 15 of 17 · 2,000 qps per core
8c/2t
5,000 qps sustainable
held through rung 4 of 6 · 625 qps per core
8c/4t
9,000 qps sustainable
held through rung 8 of 9 · 1,125 qps per core
8c/8t
11,000 qps sustainable
held through rung 10 of 12 · 1,375 qps per core
8c/16t
12,000 qps sustainable
held through rung 11 of 13 · 1,500 qps per core
16c
13,000 qps sustainable
held through rung 12 of 14 · 812 qps per core
+ +

Legend

+
Throughput panel
  • offered — requests the pacer scheduled — the independent variable
  • achieved — requests that came back at all
  • success — the achieved subset that returned HTTP 200
  • in-flight — sent and not yet answered — a count, not a rate
  • client pool — TCP connections the generator holds open; steps up when a stall makes blocked requests dial, and holds for the 120 s idle timeout
Time-share panel (stacked bands, bottom up)
  • before Envoy — queued in the generator, plus the dial
  • Envoy itself — Envoy's own work plus the loopback to the sidecar
  • sidecar — the ext_proc handler, actor resume included
  • worker leg — request on the wire to the worker and back
  • hatched — Envoy rounds request times to whole milliseconds; where hatched, that rounding exceeds 5% of the mean request and the split is mostly instrument
CPU and memory panels
  • envoy — the proxy — its CPU limit is the variable under test
  • atenet-router — the Go ext_proc sidecar that picks the worker and resumes the actor; same limit in every arm
Per-thread CPU panel
  • hottest worker — the busiest single Envoy thread (from /proc): a connection stays on its thread for life, so one thread can saturate while the container average reads idle
  • mean worker — the same threads' average — the lines hugging is balanced load, a gap opening is skew: one thread drowning; at 1.0 the hottest thread is saturated outright
Chart furniture
  • grey stripes — alternate rungs, labelled R0, R1, … along the top
  • dashed lines — ceilings, each labelled at the line itself. "envoy sheds above 20,000 in flight" is the actor cluster's circuit breaker: past it Envoy answers 503 itself and counts the rejection, set below the port budget on purpose so overload shows up as an Envoy counter instead of the kernel's opaque connect failure
  • gaps — a hole in a CPU or memory line is a window where cAdvisor returned no fresh sample — nobody looked is not the same as it went idle
+ +

Per arm, over time

+
One chart per arm. The panels share one x-axis and one set of window boundaries, so a +vertical line through them is a single interval. Hover any column for that window's full numbers.
+
+ + + +atenet-router capacity — 2c +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 + +0 + +10k + +20k + +30k +requests / s + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + +0 + +0.5 + +1.0 + +1.5 + +2.0 +cores + + + + + + + +envoy CPU limit, 2 cores — the variable under test + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + +0 + +500.0M + +1.0G +bytes + +50.0 + +100 + +150 + +200 + +250 +seconds since the arm started + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + +atenet-router +rung 0 1000 qps setpoint [warmup]------------------------------offered 966 /sachieved 965 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 32.0 pool 32.0------------------------------envoy 0.46c sidecar 0.46cwrk hottest 0.36c mean 0.21c------------------------------mean request 3.13 ms before 1.04 envoy 0.68 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 517 pool 517------------------------------envoy 0.46c sidecar 0.50cwrk hottest 0.37c mean 0.23c------------------------------mean request 14.72 ms before 9.74 envoy 1.30 sidecar 2.75 worker 0.93 resume 2.71 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1371 /sachieved 1371 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 56.0 pool 517------------------------------envoy 0.64c sidecar 0.56cwrk hottest 0.63c mean 0.33c------------------------------mean request 3.09 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 63.0 pool 517------------------------------envoy 0.88c sidecar 0.87cwrk hottest 0.70c mean 0.44c------------------------------mean request 3.17 ms before 1.19 envoy 0.63 sidecar 0.80 worker 0.55 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 73.0 pool 517------------------------------envoy 0.89c sidecar 0.88cwrk hottest 0.58c mean 0.44c------------------------------mean request 3.02 ms before 1.08 envoy 0.68 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2621 /sachieved 2621 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 228 pool 517------------------------------envoy 1.16c sidecar 0.97cwrk hottest 0.75c mean 0.58c------------------------------mean request 3.69 ms before 1.43 envoy 0.73 sidecar 0.82 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 62.0 pool 517------------------------------envoy 1.25c sidecar 1.15cwrk hottest 0.84c mean 0.62c------------------------------mean request 3.43 ms before 1.29 envoy 0.60 sidecar 0.81 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3015 /sachieved 3014 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 386 pool 517------------------------------envoy 1.26c sidecar 1.13cwrk hottest 0.73c mean 0.59c------------------------------mean request 4.12 ms before 1.66 envoy 0.66 sidecar 0.96 worker 0.84 resume 0.92 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 138 pool 386------------------------------envoy 1.60c sidecar 1.21cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.65 ms before 1.38 envoy 0.54 sidecar 0.81 worker 0.92 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 4.9 msin-flight 33.0 pool 386------------------------------envoy 1.57c sidecar 1.39cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.48 ms before 1.33 envoy 0.53 sidecar 0.80 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 33.0 pool 386------------------------------envoy 1.58c sidecar 1.40cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.52 ms before 1.34 envoy 0.52 sidecar 0.80 worker 0.87 resume 0.76 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4820 /sachieved 4820 /s (100.0% ok)p50 5.3 ms p95 303.2 msin-flight 1724 pool 1724------------------------------envoy 1.81c sidecar 1.45cwrk hottest 0.95c mean 0.90c------------------------------mean request 44.83 ms before 18.56 envoy 13.77 sidecar 1.08 worker 11.42 resume 1.04 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.4 ms p95 251.1 msin-flight 2068 pool 2070------------------------------envoy 1.82c sidecar 1.46cwrk hottest 0.94c mean 0.91c------------------------------mean request 28.05 ms before 5.36 envoy 12.78 sidecar 1.85 worker 8.06 resume 1.09 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5074 /sachieved 5066 /s (100.0% ok)p50 6.7 ms p95 46.7 msin-flight 726 pool 2070------------------------------envoy 1.86c sidecar 1.54cwrk hottest 0.95c mean 0.93c------------------------------mean request 13.05 ms before 6.88 envoy 1.72 sidecar 1.23 worker 3.23 resume 1.19 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5841 /s (99.4% ok)p50 27.9 ms p95 154.8 msin-flight 3890 pool 3890------------------------------envoy 1.90c sidecar 1.47cwrk hottest 0.96c mean 0.95c------------------------------mean request 46.79 ms before 24.92 envoy 4.65 sidecar 3.89 worker 13.32 resume 3.86 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5178 /s (99.8% ok)p50 1538.7 ms p95 2542.7 msin-flight 19k pool 19k------------------------------envoy 1.92c sidecar 1.19cwrk hottest 0.96c mean 0.96c------------------------------mean request 1528.23 ms before 204.06 envoy 1242.35 sidecar 2.95 worker 78.87 resume 2.84 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 966 /sachieved 965 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 32.0 pool 32.0------------------------------envoy 0.46c sidecar 0.46cwrk hottest 0.36c mean 0.21c------------------------------mean request 3.13 ms before 1.04 envoy 0.68 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 517 pool 517------------------------------envoy 0.46c sidecar 0.50cwrk hottest 0.37c mean 0.23c------------------------------mean request 14.72 ms before 9.74 envoy 1.30 sidecar 2.75 worker 0.93 resume 2.71 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1371 /sachieved 1371 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 56.0 pool 517------------------------------envoy 0.64c sidecar 0.56cwrk hottest 0.63c mean 0.33c------------------------------mean request 3.09 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 63.0 pool 517------------------------------envoy 0.88c sidecar 0.87cwrk hottest 0.70c mean 0.44c------------------------------mean request 3.17 ms before 1.19 envoy 0.63 sidecar 0.80 worker 0.55 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 73.0 pool 517------------------------------envoy 0.89c sidecar 0.88cwrk hottest 0.58c mean 0.44c------------------------------mean request 3.02 ms before 1.08 envoy 0.68 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2621 /sachieved 2621 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 228 pool 517------------------------------envoy 1.16c sidecar 0.97cwrk hottest 0.75c mean 0.58c------------------------------mean request 3.69 ms before 1.43 envoy 0.73 sidecar 0.82 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 62.0 pool 517------------------------------envoy 1.25c sidecar 1.15cwrk hottest 0.84c mean 0.62c------------------------------mean request 3.43 ms before 1.29 envoy 0.60 sidecar 0.81 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3015 /sachieved 3014 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 386 pool 517------------------------------envoy 1.26c sidecar 1.13cwrk hottest 0.73c mean 0.59c------------------------------mean request 4.12 ms before 1.66 envoy 0.66 sidecar 0.96 worker 0.84 resume 0.92 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 138 pool 386------------------------------envoy 1.60c sidecar 1.21cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.65 ms before 1.38 envoy 0.54 sidecar 0.81 worker 0.92 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 4.9 msin-flight 33.0 pool 386------------------------------envoy 1.57c sidecar 1.39cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.48 ms before 1.33 envoy 0.53 sidecar 0.80 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 33.0 pool 386------------------------------envoy 1.58c sidecar 1.40cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.52 ms before 1.34 envoy 0.52 sidecar 0.80 worker 0.87 resume 0.76 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4820 /sachieved 4820 /s (100.0% ok)p50 5.3 ms p95 303.2 msin-flight 1724 pool 1724------------------------------envoy 1.81c sidecar 1.45cwrk hottest 0.95c mean 0.90c------------------------------mean request 44.83 ms before 18.56 envoy 13.77 sidecar 1.08 worker 11.42 resume 1.04 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.4 ms p95 251.1 msin-flight 2068 pool 2070------------------------------envoy 1.82c sidecar 1.46cwrk hottest 0.94c mean 0.91c------------------------------mean request 28.05 ms before 5.36 envoy 12.78 sidecar 1.85 worker 8.06 resume 1.09 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5074 /sachieved 5066 /s (100.0% ok)p50 6.7 ms p95 46.7 msin-flight 726 pool 2070------------------------------envoy 1.86c sidecar 1.54cwrk hottest 0.95c mean 0.93c------------------------------mean request 13.05 ms before 6.88 envoy 1.72 sidecar 1.23 worker 3.23 resume 1.19 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5841 /s (99.4% ok)p50 27.9 ms p95 154.8 msin-flight 3890 pool 3890------------------------------envoy 1.90c sidecar 1.47cwrk hottest 0.96c mean 0.95c------------------------------mean request 46.79 ms before 24.92 envoy 4.65 sidecar 3.89 worker 13.32 resume 3.86 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5178 /s (99.8% ok)p50 1538.7 ms p95 2542.7 msin-flight 19k pool 19k------------------------------envoy 1.92c sidecar 1.19cwrk hottest 0.96c mean 0.96c------------------------------mean request 1528.23 ms before 204.06 envoy 1242.35 sidecar 2.95 worker 78.87 resume 2.84 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 966 /sachieved 965 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 32.0 pool 32.0------------------------------envoy 0.46c sidecar 0.46cwrk hottest 0.36c mean 0.21c------------------------------mean request 3.13 ms before 1.04 envoy 0.68 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 517 pool 517------------------------------envoy 0.46c sidecar 0.50cwrk hottest 0.37c mean 0.23c------------------------------mean request 14.72 ms before 9.74 envoy 1.30 sidecar 2.75 worker 0.93 resume 2.71 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1371 /sachieved 1371 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 56.0 pool 517------------------------------envoy 0.64c sidecar 0.56cwrk hottest 0.63c mean 0.33c------------------------------mean request 3.09 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 63.0 pool 517------------------------------envoy 0.88c sidecar 0.87cwrk hottest 0.70c mean 0.44c------------------------------mean request 3.17 ms before 1.19 envoy 0.63 sidecar 0.80 worker 0.55 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 73.0 pool 517------------------------------envoy 0.89c sidecar 0.88cwrk hottest 0.58c mean 0.44c------------------------------mean request 3.02 ms before 1.08 envoy 0.68 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2621 /sachieved 2621 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 228 pool 517------------------------------envoy 1.16c sidecar 0.97cwrk hottest 0.75c mean 0.58c------------------------------mean request 3.69 ms before 1.43 envoy 0.73 sidecar 0.82 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 62.0 pool 517------------------------------envoy 1.25c sidecar 1.15cwrk hottest 0.84c mean 0.62c------------------------------mean request 3.43 ms before 1.29 envoy 0.60 sidecar 0.81 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3015 /sachieved 3014 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 386 pool 517------------------------------envoy 1.26c sidecar 1.13cwrk hottest 0.73c mean 0.59c------------------------------mean request 4.12 ms before 1.66 envoy 0.66 sidecar 0.96 worker 0.84 resume 0.92 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 138 pool 386------------------------------envoy 1.60c sidecar 1.21cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.65 ms before 1.38 envoy 0.54 sidecar 0.81 worker 0.92 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 4.9 msin-flight 33.0 pool 386------------------------------envoy 1.57c sidecar 1.39cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.48 ms before 1.33 envoy 0.53 sidecar 0.80 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 33.0 pool 386------------------------------envoy 1.58c sidecar 1.40cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.52 ms before 1.34 envoy 0.52 sidecar 0.80 worker 0.87 resume 0.76 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4820 /sachieved 4820 /s (100.0% ok)p50 5.3 ms p95 303.2 msin-flight 1724 pool 1724------------------------------envoy 1.81c sidecar 1.45cwrk hottest 0.95c mean 0.90c------------------------------mean request 44.83 ms before 18.56 envoy 13.77 sidecar 1.08 worker 11.42 resume 1.04 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.4 ms p95 251.1 msin-flight 2068 pool 2070------------------------------envoy 1.82c sidecar 1.46cwrk hottest 0.94c mean 0.91c------------------------------mean request 28.05 ms before 5.36 envoy 12.78 sidecar 1.85 worker 8.06 resume 1.09 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5074 /sachieved 5066 /s (100.0% ok)p50 6.7 ms p95 46.7 msin-flight 726 pool 2070------------------------------envoy 1.86c sidecar 1.54cwrk hottest 0.95c mean 0.93c------------------------------mean request 13.05 ms before 6.88 envoy 1.72 sidecar 1.23 worker 3.23 resume 1.19 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5841 /s (99.4% ok)p50 27.9 ms p95 154.8 msin-flight 3890 pool 3890------------------------------envoy 1.90c sidecar 1.47cwrk hottest 0.96c mean 0.95c------------------------------mean request 46.79 ms before 24.92 envoy 4.65 sidecar 3.89 worker 13.32 resume 3.86 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5178 /s (99.8% ok)p50 1538.7 ms p95 2542.7 msin-flight 19k pool 19k------------------------------envoy 1.92c sidecar 1.19cwrk hottest 0.96c mean 0.96c------------------------------mean request 1528.23 ms before 204.06 envoy 1242.35 sidecar 2.95 worker 78.87 resume 2.84 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 966 /sachieved 965 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 32.0 pool 32.0------------------------------envoy 0.46c sidecar 0.46cwrk hottest 0.36c mean 0.21c------------------------------mean request 3.13 ms before 1.04 envoy 0.68 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 517 pool 517------------------------------envoy 0.46c sidecar 0.50cwrk hottest 0.37c mean 0.23c------------------------------mean request 14.72 ms before 9.74 envoy 1.30 sidecar 2.75 worker 0.93 resume 2.71 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1371 /sachieved 1371 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 56.0 pool 517------------------------------envoy 0.64c sidecar 0.56cwrk hottest 0.63c mean 0.33c------------------------------mean request 3.09 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 63.0 pool 517------------------------------envoy 0.88c sidecar 0.87cwrk hottest 0.70c mean 0.44c------------------------------mean request 3.17 ms before 1.19 envoy 0.63 sidecar 0.80 worker 0.55 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 73.0 pool 517------------------------------envoy 0.89c sidecar 0.88cwrk hottest 0.58c mean 0.44c------------------------------mean request 3.02 ms before 1.08 envoy 0.68 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2621 /sachieved 2621 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 228 pool 517------------------------------envoy 1.16c sidecar 0.97cwrk hottest 0.75c mean 0.58c------------------------------mean request 3.69 ms before 1.43 envoy 0.73 sidecar 0.82 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 62.0 pool 517------------------------------envoy 1.25c sidecar 1.15cwrk hottest 0.84c mean 0.62c------------------------------mean request 3.43 ms before 1.29 envoy 0.60 sidecar 0.81 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3015 /sachieved 3014 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 386 pool 517------------------------------envoy 1.26c sidecar 1.13cwrk hottest 0.73c mean 0.59c------------------------------mean request 4.12 ms before 1.66 envoy 0.66 sidecar 0.96 worker 0.84 resume 0.92 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 138 pool 386------------------------------envoy 1.60c sidecar 1.21cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.65 ms before 1.38 envoy 0.54 sidecar 0.81 worker 0.92 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 4.9 msin-flight 33.0 pool 386------------------------------envoy 1.57c sidecar 1.39cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.48 ms before 1.33 envoy 0.53 sidecar 0.80 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 33.0 pool 386------------------------------envoy 1.58c sidecar 1.40cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.52 ms before 1.34 envoy 0.52 sidecar 0.80 worker 0.87 resume 0.76 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4820 /sachieved 4820 /s (100.0% ok)p50 5.3 ms p95 303.2 msin-flight 1724 pool 1724------------------------------envoy 1.81c sidecar 1.45cwrk hottest 0.95c mean 0.90c------------------------------mean request 44.83 ms before 18.56 envoy 13.77 sidecar 1.08 worker 11.42 resume 1.04 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.4 ms p95 251.1 msin-flight 2068 pool 2070------------------------------envoy 1.82c sidecar 1.46cwrk hottest 0.94c mean 0.91c------------------------------mean request 28.05 ms before 5.36 envoy 12.78 sidecar 1.85 worker 8.06 resume 1.09 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5074 /sachieved 5066 /s (100.0% ok)p50 6.7 ms p95 46.7 msin-flight 726 pool 2070------------------------------envoy 1.86c sidecar 1.54cwrk hottest 0.95c mean 0.93c------------------------------mean request 13.05 ms before 6.88 envoy 1.72 sidecar 1.23 worker 3.23 resume 1.19 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5841 /s (99.4% ok)p50 27.9 ms p95 154.8 msin-flight 3890 pool 3890------------------------------envoy 1.90c sidecar 1.47cwrk hottest 0.96c mean 0.95c------------------------------mean request 46.79 ms before 24.92 envoy 4.65 sidecar 3.89 worker 13.32 resume 3.86 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5178 /s (99.8% ok)p50 1538.7 ms p95 2542.7 msin-flight 19k pool 19k------------------------------envoy 1.92c sidecar 1.19cwrk hottest 0.96c mean 0.96c------------------------------mean request 1528.23 ms before 204.06 envoy 1242.35 sidecar 2.95 worker 78.87 resume 2.84 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 966 /sachieved 965 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 32.0 pool 32.0------------------------------envoy 0.46c sidecar 0.46cwrk hottest 0.36c mean 0.21c------------------------------mean request 3.13 ms before 1.04 envoy 0.68 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 517 pool 517------------------------------envoy 0.46c sidecar 0.50cwrk hottest 0.37c mean 0.23c------------------------------mean request 14.72 ms before 9.74 envoy 1.30 sidecar 2.75 worker 0.93 resume 2.71 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1371 /sachieved 1371 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 56.0 pool 517------------------------------envoy 0.64c sidecar 0.56cwrk hottest 0.63c mean 0.33c------------------------------mean request 3.09 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 63.0 pool 517------------------------------envoy 0.88c sidecar 0.87cwrk hottest 0.70c mean 0.44c------------------------------mean request 3.17 ms before 1.19 envoy 0.63 sidecar 0.80 worker 0.55 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 73.0 pool 517------------------------------envoy 0.89c sidecar 0.88cwrk hottest 0.58c mean 0.44c------------------------------mean request 3.02 ms before 1.08 envoy 0.68 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2621 /sachieved 2621 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 228 pool 517------------------------------envoy 1.16c sidecar 0.97cwrk hottest 0.75c mean 0.58c------------------------------mean request 3.69 ms before 1.43 envoy 0.73 sidecar 0.82 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 62.0 pool 517------------------------------envoy 1.25c sidecar 1.15cwrk hottest 0.84c mean 0.62c------------------------------mean request 3.43 ms before 1.29 envoy 0.60 sidecar 0.81 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3015 /sachieved 3014 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 386 pool 517------------------------------envoy 1.26c sidecar 1.13cwrk hottest 0.73c mean 0.59c------------------------------mean request 4.12 ms before 1.66 envoy 0.66 sidecar 0.96 worker 0.84 resume 0.92 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 138 pool 386------------------------------envoy 1.60c sidecar 1.21cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.65 ms before 1.38 envoy 0.54 sidecar 0.81 worker 0.92 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 4.9 msin-flight 33.0 pool 386------------------------------envoy 1.57c sidecar 1.39cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.48 ms before 1.33 envoy 0.53 sidecar 0.80 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 33.0 pool 386------------------------------envoy 1.58c sidecar 1.40cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.52 ms before 1.34 envoy 0.52 sidecar 0.80 worker 0.87 resume 0.76 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4820 /sachieved 4820 /s (100.0% ok)p50 5.3 ms p95 303.2 msin-flight 1724 pool 1724------------------------------envoy 1.81c sidecar 1.45cwrk hottest 0.95c mean 0.90c------------------------------mean request 44.83 ms before 18.56 envoy 13.77 sidecar 1.08 worker 11.42 resume 1.04 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.4 ms p95 251.1 msin-flight 2068 pool 2070------------------------------envoy 1.82c sidecar 1.46cwrk hottest 0.94c mean 0.91c------------------------------mean request 28.05 ms before 5.36 envoy 12.78 sidecar 1.85 worker 8.06 resume 1.09 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5074 /sachieved 5066 /s (100.0% ok)p50 6.7 ms p95 46.7 msin-flight 726 pool 2070------------------------------envoy 1.86c sidecar 1.54cwrk hottest 0.95c mean 0.93c------------------------------mean request 13.05 ms before 6.88 envoy 1.72 sidecar 1.23 worker 3.23 resume 1.19 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5841 /s (99.4% ok)p50 27.9 ms p95 154.8 msin-flight 3890 pool 3890------------------------------envoy 1.90c sidecar 1.47cwrk hottest 0.96c mean 0.95c------------------------------mean request 46.79 ms before 24.92 envoy 4.65 sidecar 3.89 worker 13.32 resume 3.86 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5178 /s (99.8% ok)p50 1538.7 ms p95 2542.7 msin-flight 19k pool 19k------------------------------envoy 1.92c sidecar 1.19cwrk hottest 0.96c mean 0.96c------------------------------mean request 1528.23 ms before 204.06 envoy 1242.35 sidecar 2.95 worker 78.87 resume 2.84 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 966 /sachieved 965 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 32.0 pool 32.0------------------------------envoy 0.46c sidecar 0.46cwrk hottest 0.36c mean 0.21c------------------------------mean request 3.13 ms before 1.04 envoy 0.68 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 517 pool 517------------------------------envoy 0.46c sidecar 0.50cwrk hottest 0.37c mean 0.23c------------------------------mean request 14.72 ms before 9.74 envoy 1.30 sidecar 2.75 worker 0.93 resume 2.71 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1371 /sachieved 1371 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 56.0 pool 517------------------------------envoy 0.64c sidecar 0.56cwrk hottest 0.63c mean 0.33c------------------------------mean request 3.09 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 63.0 pool 517------------------------------envoy 0.88c sidecar 0.87cwrk hottest 0.70c mean 0.44c------------------------------mean request 3.17 ms before 1.19 envoy 0.63 sidecar 0.80 worker 0.55 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 73.0 pool 517------------------------------envoy 0.89c sidecar 0.88cwrk hottest 0.58c mean 0.44c------------------------------mean request 3.02 ms before 1.08 envoy 0.68 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2621 /sachieved 2621 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 228 pool 517------------------------------envoy 1.16c sidecar 0.97cwrk hottest 0.75c mean 0.58c------------------------------mean request 3.69 ms before 1.43 envoy 0.73 sidecar 0.82 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 62.0 pool 517------------------------------envoy 1.25c sidecar 1.15cwrk hottest 0.84c mean 0.62c------------------------------mean request 3.43 ms before 1.29 envoy 0.60 sidecar 0.81 worker 0.72 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3015 /sachieved 3014 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 386 pool 517------------------------------envoy 1.26c sidecar 1.13cwrk hottest 0.73c mean 0.59c------------------------------mean request 4.12 ms before 1.66 envoy 0.66 sidecar 0.96 worker 0.84 resume 0.92 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 138 pool 386------------------------------envoy 1.60c sidecar 1.21cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.65 ms before 1.38 envoy 0.54 sidecar 0.81 worker 0.92 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 4.9 msin-flight 33.0 pool 386------------------------------envoy 1.57c sidecar 1.39cwrk hottest 0.84c mean 0.78c------------------------------mean request 3.48 ms before 1.33 envoy 0.53 sidecar 0.80 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 33.0 pool 386------------------------------envoy 1.58c sidecar 1.40cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.52 ms before 1.34 envoy 0.52 sidecar 0.80 worker 0.87 resume 0.76 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4820 /sachieved 4820 /s (100.0% ok)p50 5.3 ms p95 303.2 msin-flight 1724 pool 1724------------------------------envoy 1.81c sidecar 1.45cwrk hottest 0.95c mean 0.90c------------------------------mean request 44.83 ms before 18.56 envoy 13.77 sidecar 1.08 worker 11.42 resume 1.04 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.4 ms p95 251.1 msin-flight 2068 pool 2070------------------------------envoy 1.82c sidecar 1.46cwrk hottest 0.94c mean 0.91c------------------------------mean request 28.05 ms before 5.36 envoy 12.78 sidecar 1.85 worker 8.06 resume 1.09 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5074 /sachieved 5066 /s (100.0% ok)p50 6.7 ms p95 46.7 msin-flight 726 pool 2070------------------------------envoy 1.86c sidecar 1.54cwrk hottest 0.95c mean 0.93c------------------------------mean request 13.05 ms before 6.88 envoy 1.72 sidecar 1.23 worker 3.23 resume 1.19 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5841 /s (99.4% ok)p50 27.9 ms p95 154.8 msin-flight 3890 pool 3890------------------------------envoy 1.90c sidecar 1.47cwrk hottest 0.96c mean 0.95c------------------------------mean request 46.79 ms before 24.92 envoy 4.65 sidecar 3.89 worker 13.32 resume 3.86 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5178 /s (99.8% ok)p50 1538.7 ms p95 2542.7 msin-flight 19k pool 19k------------------------------envoy 1.92c sidecar 1.19cwrk hottest 0.96c mean 0.96c------------------------------mean request 1528.23 ms before 204.06 envoy 1242.35 sidecar 2.95 worker 78.87 resume 2.84 (control plane) +
+ + + +atenet-router capacity — 4c +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 +R9 + +0 + +10k + +20k + +30k +requests / s + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + + +0 + +1.0 + +2.0 + +3.0 + +4.0 +cores + + + + + + +envoy CPU limit, 4 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + + +0 + +500.0M + +1.0G + +1.5G +bytes + +100 + +200 + +300 + +400 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung 0 1000 qps setpoint [warmup]------------------------------offered 975 /sachieved 975 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 30.0 pool 30.0------------------------------envoy 0.48c sidecar 0.11cwrk hottest 0.39c mean 0.12c------------------------------mean request 3.22 ms before 1.09 envoy 0.65 sidecar 0.79 worker 0.70 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 30.0------------------------------envoy 0.50c sidecar 0.50cwrk hottest 0.25c mean 0.13c------------------------------mean request 2.93 ms before 1.03 envoy 0.74 sidecar 0.78 worker 0.38 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1106 /sachieved 1106 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 45.0 pool 45.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.24c mean 0.14c------------------------------mean request 2.93 ms before 1.02 envoy 0.83 sidecar 0.81 worker 0.27 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.6 msin-flight 76.0 pool 76.0------------------------------envoy 0.99c sidecar 0.77cwrk hottest 0.42c mean 0.25c------------------------------mean request 2.94 ms before 1.07 envoy 0.79 sidecar 0.79 worker 0.29 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 23.0 pool 76.0------------------------------envoy 0.87c sidecar 0.82cwrk hottest 0.66c mean 0.22c------------------------------mean request 3.01 ms before 1.12 envoy 0.60 sidecar 0.79 worker 0.51 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2146 /sachieved 2145 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 76.0------------------------------envoy 0.92c sidecar 0.00cwrk hottest 0.65c mean 0.23c------------------------------mean request 3.06 ms before 1.16 envoy 0.63 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 49.0 pool 76.0------------------------------envoy 1.33c sidecar 0.98cwrk hottest 0.81c mean 0.33c------------------------------mean request 3.02 ms before 1.10 envoy 0.66 sidecar 0.79 worker 0.48 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 20.0 pool 76.0------------------------------envoy 1.31c sidecar 1.16cwrk hottest 0.69c mean 0.33c------------------------------mean request 2.93 ms before 1.06 envoy 0.62 sidecar 0.79 worker 0.47 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3095 /sachieved 3094 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 20.0 pool 76.0------------------------------envoy 1.34c sidecar 1.15cwrk hottest 0.62c mean 0.34c------------------------------mean request 2.90 ms before 1.05 envoy 0.65 sidecar 0.78 worker 0.41 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 29.0 pool 76.0------------------------------envoy 1.70c sidecar 0.00cwrk hottest 0.77c mean 0.42c------------------------------mean request 3.09 ms before 1.13 envoy 0.59 sidecar 0.79 worker 0.58 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 32.0 pool 76.0------------------------------envoy 1.70c sidecar 1.42cwrk hottest 0.79c mean 0.42c------------------------------mean request 3.07 ms before 1.10 envoy 0.60 sidecar 0.79 worker 0.58 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4150 /sachieved 4149 /s (100.0% ok)p50 3.0 ms p95 4.2 msin-flight 32.0 pool 49.0------------------------------envoy 1.77c sidecar 1.46cwrk hottest 0.71c mean 0.44c------------------------------mean request 3.07 ms before 1.11 envoy 0.58 sidecar 0.79 worker 0.60 resume 0.75 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 38.0 pool 49.0------------------------------envoy 2.16c sidecar 1.71cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.15 ms before 1.15 envoy 0.60 sidecar 0.79 worker 0.61 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 44.0 pool 49.0------------------------------envoy 2.12c sidecar 0.00cwrk hottest 0.83c mean 0.53c------------------------------mean request 3.14 ms before 1.11 envoy 0.60 sidecar 0.79 worker 0.63 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5360 /sachieved 5359 /s (100.0% ok)p50 3.0 ms p95 5.0 msin-flight 363 pool 363------------------------------envoy 2.28c sidecar 1.72cwrk hottest 0.87c mean 0.57c------------------------------mean request 3.53 ms before 1.23 envoy 0.63 sidecar 0.93 worker 0.73 resume 0.90 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5997 /s (100.0% ok)p50 4.9 ms p95 339.4 msin-flight 2584 pool 2584------------------------------envoy 2.46c sidecar 1.77cwrk hottest 0.94c mean 0.61c------------------------------mean request 49.98 ms before 12.76 envoy 14.65 sidecar 1.14 worker 21.43 resume 1.09 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6004 /s (99.7% ok)p50 3.7 ms p95 43.8 msin-flight 1422 pool 2584------------------------------envoy 2.51c sidecar 1.80cwrk hottest 0.88c mean 0.63c------------------------------mean request 12.40 ms before 2.96 envoy 2.45 sidecar 2.43 worker 4.55 resume 2.40 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6684 /sachieved 6682 /s (100.0% ok)p50 4.2 ms p95 10.0 msin-flight 145 pool 2584------------------------------envoy 2.75c sidecar 1.93cwrk hottest 0.89c mean 0.68c------------------------------mean request 4.97 ms before 2.03 envoy 0.82 sidecar 0.85 worker 1.27 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7002 /s (100.0% ok)p50 4.5 ms p95 10.5 msin-flight 116 pool 2584------------------------------envoy 2.83c sidecar 2.08cwrk hottest 0.90c mean 0.71c------------------------------mean request 5.31 ms before 2.08 envoy 0.92 sidecar 0.84 worker 1.46 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.7 ms p95 17.0 msin-flight 973 pool 2584------------------------------envoy 2.93c sidecar 2.09cwrk hottest 0.91c mean 0.73c------------------------------mean request 8.28 ms before 2.79 envoy 1.52 sidecar 1.44 worker 2.53 resume 1.40 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7626 /sachieved 7626 /s (100.0% ok)p50 4.9 ms p95 17.5 msin-flight 233 pool 2584------------------------------envoy 3.12c sidecar 2.25cwrk hottest 0.90c mean 0.78c------------------------------mean request 6.81 ms before 3.02 envoy 1.11 sidecar 0.86 worker 1.81 resume 0.83 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 5.5 ms p95 18.0 msin-flight 303 pool 2584------------------------------envoy 3.23c sidecar 2.27cwrk hottest 0.89c mean 0.81c------------------------------mean request 7.21 ms before 2.58 envoy 1.38 sidecar 0.91 worker 2.35 resume 0.87 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 6.6 ms p95 21.7 msin-flight 336 pool 1422------------------------------envoy 3.21c sidecar 0.00cwrk hottest 0.90c mean 0.80c------------------------------mean request 8.86 ms before 3.83 envoy 1.54 sidecar 0.90 worker 2.59 resume 0.86 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8841 /sachieved 8822 /s (100.0% ok)p50 11.6 ms p95 50.5 msin-flight 588 pool 1422------------------------------envoy 3.41c sidecar 2.26cwrk hottest 0.89c mean 0.85c------------------------------mean request 17.89 ms before 7.94 envoy 3.20 sidecar 1.04 worker 5.71 resume 1.00 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9016 /s (100.0% ok)p50 12.3 ms p95 47.1 msin-flight 555 pool 973------------------------------envoy 3.45c sidecar 2.28cwrk hottest 0.89c mean 0.86c------------------------------mean request 16.87 ms before 7.16 envoy 3.17 sidecar 0.99 worker 5.54 resume 0.95 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9005 /s (100.0% ok)p50 14.2 ms p95 43.9 msin-flight 427 pool 973------------------------------envoy 3.47c sidecar 2.37cwrk hottest 0.88c mean 0.87c------------------------------mean request 17.52 ms before 6.05 envoy 3.78 sidecar 1.02 worker 6.67 resume 0.98 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9690 /sachieved 8560 /s (97.3% ok)p50 549.8 ms p95 2201.5 msin-flight 22k pool 21k------------------------------envoy 3.52c sidecar 2.10cwrk hottest 0.89c mean 0.88c------------------------------mean request 771.52 ms before 51.87 envoy 654.61 sidecar 2.17 worker 62.87 resume 2.10 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 975 /sachieved 975 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 30.0 pool 30.0------------------------------envoy 0.48c sidecar 0.11cwrk hottest 0.39c mean 0.12c------------------------------mean request 3.22 ms before 1.09 envoy 0.65 sidecar 0.79 worker 0.70 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 30.0------------------------------envoy 0.50c sidecar 0.50cwrk hottest 0.25c mean 0.13c------------------------------mean request 2.93 ms before 1.03 envoy 0.74 sidecar 0.78 worker 0.38 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1106 /sachieved 1106 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 45.0 pool 45.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.24c mean 0.14c------------------------------mean request 2.93 ms before 1.02 envoy 0.83 sidecar 0.81 worker 0.27 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.6 msin-flight 76.0 pool 76.0------------------------------envoy 0.99c sidecar 0.77cwrk hottest 0.42c mean 0.25c------------------------------mean request 2.94 ms before 1.07 envoy 0.79 sidecar 0.79 worker 0.29 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 23.0 pool 76.0------------------------------envoy 0.87c sidecar 0.82cwrk hottest 0.66c mean 0.22c------------------------------mean request 3.01 ms before 1.12 envoy 0.60 sidecar 0.79 worker 0.51 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2146 /sachieved 2145 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 76.0------------------------------envoy 0.92c sidecar 0.00cwrk hottest 0.65c mean 0.23c------------------------------mean request 3.06 ms before 1.16 envoy 0.63 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 49.0 pool 76.0------------------------------envoy 1.33c sidecar 0.98cwrk hottest 0.81c mean 0.33c------------------------------mean request 3.02 ms before 1.10 envoy 0.66 sidecar 0.79 worker 0.48 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 20.0 pool 76.0------------------------------envoy 1.31c sidecar 1.16cwrk hottest 0.69c mean 0.33c------------------------------mean request 2.93 ms before 1.06 envoy 0.62 sidecar 0.79 worker 0.47 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3095 /sachieved 3094 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 20.0 pool 76.0------------------------------envoy 1.34c sidecar 1.15cwrk hottest 0.62c mean 0.34c------------------------------mean request 2.90 ms before 1.05 envoy 0.65 sidecar 0.78 worker 0.41 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 29.0 pool 76.0------------------------------envoy 1.70c sidecar 0.00cwrk hottest 0.77c mean 0.42c------------------------------mean request 3.09 ms before 1.13 envoy 0.59 sidecar 0.79 worker 0.58 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 32.0 pool 76.0------------------------------envoy 1.70c sidecar 1.42cwrk hottest 0.79c mean 0.42c------------------------------mean request 3.07 ms before 1.10 envoy 0.60 sidecar 0.79 worker 0.58 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4150 /sachieved 4149 /s (100.0% ok)p50 3.0 ms p95 4.2 msin-flight 32.0 pool 49.0------------------------------envoy 1.77c sidecar 1.46cwrk hottest 0.71c mean 0.44c------------------------------mean request 3.07 ms before 1.11 envoy 0.58 sidecar 0.79 worker 0.60 resume 0.75 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 38.0 pool 49.0------------------------------envoy 2.16c sidecar 1.71cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.15 ms before 1.15 envoy 0.60 sidecar 0.79 worker 0.61 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 44.0 pool 49.0------------------------------envoy 2.12c sidecar 0.00cwrk hottest 0.83c mean 0.53c------------------------------mean request 3.14 ms before 1.11 envoy 0.60 sidecar 0.79 worker 0.63 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5360 /sachieved 5359 /s (100.0% ok)p50 3.0 ms p95 5.0 msin-flight 363 pool 363------------------------------envoy 2.28c sidecar 1.72cwrk hottest 0.87c mean 0.57c------------------------------mean request 3.53 ms before 1.23 envoy 0.63 sidecar 0.93 worker 0.73 resume 0.90 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5997 /s (100.0% ok)p50 4.9 ms p95 339.4 msin-flight 2584 pool 2584------------------------------envoy 2.46c sidecar 1.77cwrk hottest 0.94c mean 0.61c------------------------------mean request 49.98 ms before 12.76 envoy 14.65 sidecar 1.14 worker 21.43 resume 1.09 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6004 /s (99.7% ok)p50 3.7 ms p95 43.8 msin-flight 1422 pool 2584------------------------------envoy 2.51c sidecar 1.80cwrk hottest 0.88c mean 0.63c------------------------------mean request 12.40 ms before 2.96 envoy 2.45 sidecar 2.43 worker 4.55 resume 2.40 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6684 /sachieved 6682 /s (100.0% ok)p50 4.2 ms p95 10.0 msin-flight 145 pool 2584------------------------------envoy 2.75c sidecar 1.93cwrk hottest 0.89c mean 0.68c------------------------------mean request 4.97 ms before 2.03 envoy 0.82 sidecar 0.85 worker 1.27 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7002 /s (100.0% ok)p50 4.5 ms p95 10.5 msin-flight 116 pool 2584------------------------------envoy 2.83c sidecar 2.08cwrk hottest 0.90c mean 0.71c------------------------------mean request 5.31 ms before 2.08 envoy 0.92 sidecar 0.84 worker 1.46 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.7 ms p95 17.0 msin-flight 973 pool 2584------------------------------envoy 2.93c sidecar 2.09cwrk hottest 0.91c mean 0.73c------------------------------mean request 8.28 ms before 2.79 envoy 1.52 sidecar 1.44 worker 2.53 resume 1.40 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7626 /sachieved 7626 /s (100.0% ok)p50 4.9 ms p95 17.5 msin-flight 233 pool 2584------------------------------envoy 3.12c sidecar 2.25cwrk hottest 0.90c mean 0.78c------------------------------mean request 6.81 ms before 3.02 envoy 1.11 sidecar 0.86 worker 1.81 resume 0.83 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 5.5 ms p95 18.0 msin-flight 303 pool 2584------------------------------envoy 3.23c sidecar 2.27cwrk hottest 0.89c mean 0.81c------------------------------mean request 7.21 ms before 2.58 envoy 1.38 sidecar 0.91 worker 2.35 resume 0.87 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 6.6 ms p95 21.7 msin-flight 336 pool 1422------------------------------envoy 3.21c sidecar 0.00cwrk hottest 0.90c mean 0.80c------------------------------mean request 8.86 ms before 3.83 envoy 1.54 sidecar 0.90 worker 2.59 resume 0.86 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8841 /sachieved 8822 /s (100.0% ok)p50 11.6 ms p95 50.5 msin-flight 588 pool 1422------------------------------envoy 3.41c sidecar 2.26cwrk hottest 0.89c mean 0.85c------------------------------mean request 17.89 ms before 7.94 envoy 3.20 sidecar 1.04 worker 5.71 resume 1.00 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9016 /s (100.0% ok)p50 12.3 ms p95 47.1 msin-flight 555 pool 973------------------------------envoy 3.45c sidecar 2.28cwrk hottest 0.89c mean 0.86c------------------------------mean request 16.87 ms before 7.16 envoy 3.17 sidecar 0.99 worker 5.54 resume 0.95 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9005 /s (100.0% ok)p50 14.2 ms p95 43.9 msin-flight 427 pool 973------------------------------envoy 3.47c sidecar 2.37cwrk hottest 0.88c mean 0.87c------------------------------mean request 17.52 ms before 6.05 envoy 3.78 sidecar 1.02 worker 6.67 resume 0.98 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9690 /sachieved 8560 /s (97.3% ok)p50 549.8 ms p95 2201.5 msin-flight 22k pool 21k------------------------------envoy 3.52c sidecar 2.10cwrk hottest 0.89c mean 0.88c------------------------------mean request 771.52 ms before 51.87 envoy 654.61 sidecar 2.17 worker 62.87 resume 2.10 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 975 /sachieved 975 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 30.0 pool 30.0------------------------------envoy 0.48c sidecar 0.11cwrk hottest 0.39c mean 0.12c------------------------------mean request 3.22 ms before 1.09 envoy 0.65 sidecar 0.79 worker 0.70 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 30.0------------------------------envoy 0.50c sidecar 0.50cwrk hottest 0.25c mean 0.13c------------------------------mean request 2.93 ms before 1.03 envoy 0.74 sidecar 0.78 worker 0.38 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1106 /sachieved 1106 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 45.0 pool 45.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.24c mean 0.14c------------------------------mean request 2.93 ms before 1.02 envoy 0.83 sidecar 0.81 worker 0.27 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.6 msin-flight 76.0 pool 76.0------------------------------envoy 0.99c sidecar 0.77cwrk hottest 0.42c mean 0.25c------------------------------mean request 2.94 ms before 1.07 envoy 0.79 sidecar 0.79 worker 0.29 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 23.0 pool 76.0------------------------------envoy 0.87c sidecar 0.82cwrk hottest 0.66c mean 0.22c------------------------------mean request 3.01 ms before 1.12 envoy 0.60 sidecar 0.79 worker 0.51 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2146 /sachieved 2145 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 76.0------------------------------envoy 0.92c sidecar 0.00cwrk hottest 0.65c mean 0.23c------------------------------mean request 3.06 ms before 1.16 envoy 0.63 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 49.0 pool 76.0------------------------------envoy 1.33c sidecar 0.98cwrk hottest 0.81c mean 0.33c------------------------------mean request 3.02 ms before 1.10 envoy 0.66 sidecar 0.79 worker 0.48 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 20.0 pool 76.0------------------------------envoy 1.31c sidecar 1.16cwrk hottest 0.69c mean 0.33c------------------------------mean request 2.93 ms before 1.06 envoy 0.62 sidecar 0.79 worker 0.47 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3095 /sachieved 3094 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 20.0 pool 76.0------------------------------envoy 1.34c sidecar 1.15cwrk hottest 0.62c mean 0.34c------------------------------mean request 2.90 ms before 1.05 envoy 0.65 sidecar 0.78 worker 0.41 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 29.0 pool 76.0------------------------------envoy 1.70c sidecar 0.00cwrk hottest 0.77c mean 0.42c------------------------------mean request 3.09 ms before 1.13 envoy 0.59 sidecar 0.79 worker 0.58 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 32.0 pool 76.0------------------------------envoy 1.70c sidecar 1.42cwrk hottest 0.79c mean 0.42c------------------------------mean request 3.07 ms before 1.10 envoy 0.60 sidecar 0.79 worker 0.58 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4150 /sachieved 4149 /s (100.0% ok)p50 3.0 ms p95 4.2 msin-flight 32.0 pool 49.0------------------------------envoy 1.77c sidecar 1.46cwrk hottest 0.71c mean 0.44c------------------------------mean request 3.07 ms before 1.11 envoy 0.58 sidecar 0.79 worker 0.60 resume 0.75 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 38.0 pool 49.0------------------------------envoy 2.16c sidecar 1.71cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.15 ms before 1.15 envoy 0.60 sidecar 0.79 worker 0.61 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 44.0 pool 49.0------------------------------envoy 2.12c sidecar 0.00cwrk hottest 0.83c mean 0.53c------------------------------mean request 3.14 ms before 1.11 envoy 0.60 sidecar 0.79 worker 0.63 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5360 /sachieved 5359 /s (100.0% ok)p50 3.0 ms p95 5.0 msin-flight 363 pool 363------------------------------envoy 2.28c sidecar 1.72cwrk hottest 0.87c mean 0.57c------------------------------mean request 3.53 ms before 1.23 envoy 0.63 sidecar 0.93 worker 0.73 resume 0.90 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5997 /s (100.0% ok)p50 4.9 ms p95 339.4 msin-flight 2584 pool 2584------------------------------envoy 2.46c sidecar 1.77cwrk hottest 0.94c mean 0.61c------------------------------mean request 49.98 ms before 12.76 envoy 14.65 sidecar 1.14 worker 21.43 resume 1.09 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6004 /s (99.7% ok)p50 3.7 ms p95 43.8 msin-flight 1422 pool 2584------------------------------envoy 2.51c sidecar 1.80cwrk hottest 0.88c mean 0.63c------------------------------mean request 12.40 ms before 2.96 envoy 2.45 sidecar 2.43 worker 4.55 resume 2.40 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6684 /sachieved 6682 /s (100.0% ok)p50 4.2 ms p95 10.0 msin-flight 145 pool 2584------------------------------envoy 2.75c sidecar 1.93cwrk hottest 0.89c mean 0.68c------------------------------mean request 4.97 ms before 2.03 envoy 0.82 sidecar 0.85 worker 1.27 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7002 /s (100.0% ok)p50 4.5 ms p95 10.5 msin-flight 116 pool 2584------------------------------envoy 2.83c sidecar 2.08cwrk hottest 0.90c mean 0.71c------------------------------mean request 5.31 ms before 2.08 envoy 0.92 sidecar 0.84 worker 1.46 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.7 ms p95 17.0 msin-flight 973 pool 2584------------------------------envoy 2.93c sidecar 2.09cwrk hottest 0.91c mean 0.73c------------------------------mean request 8.28 ms before 2.79 envoy 1.52 sidecar 1.44 worker 2.53 resume 1.40 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7626 /sachieved 7626 /s (100.0% ok)p50 4.9 ms p95 17.5 msin-flight 233 pool 2584------------------------------envoy 3.12c sidecar 2.25cwrk hottest 0.90c mean 0.78c------------------------------mean request 6.81 ms before 3.02 envoy 1.11 sidecar 0.86 worker 1.81 resume 0.83 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 5.5 ms p95 18.0 msin-flight 303 pool 2584------------------------------envoy 3.23c sidecar 2.27cwrk hottest 0.89c mean 0.81c------------------------------mean request 7.21 ms before 2.58 envoy 1.38 sidecar 0.91 worker 2.35 resume 0.87 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 6.6 ms p95 21.7 msin-flight 336 pool 1422------------------------------envoy 3.21c sidecar 0.00cwrk hottest 0.90c mean 0.80c------------------------------mean request 8.86 ms before 3.83 envoy 1.54 sidecar 0.90 worker 2.59 resume 0.86 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8841 /sachieved 8822 /s (100.0% ok)p50 11.6 ms p95 50.5 msin-flight 588 pool 1422------------------------------envoy 3.41c sidecar 2.26cwrk hottest 0.89c mean 0.85c------------------------------mean request 17.89 ms before 7.94 envoy 3.20 sidecar 1.04 worker 5.71 resume 1.00 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9016 /s (100.0% ok)p50 12.3 ms p95 47.1 msin-flight 555 pool 973------------------------------envoy 3.45c sidecar 2.28cwrk hottest 0.89c mean 0.86c------------------------------mean request 16.87 ms before 7.16 envoy 3.17 sidecar 0.99 worker 5.54 resume 0.95 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9005 /s (100.0% ok)p50 14.2 ms p95 43.9 msin-flight 427 pool 973------------------------------envoy 3.47c sidecar 2.37cwrk hottest 0.88c mean 0.87c------------------------------mean request 17.52 ms before 6.05 envoy 3.78 sidecar 1.02 worker 6.67 resume 0.98 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9690 /sachieved 8560 /s (97.3% ok)p50 549.8 ms p95 2201.5 msin-flight 22k pool 21k------------------------------envoy 3.52c sidecar 2.10cwrk hottest 0.89c mean 0.88c------------------------------mean request 771.52 ms before 51.87 envoy 654.61 sidecar 2.17 worker 62.87 resume 2.10 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 975 /sachieved 975 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 30.0 pool 30.0------------------------------envoy 0.48c sidecar 0.11cwrk hottest 0.39c mean 0.12c------------------------------mean request 3.22 ms before 1.09 envoy 0.65 sidecar 0.79 worker 0.70 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 30.0------------------------------envoy 0.50c sidecar 0.50cwrk hottest 0.25c mean 0.13c------------------------------mean request 2.93 ms before 1.03 envoy 0.74 sidecar 0.78 worker 0.38 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1106 /sachieved 1106 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 45.0 pool 45.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.24c mean 0.14c------------------------------mean request 2.93 ms before 1.02 envoy 0.83 sidecar 0.81 worker 0.27 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.6 msin-flight 76.0 pool 76.0------------------------------envoy 0.99c sidecar 0.77cwrk hottest 0.42c mean 0.25c------------------------------mean request 2.94 ms before 1.07 envoy 0.79 sidecar 0.79 worker 0.29 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 23.0 pool 76.0------------------------------envoy 0.87c sidecar 0.82cwrk hottest 0.66c mean 0.22c------------------------------mean request 3.01 ms before 1.12 envoy 0.60 sidecar 0.79 worker 0.51 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2146 /sachieved 2145 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 76.0------------------------------envoy 0.92c sidecar 0.00cwrk hottest 0.65c mean 0.23c------------------------------mean request 3.06 ms before 1.16 envoy 0.63 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 49.0 pool 76.0------------------------------envoy 1.33c sidecar 0.98cwrk hottest 0.81c mean 0.33c------------------------------mean request 3.02 ms before 1.10 envoy 0.66 sidecar 0.79 worker 0.48 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 20.0 pool 76.0------------------------------envoy 1.31c sidecar 1.16cwrk hottest 0.69c mean 0.33c------------------------------mean request 2.93 ms before 1.06 envoy 0.62 sidecar 0.79 worker 0.47 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3095 /sachieved 3094 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 20.0 pool 76.0------------------------------envoy 1.34c sidecar 1.15cwrk hottest 0.62c mean 0.34c------------------------------mean request 2.90 ms before 1.05 envoy 0.65 sidecar 0.78 worker 0.41 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 29.0 pool 76.0------------------------------envoy 1.70c sidecar 0.00cwrk hottest 0.77c mean 0.42c------------------------------mean request 3.09 ms before 1.13 envoy 0.59 sidecar 0.79 worker 0.58 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 32.0 pool 76.0------------------------------envoy 1.70c sidecar 1.42cwrk hottest 0.79c mean 0.42c------------------------------mean request 3.07 ms before 1.10 envoy 0.60 sidecar 0.79 worker 0.58 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4150 /sachieved 4149 /s (100.0% ok)p50 3.0 ms p95 4.2 msin-flight 32.0 pool 49.0------------------------------envoy 1.77c sidecar 1.46cwrk hottest 0.71c mean 0.44c------------------------------mean request 3.07 ms before 1.11 envoy 0.58 sidecar 0.79 worker 0.60 resume 0.75 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 38.0 pool 49.0------------------------------envoy 2.16c sidecar 1.71cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.15 ms before 1.15 envoy 0.60 sidecar 0.79 worker 0.61 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 44.0 pool 49.0------------------------------envoy 2.12c sidecar 0.00cwrk hottest 0.83c mean 0.53c------------------------------mean request 3.14 ms before 1.11 envoy 0.60 sidecar 0.79 worker 0.63 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5360 /sachieved 5359 /s (100.0% ok)p50 3.0 ms p95 5.0 msin-flight 363 pool 363------------------------------envoy 2.28c sidecar 1.72cwrk hottest 0.87c mean 0.57c------------------------------mean request 3.53 ms before 1.23 envoy 0.63 sidecar 0.93 worker 0.73 resume 0.90 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5997 /s (100.0% ok)p50 4.9 ms p95 339.4 msin-flight 2584 pool 2584------------------------------envoy 2.46c sidecar 1.77cwrk hottest 0.94c mean 0.61c------------------------------mean request 49.98 ms before 12.76 envoy 14.65 sidecar 1.14 worker 21.43 resume 1.09 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6004 /s (99.7% ok)p50 3.7 ms p95 43.8 msin-flight 1422 pool 2584------------------------------envoy 2.51c sidecar 1.80cwrk hottest 0.88c mean 0.63c------------------------------mean request 12.40 ms before 2.96 envoy 2.45 sidecar 2.43 worker 4.55 resume 2.40 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6684 /sachieved 6682 /s (100.0% ok)p50 4.2 ms p95 10.0 msin-flight 145 pool 2584------------------------------envoy 2.75c sidecar 1.93cwrk hottest 0.89c mean 0.68c------------------------------mean request 4.97 ms before 2.03 envoy 0.82 sidecar 0.85 worker 1.27 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7002 /s (100.0% ok)p50 4.5 ms p95 10.5 msin-flight 116 pool 2584------------------------------envoy 2.83c sidecar 2.08cwrk hottest 0.90c mean 0.71c------------------------------mean request 5.31 ms before 2.08 envoy 0.92 sidecar 0.84 worker 1.46 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.7 ms p95 17.0 msin-flight 973 pool 2584------------------------------envoy 2.93c sidecar 2.09cwrk hottest 0.91c mean 0.73c------------------------------mean request 8.28 ms before 2.79 envoy 1.52 sidecar 1.44 worker 2.53 resume 1.40 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7626 /sachieved 7626 /s (100.0% ok)p50 4.9 ms p95 17.5 msin-flight 233 pool 2584------------------------------envoy 3.12c sidecar 2.25cwrk hottest 0.90c mean 0.78c------------------------------mean request 6.81 ms before 3.02 envoy 1.11 sidecar 0.86 worker 1.81 resume 0.83 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 5.5 ms p95 18.0 msin-flight 303 pool 2584------------------------------envoy 3.23c sidecar 2.27cwrk hottest 0.89c mean 0.81c------------------------------mean request 7.21 ms before 2.58 envoy 1.38 sidecar 0.91 worker 2.35 resume 0.87 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 6.6 ms p95 21.7 msin-flight 336 pool 1422------------------------------envoy 3.21c sidecar 0.00cwrk hottest 0.90c mean 0.80c------------------------------mean request 8.86 ms before 3.83 envoy 1.54 sidecar 0.90 worker 2.59 resume 0.86 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8841 /sachieved 8822 /s (100.0% ok)p50 11.6 ms p95 50.5 msin-flight 588 pool 1422------------------------------envoy 3.41c sidecar 2.26cwrk hottest 0.89c mean 0.85c------------------------------mean request 17.89 ms before 7.94 envoy 3.20 sidecar 1.04 worker 5.71 resume 1.00 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9016 /s (100.0% ok)p50 12.3 ms p95 47.1 msin-flight 555 pool 973------------------------------envoy 3.45c sidecar 2.28cwrk hottest 0.89c mean 0.86c------------------------------mean request 16.87 ms before 7.16 envoy 3.17 sidecar 0.99 worker 5.54 resume 0.95 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9005 /s (100.0% ok)p50 14.2 ms p95 43.9 msin-flight 427 pool 973------------------------------envoy 3.47c sidecar 2.37cwrk hottest 0.88c mean 0.87c------------------------------mean request 17.52 ms before 6.05 envoy 3.78 sidecar 1.02 worker 6.67 resume 0.98 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9690 /sachieved 8560 /s (97.3% ok)p50 549.8 ms p95 2201.5 msin-flight 22k pool 21k------------------------------envoy 3.52c sidecar 2.10cwrk hottest 0.89c mean 0.88c------------------------------mean request 771.52 ms before 51.87 envoy 654.61 sidecar 2.17 worker 62.87 resume 2.10 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 975 /sachieved 975 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 30.0 pool 30.0------------------------------envoy 0.48c sidecar 0.11cwrk hottest 0.39c mean 0.12c------------------------------mean request 3.22 ms before 1.09 envoy 0.65 sidecar 0.79 worker 0.70 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 30.0------------------------------envoy 0.50c sidecar 0.50cwrk hottest 0.25c mean 0.13c------------------------------mean request 2.93 ms before 1.03 envoy 0.74 sidecar 0.78 worker 0.38 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1106 /sachieved 1106 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 45.0 pool 45.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.24c mean 0.14c------------------------------mean request 2.93 ms before 1.02 envoy 0.83 sidecar 0.81 worker 0.27 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.6 msin-flight 76.0 pool 76.0------------------------------envoy 0.99c sidecar 0.77cwrk hottest 0.42c mean 0.25c------------------------------mean request 2.94 ms before 1.07 envoy 0.79 sidecar 0.79 worker 0.29 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 23.0 pool 76.0------------------------------envoy 0.87c sidecar 0.82cwrk hottest 0.66c mean 0.22c------------------------------mean request 3.01 ms before 1.12 envoy 0.60 sidecar 0.79 worker 0.51 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2146 /sachieved 2145 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 76.0------------------------------envoy 0.92c sidecar 0.00cwrk hottest 0.65c mean 0.23c------------------------------mean request 3.06 ms before 1.16 envoy 0.63 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 49.0 pool 76.0------------------------------envoy 1.33c sidecar 0.98cwrk hottest 0.81c mean 0.33c------------------------------mean request 3.02 ms before 1.10 envoy 0.66 sidecar 0.79 worker 0.48 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 20.0 pool 76.0------------------------------envoy 1.31c sidecar 1.16cwrk hottest 0.69c mean 0.33c------------------------------mean request 2.93 ms before 1.06 envoy 0.62 sidecar 0.79 worker 0.47 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3095 /sachieved 3094 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 20.0 pool 76.0------------------------------envoy 1.34c sidecar 1.15cwrk hottest 0.62c mean 0.34c------------------------------mean request 2.90 ms before 1.05 envoy 0.65 sidecar 0.78 worker 0.41 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 29.0 pool 76.0------------------------------envoy 1.70c sidecar 0.00cwrk hottest 0.77c mean 0.42c------------------------------mean request 3.09 ms before 1.13 envoy 0.59 sidecar 0.79 worker 0.58 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 32.0 pool 76.0------------------------------envoy 1.70c sidecar 1.42cwrk hottest 0.79c mean 0.42c------------------------------mean request 3.07 ms before 1.10 envoy 0.60 sidecar 0.79 worker 0.58 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4150 /sachieved 4149 /s (100.0% ok)p50 3.0 ms p95 4.2 msin-flight 32.0 pool 49.0------------------------------envoy 1.77c sidecar 1.46cwrk hottest 0.71c mean 0.44c------------------------------mean request 3.07 ms before 1.11 envoy 0.58 sidecar 0.79 worker 0.60 resume 0.75 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 38.0 pool 49.0------------------------------envoy 2.16c sidecar 1.71cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.15 ms before 1.15 envoy 0.60 sidecar 0.79 worker 0.61 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 44.0 pool 49.0------------------------------envoy 2.12c sidecar 0.00cwrk hottest 0.83c mean 0.53c------------------------------mean request 3.14 ms before 1.11 envoy 0.60 sidecar 0.79 worker 0.63 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5360 /sachieved 5359 /s (100.0% ok)p50 3.0 ms p95 5.0 msin-flight 363 pool 363------------------------------envoy 2.28c sidecar 1.72cwrk hottest 0.87c mean 0.57c------------------------------mean request 3.53 ms before 1.23 envoy 0.63 sidecar 0.93 worker 0.73 resume 0.90 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5997 /s (100.0% ok)p50 4.9 ms p95 339.4 msin-flight 2584 pool 2584------------------------------envoy 2.46c sidecar 1.77cwrk hottest 0.94c mean 0.61c------------------------------mean request 49.98 ms before 12.76 envoy 14.65 sidecar 1.14 worker 21.43 resume 1.09 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6004 /s (99.7% ok)p50 3.7 ms p95 43.8 msin-flight 1422 pool 2584------------------------------envoy 2.51c sidecar 1.80cwrk hottest 0.88c mean 0.63c------------------------------mean request 12.40 ms before 2.96 envoy 2.45 sidecar 2.43 worker 4.55 resume 2.40 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6684 /sachieved 6682 /s (100.0% ok)p50 4.2 ms p95 10.0 msin-flight 145 pool 2584------------------------------envoy 2.75c sidecar 1.93cwrk hottest 0.89c mean 0.68c------------------------------mean request 4.97 ms before 2.03 envoy 0.82 sidecar 0.85 worker 1.27 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7002 /s (100.0% ok)p50 4.5 ms p95 10.5 msin-flight 116 pool 2584------------------------------envoy 2.83c sidecar 2.08cwrk hottest 0.90c mean 0.71c------------------------------mean request 5.31 ms before 2.08 envoy 0.92 sidecar 0.84 worker 1.46 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.7 ms p95 17.0 msin-flight 973 pool 2584------------------------------envoy 2.93c sidecar 2.09cwrk hottest 0.91c mean 0.73c------------------------------mean request 8.28 ms before 2.79 envoy 1.52 sidecar 1.44 worker 2.53 resume 1.40 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7626 /sachieved 7626 /s (100.0% ok)p50 4.9 ms p95 17.5 msin-flight 233 pool 2584------------------------------envoy 3.12c sidecar 2.25cwrk hottest 0.90c mean 0.78c------------------------------mean request 6.81 ms before 3.02 envoy 1.11 sidecar 0.86 worker 1.81 resume 0.83 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 5.5 ms p95 18.0 msin-flight 303 pool 2584------------------------------envoy 3.23c sidecar 2.27cwrk hottest 0.89c mean 0.81c------------------------------mean request 7.21 ms before 2.58 envoy 1.38 sidecar 0.91 worker 2.35 resume 0.87 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 6.6 ms p95 21.7 msin-flight 336 pool 1422------------------------------envoy 3.21c sidecar 0.00cwrk hottest 0.90c mean 0.80c------------------------------mean request 8.86 ms before 3.83 envoy 1.54 sidecar 0.90 worker 2.59 resume 0.86 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8841 /sachieved 8822 /s (100.0% ok)p50 11.6 ms p95 50.5 msin-flight 588 pool 1422------------------------------envoy 3.41c sidecar 2.26cwrk hottest 0.89c mean 0.85c------------------------------mean request 17.89 ms before 7.94 envoy 3.20 sidecar 1.04 worker 5.71 resume 1.00 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9016 /s (100.0% ok)p50 12.3 ms p95 47.1 msin-flight 555 pool 973------------------------------envoy 3.45c sidecar 2.28cwrk hottest 0.89c mean 0.86c------------------------------mean request 16.87 ms before 7.16 envoy 3.17 sidecar 0.99 worker 5.54 resume 0.95 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9005 /s (100.0% ok)p50 14.2 ms p95 43.9 msin-flight 427 pool 973------------------------------envoy 3.47c sidecar 2.37cwrk hottest 0.88c mean 0.87c------------------------------mean request 17.52 ms before 6.05 envoy 3.78 sidecar 1.02 worker 6.67 resume 0.98 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9690 /sachieved 8560 /s (97.3% ok)p50 549.8 ms p95 2201.5 msin-flight 22k pool 21k------------------------------envoy 3.52c sidecar 2.10cwrk hottest 0.89c mean 0.88c------------------------------mean request 771.52 ms before 51.87 envoy 654.61 sidecar 2.17 worker 62.87 resume 2.10 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 975 /sachieved 975 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 30.0 pool 30.0------------------------------envoy 0.48c sidecar 0.11cwrk hottest 0.39c mean 0.12c------------------------------mean request 3.22 ms before 1.09 envoy 0.65 sidecar 0.79 worker 0.70 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 30.0------------------------------envoy 0.50c sidecar 0.50cwrk hottest 0.25c mean 0.13c------------------------------mean request 2.93 ms before 1.03 envoy 0.74 sidecar 0.78 worker 0.38 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1106 /sachieved 1106 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 45.0 pool 45.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.24c mean 0.14c------------------------------mean request 2.93 ms before 1.02 envoy 0.83 sidecar 0.81 worker 0.27 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.6 msin-flight 76.0 pool 76.0------------------------------envoy 0.99c sidecar 0.77cwrk hottest 0.42c mean 0.25c------------------------------mean request 2.94 ms before 1.07 envoy 0.79 sidecar 0.79 worker 0.29 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 23.0 pool 76.0------------------------------envoy 0.87c sidecar 0.82cwrk hottest 0.66c mean 0.22c------------------------------mean request 3.01 ms before 1.12 envoy 0.60 sidecar 0.79 worker 0.51 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2146 /sachieved 2145 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 76.0------------------------------envoy 0.92c sidecar 0.00cwrk hottest 0.65c mean 0.23c------------------------------mean request 3.06 ms before 1.16 envoy 0.63 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 49.0 pool 76.0------------------------------envoy 1.33c sidecar 0.98cwrk hottest 0.81c mean 0.33c------------------------------mean request 3.02 ms before 1.10 envoy 0.66 sidecar 0.79 worker 0.48 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 20.0 pool 76.0------------------------------envoy 1.31c sidecar 1.16cwrk hottest 0.69c mean 0.33c------------------------------mean request 2.93 ms before 1.06 envoy 0.62 sidecar 0.79 worker 0.47 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3095 /sachieved 3094 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 20.0 pool 76.0------------------------------envoy 1.34c sidecar 1.15cwrk hottest 0.62c mean 0.34c------------------------------mean request 2.90 ms before 1.05 envoy 0.65 sidecar 0.78 worker 0.41 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 29.0 pool 76.0------------------------------envoy 1.70c sidecar 0.00cwrk hottest 0.77c mean 0.42c------------------------------mean request 3.09 ms before 1.13 envoy 0.59 sidecar 0.79 worker 0.58 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 32.0 pool 76.0------------------------------envoy 1.70c sidecar 1.42cwrk hottest 0.79c mean 0.42c------------------------------mean request 3.07 ms before 1.10 envoy 0.60 sidecar 0.79 worker 0.58 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4150 /sachieved 4149 /s (100.0% ok)p50 3.0 ms p95 4.2 msin-flight 32.0 pool 49.0------------------------------envoy 1.77c sidecar 1.46cwrk hottest 0.71c mean 0.44c------------------------------mean request 3.07 ms before 1.11 envoy 0.58 sidecar 0.79 worker 0.60 resume 0.75 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 38.0 pool 49.0------------------------------envoy 2.16c sidecar 1.71cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.15 ms before 1.15 envoy 0.60 sidecar 0.79 worker 0.61 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 44.0 pool 49.0------------------------------envoy 2.12c sidecar 0.00cwrk hottest 0.83c mean 0.53c------------------------------mean request 3.14 ms before 1.11 envoy 0.60 sidecar 0.79 worker 0.63 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5360 /sachieved 5359 /s (100.0% ok)p50 3.0 ms p95 5.0 msin-flight 363 pool 363------------------------------envoy 2.28c sidecar 1.72cwrk hottest 0.87c mean 0.57c------------------------------mean request 3.53 ms before 1.23 envoy 0.63 sidecar 0.93 worker 0.73 resume 0.90 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5997 /s (100.0% ok)p50 4.9 ms p95 339.4 msin-flight 2584 pool 2584------------------------------envoy 2.46c sidecar 1.77cwrk hottest 0.94c mean 0.61c------------------------------mean request 49.98 ms before 12.76 envoy 14.65 sidecar 1.14 worker 21.43 resume 1.09 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6004 /s (99.7% ok)p50 3.7 ms p95 43.8 msin-flight 1422 pool 2584------------------------------envoy 2.51c sidecar 1.80cwrk hottest 0.88c mean 0.63c------------------------------mean request 12.40 ms before 2.96 envoy 2.45 sidecar 2.43 worker 4.55 resume 2.40 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6684 /sachieved 6682 /s (100.0% ok)p50 4.2 ms p95 10.0 msin-flight 145 pool 2584------------------------------envoy 2.75c sidecar 1.93cwrk hottest 0.89c mean 0.68c------------------------------mean request 4.97 ms before 2.03 envoy 0.82 sidecar 0.85 worker 1.27 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7002 /s (100.0% ok)p50 4.5 ms p95 10.5 msin-flight 116 pool 2584------------------------------envoy 2.83c sidecar 2.08cwrk hottest 0.90c mean 0.71c------------------------------mean request 5.31 ms before 2.08 envoy 0.92 sidecar 0.84 worker 1.46 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.7 ms p95 17.0 msin-flight 973 pool 2584------------------------------envoy 2.93c sidecar 2.09cwrk hottest 0.91c mean 0.73c------------------------------mean request 8.28 ms before 2.79 envoy 1.52 sidecar 1.44 worker 2.53 resume 1.40 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7626 /sachieved 7626 /s (100.0% ok)p50 4.9 ms p95 17.5 msin-flight 233 pool 2584------------------------------envoy 3.12c sidecar 2.25cwrk hottest 0.90c mean 0.78c------------------------------mean request 6.81 ms before 3.02 envoy 1.11 sidecar 0.86 worker 1.81 resume 0.83 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 5.5 ms p95 18.0 msin-flight 303 pool 2584------------------------------envoy 3.23c sidecar 2.27cwrk hottest 0.89c mean 0.81c------------------------------mean request 7.21 ms before 2.58 envoy 1.38 sidecar 0.91 worker 2.35 resume 0.87 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 6.6 ms p95 21.7 msin-flight 336 pool 1422------------------------------envoy 3.21c sidecar 0.00cwrk hottest 0.90c mean 0.80c------------------------------mean request 8.86 ms before 3.83 envoy 1.54 sidecar 0.90 worker 2.59 resume 0.86 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8841 /sachieved 8822 /s (100.0% ok)p50 11.6 ms p95 50.5 msin-flight 588 pool 1422------------------------------envoy 3.41c sidecar 2.26cwrk hottest 0.89c mean 0.85c------------------------------mean request 17.89 ms before 7.94 envoy 3.20 sidecar 1.04 worker 5.71 resume 1.00 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9016 /s (100.0% ok)p50 12.3 ms p95 47.1 msin-flight 555 pool 973------------------------------envoy 3.45c sidecar 2.28cwrk hottest 0.89c mean 0.86c------------------------------mean request 16.87 ms before 7.16 envoy 3.17 sidecar 0.99 worker 5.54 resume 0.95 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9005 /s (100.0% ok)p50 14.2 ms p95 43.9 msin-flight 427 pool 973------------------------------envoy 3.47c sidecar 2.37cwrk hottest 0.88c mean 0.87c------------------------------mean request 17.52 ms before 6.05 envoy 3.78 sidecar 1.02 worker 6.67 resume 0.98 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9690 /sachieved 8560 /s (97.3% ok)p50 549.8 ms p95 2201.5 msin-flight 22k pool 21k------------------------------envoy 3.52c sidecar 2.10cwrk hottest 0.89c mean 0.88c------------------------------mean request 771.52 ms before 51.87 envoy 654.61 sidecar 2.17 worker 62.87 resume 2.10 (control plane) +
+ + + +atenet-router capacity — 2×4c +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 +R9 +R10 +R11 +R12 +R13 +R14 +R15 +R16 +R17 + +0 + +10k + +20k + +30k + +40k +requests / s + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + + + + + + + + + + +0 + +1.0 + +2.0 + +3.0 + +4.0 +cores + + + + + + +envoy CPU limit, 4 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + + + + + + + + + + +0 + +250.0M + +500.0M + +750.0M + +1.0G +bytes + +200 + +400 + +600 + +800 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 267 /sachieved 267 /s (100.0% ok)p50 2.9 ms p95 5.8 msin-flight 20.0 pool 23.0------------------------------envoy 0.08c sidecar 0.09cwrk hottest 0.13c mean 0.07c------------------------------mean request 3.39 ms before 0.95 envoy 0.73 sidecar 0.79 worker 0.92 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.5 msin-flight 5.0 pool 23.0------------------------------envoy 0.11c sidecar 0.26cwrk hottest 0.11c mean 0.06c------------------------------mean request 2.88 ms before 1.06 envoy 0.78 sidecar 0.79 worker 0.25 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.4 msin-flight 5.0 pool 23.0------------------------------envoy 0.26c sidecar 0.26cwrk hottest 0.12c mean 0.06c------------------------------mean request 2.86 ms before 1.03 envoy 0.80 sidecar 0.79 worker 0.23 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1412 /sachieved 1412 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 24.0------------------------------envoy 0.24c sidecar 0.34cwrk hottest 0.27c mean 0.09c------------------------------mean request 2.81 ms before 1.00 envoy 0.76 sidecar 0.80 worker 0.25 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.51c sidecar 0.49cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.92 envoy 0.85 sidecar 0.83 worker 0.18 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 10.0 pool 24.0------------------------------envoy 0.51c sidecar 0.51cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.95 envoy 0.86 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2270 /sachieved 2270 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.52c sidecar 0.52cwrk hottest 0.35c mean 0.15c------------------------------mean request 2.76 ms before 0.94 envoy 0.84 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 17.0 pool 24.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.35c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.16 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 22.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.36c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.17 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3170 /sachieved 3170 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 24.0 pool 25.0------------------------------envoy 0.81c sidecar 0.73cwrk hottest 0.35c mean 0.20c------------------------------mean request 2.75 ms before 0.87 envoy 0.84 sidecar 0.84 worker 0.21 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 4.2 msin-flight 532 pool 704------------------------------envoy 1.01c sidecar 0.85cwrk hottest 0.67c mean 0.25c------------------------------mean request 5.31 ms before 2.81 envoy 0.91 sidecar 1.14 worker 0.46 resume 1.09 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 49.0 pool 704------------------------------envoy 0.96c sidecar 0.91cwrk hottest 0.70c mean 0.23c------------------------------mean request 2.93 ms before 0.99 envoy 0.68 sidecar 0.83 worker 0.43 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4491 /sachieved 4491 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 51.0 pool 704------------------------------envoy 1.03c sidecar 0.96cwrk hottest 0.82c mean 0.26c------------------------------mean request 3.06 ms before 1.08 envoy 0.65 sidecar 0.82 worker 0.51 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 44.0 pool 704------------------------------envoy 1.16c sidecar 1.08cwrk hottest 0.84c mean 0.28c------------------------------mean request 3.11 ms before 1.07 envoy 0.61 sidecar 0.82 worker 0.61 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 53.0 pool 704------------------------------envoy 1.08c sidecar 1.09cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.39 ms before 1.16 envoy 0.62 sidecar 0.82 worker 0.79 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5362 /sachieved 5362 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 154 pool 704------------------------------envoy 1.24c sidecar 1.13cwrk hottest 0.83c mean 0.31c------------------------------mean request 3.23 ms before 1.08 envoy 0.64 sidecar 0.84 worker 0.67 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 5.1 msin-flight 1867 pool 1881------------------------------envoy 1.32c sidecar 1.27cwrk hottest 0.83c mean 0.37c------------------------------mean request 7.34 ms before 1.36 envoy 1.23 sidecar 2.79 worker 1.96 resume 2.75 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 4.9 msin-flight 100 pool 1881------------------------------envoy 1.46c sidecar 1.29cwrk hottest 0.84c mean 0.37c------------------------------mean request 3.25 ms before 1.11 envoy 0.67 sidecar 0.83 worker 0.64 resume 0.79 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6758 /sachieved 6758 /s (100.0% ok)p50 3.4 ms p95 8.2 msin-flight 103 pool 1881------------------------------envoy 1.60c sidecar 1.40cwrk hottest 0.92c mean 0.40c------------------------------mean request 4.09 ms before 1.55 envoy 0.79 sidecar 0.85 worker 0.89 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 106 pool 1881------------------------------envoy 1.75c sidecar 1.44cwrk hottest 0.85c mean 0.44c------------------------------mean request 3.61 ms before 1.36 envoy 0.70 sidecar 0.86 worker 0.69 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7116 /sachieved 7116 /s (100.0% ok)p50 3.4 ms p95 14.4 msin-flight 1611 pool 1881------------------------------envoy 1.69c sidecar 1.43cwrk hottest 0.90c mean 0.42c------------------------------mean request 8.76 ms before 2.25 envoy 1.03 sidecar 3.35 worker 2.13 resume 3.01 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 4.6 ms p95 16.4 msin-flight 150 pool 1881------------------------------envoy 1.74c sidecar 1.43cwrk hottest 0.97c mean 0.44c------------------------------mean request 6.39 ms before 1.79 envoy 1.44 sidecar 0.89 worker 2.27 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 4.3 ms p95 18.2 msin-flight 2220 pool 2252------------------------------envoy 1.79c sidecar 1.51cwrk hottest 0.95c mean 0.44c------------------------------mean request 11.66 ms before 5.04 envoy 1.00 sidecar 2.95 worker 2.67 resume 2.91 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8001 /s (100.0% ok)p50 3.0 ms p95 5.7 msin-flight 210 pool 2252------------------------------envoy 1.94c sidecar 1.65cwrk hottest 0.73c mean 0.51c------------------------------mean request 3.54 ms before 1.18 envoy 0.68 sidecar 0.98 worker 0.70 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8253 /sachieved 8253 /s (100.0% ok)p50 3.2 ms p95 6.4 msin-flight 219 pool 2252------------------------------envoy 2.10c sidecar 1.69cwrk hottest 0.80c mean 0.54c------------------------------mean request 3.71 ms before 1.28 envoy 0.69 sidecar 0.96 worker 0.78 resume 0.91 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.2 ms p95 8.4 msin-flight 727 pool 2252------------------------------envoy 2.11c sidecar 1.81cwrk hottest 0.80c mean 0.57c------------------------------mean request 4.39 ms before 1.55 envoy 0.74 sidecar 1.21 worker 0.89 resume 1.16 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.3 ms p95 7.8 msin-flight 532 pool 2252------------------------------envoy 2.22c sidecar 1.79cwrk hottest 0.85c mean 0.55c------------------------------mean request 4.05 ms before 1.60 envoy 0.53 sidecar 1.03 worker 0.89 resume 0.98 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9338 /sachieved 9332 /s (100.0% ok)p50 3.6 ms p95 18.8 msin-flight 225 pool 2252------------------------------envoy 2.34c sidecar 1.75cwrk hottest 0.92c mean 0.58c------------------------------mean request 6.36 ms before 3.08 envoy 1.04 sidecar 0.96 worker 1.28 resume 0.92 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 4.0 ms p95 19.9 msin-flight 1033 pool 2252------------------------------envoy 2.28c sidecar 1.90cwrk hottest 0.93c mean 0.61c------------------------------mean request 6.90 ms before 3.21 envoy 1.03 sidecar 1.00 worker 1.67 resume 0.95 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (98.9% ok)p50 3.7 ms p95 78.4 msin-flight 2933 pool 3285------------------------------envoy 2.52c sidecar 1.90cwrk hottest 0.85c mean 0.63c------------------------------mean request 17.32 ms before 4.47 envoy 3.20 sidecar 4.59 worker 5.06 resume 4.58 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 3.7 ms p95 10.9 msin-flight 212 pool 3285------------------------------envoy 2.39c sidecar 1.90cwrk hottest 0.87c mean 0.60c------------------------------mean request 4.75 ms before 1.96 envoy 0.74 sidecar 0.90 worker 1.16 resume 0.86 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 5.3 ms p95 305.1 msin-flight 5540 pool 6436------------------------------envoy 2.64c sidecar 1.87cwrk hottest 0.80c mean 0.66c------------------------------mean request 52.23 ms before 29.39 envoy 8.65 sidecar 1.38 worker 12.81 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.2 ms p95 159.1 msin-flight 2403 pool 6436------------------------------envoy 2.73c sidecar 2.01cwrk hottest 0.92c mean 0.68c------------------------------mean request 30.18 ms before 18.11 envoy 4.11 sidecar 1.36 worker 6.59 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.0 ms p95 16.4 msin-flight 659 pool 6436------------------------------envoy 2.80c sidecar 2.07cwrk hottest 0.92c mean 0.70c------------------------------mean request 6.04 ms before 1.92 envoy 1.23 sidecar 0.91 worker 1.98 resume 0.86 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.3 ms p95 14.0 msin-flight 324 pool 6436------------------------------envoy 3.03c sidecar 2.20cwrk hottest 0.91c mean 0.72c------------------------------mean request 5.81 ms before 2.37 envoy 1.00 sidecar 0.88 worker 1.57 resume 0.83 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.1 ms p95 12.2 msin-flight 350 pool 6436------------------------------envoy 3.03c sidecar 2.26cwrk hottest 0.90c mean 0.76c------------------------------mean request 5.46 ms before 2.19 envoy 0.94 sidecar 0.85 worker 1.48 resume 0.80 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.5 ms p95 15.3 msin-flight 543 pool 6436------------------------------envoy 3.18c sidecar 2.31cwrk hottest 0.86c mean 0.80c------------------------------mean request 6.22 ms before 2.76 envoy 0.94 sidecar 0.96 worker 1.55 resume 0.92 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.9 ms p95 17.1 msin-flight 550 pool 6436------------------------------envoy 3.24c sidecar 2.39cwrk hottest 0.90c mean 0.81c------------------------------mean request 6.87 ms before 2.78 envoy 1.15 sidecar 0.99 worker 1.95 resume 0.95 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.8 ms p95 16.2 msin-flight 2118 pool 6436------------------------------envoy 3.26c sidecar 2.39cwrk hottest 0.87c mean 0.82c------------------------------mean request 6.61 ms before 2.56 envoy 1.24 sidecar 0.90 worker 1.91 resume 0.85 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 5.8 ms p95 29.9 msin-flight 2410 pool 2587------------------------------envoy 3.25c sidecar 2.40cwrk hottest 0.89c mean 0.82c------------------------------mean request 12.02 ms before 3.81 envoy 2.92 sidecar 0.93 worker 4.35 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.1 ms p95 51.4 msin-flight 2423 pool 2714------------------------------envoy 3.39c sidecar 2.50cwrk hottest 0.90c mean 0.83c------------------------------mean request 13.65 ms before 3.82 envoy 2.87 sidecar 1.93 worker 5.04 resume 1.88 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 6.9 ms p95 18.6 msin-flight 492 pool 2714------------------------------envoy 3.44c sidecar 2.41cwrk hottest 0.90c mean 0.84c------------------------------mean request 8.48 ms before 2.88 envoy 1.74 sidecar 0.94 worker 2.92 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.4 ms p95 25.4 msin-flight 511 pool 2714------------------------------envoy 3.41c sidecar 2.45cwrk hottest 0.89c mean 0.85c------------------------------mean request 10.12 ms before 3.62 envoy 2.03 sidecar 0.97 worker 3.50 resume 0.93 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 10.6 ms p95 28.3 msin-flight 414 pool 2714------------------------------envoy 3.45c sidecar 2.51cwrk hottest 0.90c mean 0.86c------------------------------mean request 12.62 ms before 4.68 envoy 2.54 sidecar 1.00 worker 4.41 resume 0.95 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 9.8 ms p95 44.5 msin-flight 2294 pool 2714------------------------------envoy 3.47c sidecar 2.52cwrk hottest 0.90c mean 0.86c------------------------------mean request 15.59 ms before 4.75 envoy 3.45 sidecar 1.63 worker 5.76 resume 1.58 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (99.6% ok)p50 16.1 ms p95 96.4 msin-flight 3565 pool 3690------------------------------envoy 3.43c sidecar 2.49cwrk hottest 0.89c mean 0.87c------------------------------mean request 27.16 ms before 10.58 envoy 5.05 sidecar 2.46 worker 9.07 resume 2.41 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 21.6 ms p95 77.4 msin-flight 1131 pool 3690------------------------------envoy 3.52c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 28.65 ms before 9.47 envoy 6.52 sidecar 1.23 worker 11.44 resume 1.18 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 20.3 ms p95 60.1 msin-flight 1072 pool 3690------------------------------envoy 3.53c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 24.95 ms before 10.84 envoy 4.66 sidecar 1.15 worker 8.30 resume 1.10 (control plane) +rung 16 17k qps setpoint [warmup]------------------------------offered 17k /sachieved 16k /s (100.0% ok)p50 40.0 ms p95 934.2 msin-flight 9399 pool 11k------------------------------envoy 3.51c sidecar 2.43cwrk hottest 0.90c mean 0.88c------------------------------mean request 224.00 ms before -136.73 envoy 277.87 sidecar 2.29 worker 80.57 resume 2.17 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.8% ok)p50 137.8 ms p95 1400.4 msin-flight 13k pool 14k------------------------------envoy 3.49c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 617.96 ms before -445.83 envoy 943.82 sidecar 2.69 worker 117.28 resume 2.51 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.9% ok)p50 88.9 ms p95 1634.3 msin-flight 16k pool 17k------------------------------envoy 3.53c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 747.00 ms before -632.55 envoy 1249.91 sidecar 2.94 worker 126.70 resume 2.76 (control plane) +rung 17 18k qps setpoint [warmup]------------------------------offered 18k /sachieved 16k /s (95.5% ok)p50 846.1 ms p95 2730.7 msin-flight 36k pool 35k------------------------------envoy 3.48c sidecar 2.37cwrk hottest 0.89c mean 0.88c------------------------------mean request 1164.59 ms before -711.52 envoy 1744.49 sidecar 4.67 worker 126.95 resume 3.64 (control plane) +rung -1 0 qps setpoint------------------------------offered 267 /sachieved 267 /s (100.0% ok)p50 2.9 ms p95 5.8 msin-flight 20.0 pool 23.0------------------------------envoy 0.08c sidecar 0.09cwrk hottest 0.13c mean 0.07c------------------------------mean request 3.39 ms before 0.95 envoy 0.73 sidecar 0.79 worker 0.92 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.5 msin-flight 5.0 pool 23.0------------------------------envoy 0.11c sidecar 0.26cwrk hottest 0.11c mean 0.06c------------------------------mean request 2.88 ms before 1.06 envoy 0.78 sidecar 0.79 worker 0.25 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.4 msin-flight 5.0 pool 23.0------------------------------envoy 0.26c sidecar 0.26cwrk hottest 0.12c mean 0.06c------------------------------mean request 2.86 ms before 1.03 envoy 0.80 sidecar 0.79 worker 0.23 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1412 /sachieved 1412 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 24.0------------------------------envoy 0.24c sidecar 0.34cwrk hottest 0.27c mean 0.09c------------------------------mean request 2.81 ms before 1.00 envoy 0.76 sidecar 0.80 worker 0.25 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.51c sidecar 0.49cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.92 envoy 0.85 sidecar 0.83 worker 0.18 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 10.0 pool 24.0------------------------------envoy 0.51c sidecar 0.51cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.95 envoy 0.86 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2270 /sachieved 2270 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.52c sidecar 0.52cwrk hottest 0.35c mean 0.15c------------------------------mean request 2.76 ms before 0.94 envoy 0.84 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 17.0 pool 24.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.35c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.16 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 22.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.36c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.17 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3170 /sachieved 3170 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 24.0 pool 25.0------------------------------envoy 0.81c sidecar 0.73cwrk hottest 0.35c mean 0.20c------------------------------mean request 2.75 ms before 0.87 envoy 0.84 sidecar 0.84 worker 0.21 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 4.2 msin-flight 532 pool 704------------------------------envoy 1.01c sidecar 0.85cwrk hottest 0.67c mean 0.25c------------------------------mean request 5.31 ms before 2.81 envoy 0.91 sidecar 1.14 worker 0.46 resume 1.09 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 49.0 pool 704------------------------------envoy 0.96c sidecar 0.91cwrk hottest 0.70c mean 0.23c------------------------------mean request 2.93 ms before 0.99 envoy 0.68 sidecar 0.83 worker 0.43 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4491 /sachieved 4491 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 51.0 pool 704------------------------------envoy 1.03c sidecar 0.96cwrk hottest 0.82c mean 0.26c------------------------------mean request 3.06 ms before 1.08 envoy 0.65 sidecar 0.82 worker 0.51 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 44.0 pool 704------------------------------envoy 1.16c sidecar 1.08cwrk hottest 0.84c mean 0.28c------------------------------mean request 3.11 ms before 1.07 envoy 0.61 sidecar 0.82 worker 0.61 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 53.0 pool 704------------------------------envoy 1.08c sidecar 1.09cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.39 ms before 1.16 envoy 0.62 sidecar 0.82 worker 0.79 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5362 /sachieved 5362 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 154 pool 704------------------------------envoy 1.24c sidecar 1.13cwrk hottest 0.83c mean 0.31c------------------------------mean request 3.23 ms before 1.08 envoy 0.64 sidecar 0.84 worker 0.67 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 5.1 msin-flight 1867 pool 1881------------------------------envoy 1.32c sidecar 1.27cwrk hottest 0.83c mean 0.37c------------------------------mean request 7.34 ms before 1.36 envoy 1.23 sidecar 2.79 worker 1.96 resume 2.75 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 4.9 msin-flight 100 pool 1881------------------------------envoy 1.46c sidecar 1.29cwrk hottest 0.84c mean 0.37c------------------------------mean request 3.25 ms before 1.11 envoy 0.67 sidecar 0.83 worker 0.64 resume 0.79 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6758 /sachieved 6758 /s (100.0% ok)p50 3.4 ms p95 8.2 msin-flight 103 pool 1881------------------------------envoy 1.60c sidecar 1.40cwrk hottest 0.92c mean 0.40c------------------------------mean request 4.09 ms before 1.55 envoy 0.79 sidecar 0.85 worker 0.89 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 106 pool 1881------------------------------envoy 1.75c sidecar 1.44cwrk hottest 0.85c mean 0.44c------------------------------mean request 3.61 ms before 1.36 envoy 0.70 sidecar 0.86 worker 0.69 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7116 /sachieved 7116 /s (100.0% ok)p50 3.4 ms p95 14.4 msin-flight 1611 pool 1881------------------------------envoy 1.69c sidecar 1.43cwrk hottest 0.90c mean 0.42c------------------------------mean request 8.76 ms before 2.25 envoy 1.03 sidecar 3.35 worker 2.13 resume 3.01 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 4.6 ms p95 16.4 msin-flight 150 pool 1881------------------------------envoy 1.74c sidecar 1.43cwrk hottest 0.97c mean 0.44c------------------------------mean request 6.39 ms before 1.79 envoy 1.44 sidecar 0.89 worker 2.27 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 4.3 ms p95 18.2 msin-flight 2220 pool 2252------------------------------envoy 1.79c sidecar 1.51cwrk hottest 0.95c mean 0.44c------------------------------mean request 11.66 ms before 5.04 envoy 1.00 sidecar 2.95 worker 2.67 resume 2.91 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8001 /s (100.0% ok)p50 3.0 ms p95 5.7 msin-flight 210 pool 2252------------------------------envoy 1.94c sidecar 1.65cwrk hottest 0.73c mean 0.51c------------------------------mean request 3.54 ms before 1.18 envoy 0.68 sidecar 0.98 worker 0.70 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8253 /sachieved 8253 /s (100.0% ok)p50 3.2 ms p95 6.4 msin-flight 219 pool 2252------------------------------envoy 2.10c sidecar 1.69cwrk hottest 0.80c mean 0.54c------------------------------mean request 3.71 ms before 1.28 envoy 0.69 sidecar 0.96 worker 0.78 resume 0.91 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.2 ms p95 8.4 msin-flight 727 pool 2252------------------------------envoy 2.11c sidecar 1.81cwrk hottest 0.80c mean 0.57c------------------------------mean request 4.39 ms before 1.55 envoy 0.74 sidecar 1.21 worker 0.89 resume 1.16 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.3 ms p95 7.8 msin-flight 532 pool 2252------------------------------envoy 2.22c sidecar 1.79cwrk hottest 0.85c mean 0.55c------------------------------mean request 4.05 ms before 1.60 envoy 0.53 sidecar 1.03 worker 0.89 resume 0.98 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9338 /sachieved 9332 /s (100.0% ok)p50 3.6 ms p95 18.8 msin-flight 225 pool 2252------------------------------envoy 2.34c sidecar 1.75cwrk hottest 0.92c mean 0.58c------------------------------mean request 6.36 ms before 3.08 envoy 1.04 sidecar 0.96 worker 1.28 resume 0.92 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 4.0 ms p95 19.9 msin-flight 1033 pool 2252------------------------------envoy 2.28c sidecar 1.90cwrk hottest 0.93c mean 0.61c------------------------------mean request 6.90 ms before 3.21 envoy 1.03 sidecar 1.00 worker 1.67 resume 0.95 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (98.9% ok)p50 3.7 ms p95 78.4 msin-flight 2933 pool 3285------------------------------envoy 2.52c sidecar 1.90cwrk hottest 0.85c mean 0.63c------------------------------mean request 17.32 ms before 4.47 envoy 3.20 sidecar 4.59 worker 5.06 resume 4.58 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 3.7 ms p95 10.9 msin-flight 212 pool 3285------------------------------envoy 2.39c sidecar 1.90cwrk hottest 0.87c mean 0.60c------------------------------mean request 4.75 ms before 1.96 envoy 0.74 sidecar 0.90 worker 1.16 resume 0.86 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 5.3 ms p95 305.1 msin-flight 5540 pool 6436------------------------------envoy 2.64c sidecar 1.87cwrk hottest 0.80c mean 0.66c------------------------------mean request 52.23 ms before 29.39 envoy 8.65 sidecar 1.38 worker 12.81 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.2 ms p95 159.1 msin-flight 2403 pool 6436------------------------------envoy 2.73c sidecar 2.01cwrk hottest 0.92c mean 0.68c------------------------------mean request 30.18 ms before 18.11 envoy 4.11 sidecar 1.36 worker 6.59 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.0 ms p95 16.4 msin-flight 659 pool 6436------------------------------envoy 2.80c sidecar 2.07cwrk hottest 0.92c mean 0.70c------------------------------mean request 6.04 ms before 1.92 envoy 1.23 sidecar 0.91 worker 1.98 resume 0.86 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.3 ms p95 14.0 msin-flight 324 pool 6436------------------------------envoy 3.03c sidecar 2.20cwrk hottest 0.91c mean 0.72c------------------------------mean request 5.81 ms before 2.37 envoy 1.00 sidecar 0.88 worker 1.57 resume 0.83 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.1 ms p95 12.2 msin-flight 350 pool 6436------------------------------envoy 3.03c sidecar 2.26cwrk hottest 0.90c mean 0.76c------------------------------mean request 5.46 ms before 2.19 envoy 0.94 sidecar 0.85 worker 1.48 resume 0.80 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.5 ms p95 15.3 msin-flight 543 pool 6436------------------------------envoy 3.18c sidecar 2.31cwrk hottest 0.86c mean 0.80c------------------------------mean request 6.22 ms before 2.76 envoy 0.94 sidecar 0.96 worker 1.55 resume 0.92 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.9 ms p95 17.1 msin-flight 550 pool 6436------------------------------envoy 3.24c sidecar 2.39cwrk hottest 0.90c mean 0.81c------------------------------mean request 6.87 ms before 2.78 envoy 1.15 sidecar 0.99 worker 1.95 resume 0.95 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.8 ms p95 16.2 msin-flight 2118 pool 6436------------------------------envoy 3.26c sidecar 2.39cwrk hottest 0.87c mean 0.82c------------------------------mean request 6.61 ms before 2.56 envoy 1.24 sidecar 0.90 worker 1.91 resume 0.85 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 5.8 ms p95 29.9 msin-flight 2410 pool 2587------------------------------envoy 3.25c sidecar 2.40cwrk hottest 0.89c mean 0.82c------------------------------mean request 12.02 ms before 3.81 envoy 2.92 sidecar 0.93 worker 4.35 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.1 ms p95 51.4 msin-flight 2423 pool 2714------------------------------envoy 3.39c sidecar 2.50cwrk hottest 0.90c mean 0.83c------------------------------mean request 13.65 ms before 3.82 envoy 2.87 sidecar 1.93 worker 5.04 resume 1.88 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 6.9 ms p95 18.6 msin-flight 492 pool 2714------------------------------envoy 3.44c sidecar 2.41cwrk hottest 0.90c mean 0.84c------------------------------mean request 8.48 ms before 2.88 envoy 1.74 sidecar 0.94 worker 2.92 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.4 ms p95 25.4 msin-flight 511 pool 2714------------------------------envoy 3.41c sidecar 2.45cwrk hottest 0.89c mean 0.85c------------------------------mean request 10.12 ms before 3.62 envoy 2.03 sidecar 0.97 worker 3.50 resume 0.93 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 10.6 ms p95 28.3 msin-flight 414 pool 2714------------------------------envoy 3.45c sidecar 2.51cwrk hottest 0.90c mean 0.86c------------------------------mean request 12.62 ms before 4.68 envoy 2.54 sidecar 1.00 worker 4.41 resume 0.95 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 9.8 ms p95 44.5 msin-flight 2294 pool 2714------------------------------envoy 3.47c sidecar 2.52cwrk hottest 0.90c mean 0.86c------------------------------mean request 15.59 ms before 4.75 envoy 3.45 sidecar 1.63 worker 5.76 resume 1.58 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (99.6% ok)p50 16.1 ms p95 96.4 msin-flight 3565 pool 3690------------------------------envoy 3.43c sidecar 2.49cwrk hottest 0.89c mean 0.87c------------------------------mean request 27.16 ms before 10.58 envoy 5.05 sidecar 2.46 worker 9.07 resume 2.41 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 21.6 ms p95 77.4 msin-flight 1131 pool 3690------------------------------envoy 3.52c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 28.65 ms before 9.47 envoy 6.52 sidecar 1.23 worker 11.44 resume 1.18 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 20.3 ms p95 60.1 msin-flight 1072 pool 3690------------------------------envoy 3.53c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 24.95 ms before 10.84 envoy 4.66 sidecar 1.15 worker 8.30 resume 1.10 (control plane) +rung 16 17k qps setpoint [warmup]------------------------------offered 17k /sachieved 16k /s (100.0% ok)p50 40.0 ms p95 934.2 msin-flight 9399 pool 11k------------------------------envoy 3.51c sidecar 2.43cwrk hottest 0.90c mean 0.88c------------------------------mean request 224.00 ms before -136.73 envoy 277.87 sidecar 2.29 worker 80.57 resume 2.17 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.8% ok)p50 137.8 ms p95 1400.4 msin-flight 13k pool 14k------------------------------envoy 3.49c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 617.96 ms before -445.83 envoy 943.82 sidecar 2.69 worker 117.28 resume 2.51 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.9% ok)p50 88.9 ms p95 1634.3 msin-flight 16k pool 17k------------------------------envoy 3.53c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 747.00 ms before -632.55 envoy 1249.91 sidecar 2.94 worker 126.70 resume 2.76 (control plane) +rung 17 18k qps setpoint [warmup]------------------------------offered 18k /sachieved 16k /s (95.5% ok)p50 846.1 ms p95 2730.7 msin-flight 36k pool 35k------------------------------envoy 3.48c sidecar 2.37cwrk hottest 0.89c mean 0.88c------------------------------mean request 1164.59 ms before -711.52 envoy 1744.49 sidecar 4.67 worker 126.95 resume 3.64 (control plane) +rung -1 0 qps setpoint------------------------------offered 267 /sachieved 267 /s (100.0% ok)p50 2.9 ms p95 5.8 msin-flight 20.0 pool 23.0------------------------------envoy 0.08c sidecar 0.09cwrk hottest 0.13c mean 0.07c------------------------------mean request 3.39 ms before 0.95 envoy 0.73 sidecar 0.79 worker 0.92 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.5 msin-flight 5.0 pool 23.0------------------------------envoy 0.11c sidecar 0.26cwrk hottest 0.11c mean 0.06c------------------------------mean request 2.88 ms before 1.06 envoy 0.78 sidecar 0.79 worker 0.25 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.4 msin-flight 5.0 pool 23.0------------------------------envoy 0.26c sidecar 0.26cwrk hottest 0.12c mean 0.06c------------------------------mean request 2.86 ms before 1.03 envoy 0.80 sidecar 0.79 worker 0.23 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1412 /sachieved 1412 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 24.0------------------------------envoy 0.24c sidecar 0.34cwrk hottest 0.27c mean 0.09c------------------------------mean request 2.81 ms before 1.00 envoy 0.76 sidecar 0.80 worker 0.25 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.51c sidecar 0.49cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.92 envoy 0.85 sidecar 0.83 worker 0.18 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 10.0 pool 24.0------------------------------envoy 0.51c sidecar 0.51cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.95 envoy 0.86 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2270 /sachieved 2270 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.52c sidecar 0.52cwrk hottest 0.35c mean 0.15c------------------------------mean request 2.76 ms before 0.94 envoy 0.84 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 17.0 pool 24.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.35c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.16 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 22.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.36c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.17 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3170 /sachieved 3170 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 24.0 pool 25.0------------------------------envoy 0.81c sidecar 0.73cwrk hottest 0.35c mean 0.20c------------------------------mean request 2.75 ms before 0.87 envoy 0.84 sidecar 0.84 worker 0.21 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 4.2 msin-flight 532 pool 704------------------------------envoy 1.01c sidecar 0.85cwrk hottest 0.67c mean 0.25c------------------------------mean request 5.31 ms before 2.81 envoy 0.91 sidecar 1.14 worker 0.46 resume 1.09 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 49.0 pool 704------------------------------envoy 0.96c sidecar 0.91cwrk hottest 0.70c mean 0.23c------------------------------mean request 2.93 ms before 0.99 envoy 0.68 sidecar 0.83 worker 0.43 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4491 /sachieved 4491 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 51.0 pool 704------------------------------envoy 1.03c sidecar 0.96cwrk hottest 0.82c mean 0.26c------------------------------mean request 3.06 ms before 1.08 envoy 0.65 sidecar 0.82 worker 0.51 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 44.0 pool 704------------------------------envoy 1.16c sidecar 1.08cwrk hottest 0.84c mean 0.28c------------------------------mean request 3.11 ms before 1.07 envoy 0.61 sidecar 0.82 worker 0.61 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 53.0 pool 704------------------------------envoy 1.08c sidecar 1.09cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.39 ms before 1.16 envoy 0.62 sidecar 0.82 worker 0.79 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5362 /sachieved 5362 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 154 pool 704------------------------------envoy 1.24c sidecar 1.13cwrk hottest 0.83c mean 0.31c------------------------------mean request 3.23 ms before 1.08 envoy 0.64 sidecar 0.84 worker 0.67 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 5.1 msin-flight 1867 pool 1881------------------------------envoy 1.32c sidecar 1.27cwrk hottest 0.83c mean 0.37c------------------------------mean request 7.34 ms before 1.36 envoy 1.23 sidecar 2.79 worker 1.96 resume 2.75 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 4.9 msin-flight 100 pool 1881------------------------------envoy 1.46c sidecar 1.29cwrk hottest 0.84c mean 0.37c------------------------------mean request 3.25 ms before 1.11 envoy 0.67 sidecar 0.83 worker 0.64 resume 0.79 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6758 /sachieved 6758 /s (100.0% ok)p50 3.4 ms p95 8.2 msin-flight 103 pool 1881------------------------------envoy 1.60c sidecar 1.40cwrk hottest 0.92c mean 0.40c------------------------------mean request 4.09 ms before 1.55 envoy 0.79 sidecar 0.85 worker 0.89 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 106 pool 1881------------------------------envoy 1.75c sidecar 1.44cwrk hottest 0.85c mean 0.44c------------------------------mean request 3.61 ms before 1.36 envoy 0.70 sidecar 0.86 worker 0.69 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7116 /sachieved 7116 /s (100.0% ok)p50 3.4 ms p95 14.4 msin-flight 1611 pool 1881------------------------------envoy 1.69c sidecar 1.43cwrk hottest 0.90c mean 0.42c------------------------------mean request 8.76 ms before 2.25 envoy 1.03 sidecar 3.35 worker 2.13 resume 3.01 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 4.6 ms p95 16.4 msin-flight 150 pool 1881------------------------------envoy 1.74c sidecar 1.43cwrk hottest 0.97c mean 0.44c------------------------------mean request 6.39 ms before 1.79 envoy 1.44 sidecar 0.89 worker 2.27 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 4.3 ms p95 18.2 msin-flight 2220 pool 2252------------------------------envoy 1.79c sidecar 1.51cwrk hottest 0.95c mean 0.44c------------------------------mean request 11.66 ms before 5.04 envoy 1.00 sidecar 2.95 worker 2.67 resume 2.91 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8001 /s (100.0% ok)p50 3.0 ms p95 5.7 msin-flight 210 pool 2252------------------------------envoy 1.94c sidecar 1.65cwrk hottest 0.73c mean 0.51c------------------------------mean request 3.54 ms before 1.18 envoy 0.68 sidecar 0.98 worker 0.70 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8253 /sachieved 8253 /s (100.0% ok)p50 3.2 ms p95 6.4 msin-flight 219 pool 2252------------------------------envoy 2.10c sidecar 1.69cwrk hottest 0.80c mean 0.54c------------------------------mean request 3.71 ms before 1.28 envoy 0.69 sidecar 0.96 worker 0.78 resume 0.91 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.2 ms p95 8.4 msin-flight 727 pool 2252------------------------------envoy 2.11c sidecar 1.81cwrk hottest 0.80c mean 0.57c------------------------------mean request 4.39 ms before 1.55 envoy 0.74 sidecar 1.21 worker 0.89 resume 1.16 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.3 ms p95 7.8 msin-flight 532 pool 2252------------------------------envoy 2.22c sidecar 1.79cwrk hottest 0.85c mean 0.55c------------------------------mean request 4.05 ms before 1.60 envoy 0.53 sidecar 1.03 worker 0.89 resume 0.98 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9338 /sachieved 9332 /s (100.0% ok)p50 3.6 ms p95 18.8 msin-flight 225 pool 2252------------------------------envoy 2.34c sidecar 1.75cwrk hottest 0.92c mean 0.58c------------------------------mean request 6.36 ms before 3.08 envoy 1.04 sidecar 0.96 worker 1.28 resume 0.92 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 4.0 ms p95 19.9 msin-flight 1033 pool 2252------------------------------envoy 2.28c sidecar 1.90cwrk hottest 0.93c mean 0.61c------------------------------mean request 6.90 ms before 3.21 envoy 1.03 sidecar 1.00 worker 1.67 resume 0.95 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (98.9% ok)p50 3.7 ms p95 78.4 msin-flight 2933 pool 3285------------------------------envoy 2.52c sidecar 1.90cwrk hottest 0.85c mean 0.63c------------------------------mean request 17.32 ms before 4.47 envoy 3.20 sidecar 4.59 worker 5.06 resume 4.58 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 3.7 ms p95 10.9 msin-flight 212 pool 3285------------------------------envoy 2.39c sidecar 1.90cwrk hottest 0.87c mean 0.60c------------------------------mean request 4.75 ms before 1.96 envoy 0.74 sidecar 0.90 worker 1.16 resume 0.86 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 5.3 ms p95 305.1 msin-flight 5540 pool 6436------------------------------envoy 2.64c sidecar 1.87cwrk hottest 0.80c mean 0.66c------------------------------mean request 52.23 ms before 29.39 envoy 8.65 sidecar 1.38 worker 12.81 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.2 ms p95 159.1 msin-flight 2403 pool 6436------------------------------envoy 2.73c sidecar 2.01cwrk hottest 0.92c mean 0.68c------------------------------mean request 30.18 ms before 18.11 envoy 4.11 sidecar 1.36 worker 6.59 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.0 ms p95 16.4 msin-flight 659 pool 6436------------------------------envoy 2.80c sidecar 2.07cwrk hottest 0.92c mean 0.70c------------------------------mean request 6.04 ms before 1.92 envoy 1.23 sidecar 0.91 worker 1.98 resume 0.86 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.3 ms p95 14.0 msin-flight 324 pool 6436------------------------------envoy 3.03c sidecar 2.20cwrk hottest 0.91c mean 0.72c------------------------------mean request 5.81 ms before 2.37 envoy 1.00 sidecar 0.88 worker 1.57 resume 0.83 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.1 ms p95 12.2 msin-flight 350 pool 6436------------------------------envoy 3.03c sidecar 2.26cwrk hottest 0.90c mean 0.76c------------------------------mean request 5.46 ms before 2.19 envoy 0.94 sidecar 0.85 worker 1.48 resume 0.80 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.5 ms p95 15.3 msin-flight 543 pool 6436------------------------------envoy 3.18c sidecar 2.31cwrk hottest 0.86c mean 0.80c------------------------------mean request 6.22 ms before 2.76 envoy 0.94 sidecar 0.96 worker 1.55 resume 0.92 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.9 ms p95 17.1 msin-flight 550 pool 6436------------------------------envoy 3.24c sidecar 2.39cwrk hottest 0.90c mean 0.81c------------------------------mean request 6.87 ms before 2.78 envoy 1.15 sidecar 0.99 worker 1.95 resume 0.95 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.8 ms p95 16.2 msin-flight 2118 pool 6436------------------------------envoy 3.26c sidecar 2.39cwrk hottest 0.87c mean 0.82c------------------------------mean request 6.61 ms before 2.56 envoy 1.24 sidecar 0.90 worker 1.91 resume 0.85 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 5.8 ms p95 29.9 msin-flight 2410 pool 2587------------------------------envoy 3.25c sidecar 2.40cwrk hottest 0.89c mean 0.82c------------------------------mean request 12.02 ms before 3.81 envoy 2.92 sidecar 0.93 worker 4.35 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.1 ms p95 51.4 msin-flight 2423 pool 2714------------------------------envoy 3.39c sidecar 2.50cwrk hottest 0.90c mean 0.83c------------------------------mean request 13.65 ms before 3.82 envoy 2.87 sidecar 1.93 worker 5.04 resume 1.88 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 6.9 ms p95 18.6 msin-flight 492 pool 2714------------------------------envoy 3.44c sidecar 2.41cwrk hottest 0.90c mean 0.84c------------------------------mean request 8.48 ms before 2.88 envoy 1.74 sidecar 0.94 worker 2.92 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.4 ms p95 25.4 msin-flight 511 pool 2714------------------------------envoy 3.41c sidecar 2.45cwrk hottest 0.89c mean 0.85c------------------------------mean request 10.12 ms before 3.62 envoy 2.03 sidecar 0.97 worker 3.50 resume 0.93 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 10.6 ms p95 28.3 msin-flight 414 pool 2714------------------------------envoy 3.45c sidecar 2.51cwrk hottest 0.90c mean 0.86c------------------------------mean request 12.62 ms before 4.68 envoy 2.54 sidecar 1.00 worker 4.41 resume 0.95 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 9.8 ms p95 44.5 msin-flight 2294 pool 2714------------------------------envoy 3.47c sidecar 2.52cwrk hottest 0.90c mean 0.86c------------------------------mean request 15.59 ms before 4.75 envoy 3.45 sidecar 1.63 worker 5.76 resume 1.58 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (99.6% ok)p50 16.1 ms p95 96.4 msin-flight 3565 pool 3690------------------------------envoy 3.43c sidecar 2.49cwrk hottest 0.89c mean 0.87c------------------------------mean request 27.16 ms before 10.58 envoy 5.05 sidecar 2.46 worker 9.07 resume 2.41 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 21.6 ms p95 77.4 msin-flight 1131 pool 3690------------------------------envoy 3.52c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 28.65 ms before 9.47 envoy 6.52 sidecar 1.23 worker 11.44 resume 1.18 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 20.3 ms p95 60.1 msin-flight 1072 pool 3690------------------------------envoy 3.53c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 24.95 ms before 10.84 envoy 4.66 sidecar 1.15 worker 8.30 resume 1.10 (control plane) +rung 16 17k qps setpoint [warmup]------------------------------offered 17k /sachieved 16k /s (100.0% ok)p50 40.0 ms p95 934.2 msin-flight 9399 pool 11k------------------------------envoy 3.51c sidecar 2.43cwrk hottest 0.90c mean 0.88c------------------------------mean request 224.00 ms before -136.73 envoy 277.87 sidecar 2.29 worker 80.57 resume 2.17 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.8% ok)p50 137.8 ms p95 1400.4 msin-flight 13k pool 14k------------------------------envoy 3.49c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 617.96 ms before -445.83 envoy 943.82 sidecar 2.69 worker 117.28 resume 2.51 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.9% ok)p50 88.9 ms p95 1634.3 msin-flight 16k pool 17k------------------------------envoy 3.53c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 747.00 ms before -632.55 envoy 1249.91 sidecar 2.94 worker 126.70 resume 2.76 (control plane) +rung 17 18k qps setpoint [warmup]------------------------------offered 18k /sachieved 16k /s (95.5% ok)p50 846.1 ms p95 2730.7 msin-flight 36k pool 35k------------------------------envoy 3.48c sidecar 2.37cwrk hottest 0.89c mean 0.88c------------------------------mean request 1164.59 ms before -711.52 envoy 1744.49 sidecar 4.67 worker 126.95 resume 3.64 (control plane) +rung -1 0 qps setpoint------------------------------offered 267 /sachieved 267 /s (100.0% ok)p50 2.9 ms p95 5.8 msin-flight 20.0 pool 23.0------------------------------envoy 0.08c sidecar 0.09cwrk hottest 0.13c mean 0.07c------------------------------mean request 3.39 ms before 0.95 envoy 0.73 sidecar 0.79 worker 0.92 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.5 msin-flight 5.0 pool 23.0------------------------------envoy 0.11c sidecar 0.26cwrk hottest 0.11c mean 0.06c------------------------------mean request 2.88 ms before 1.06 envoy 0.78 sidecar 0.79 worker 0.25 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.4 msin-flight 5.0 pool 23.0------------------------------envoy 0.26c sidecar 0.26cwrk hottest 0.12c mean 0.06c------------------------------mean request 2.86 ms before 1.03 envoy 0.80 sidecar 0.79 worker 0.23 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1412 /sachieved 1412 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 24.0------------------------------envoy 0.24c sidecar 0.34cwrk hottest 0.27c mean 0.09c------------------------------mean request 2.81 ms before 1.00 envoy 0.76 sidecar 0.80 worker 0.25 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.51c sidecar 0.49cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.92 envoy 0.85 sidecar 0.83 worker 0.18 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 10.0 pool 24.0------------------------------envoy 0.51c sidecar 0.51cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.95 envoy 0.86 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2270 /sachieved 2270 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.52c sidecar 0.52cwrk hottest 0.35c mean 0.15c------------------------------mean request 2.76 ms before 0.94 envoy 0.84 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 17.0 pool 24.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.35c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.16 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 22.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.36c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.17 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3170 /sachieved 3170 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 24.0 pool 25.0------------------------------envoy 0.81c sidecar 0.73cwrk hottest 0.35c mean 0.20c------------------------------mean request 2.75 ms before 0.87 envoy 0.84 sidecar 0.84 worker 0.21 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 4.2 msin-flight 532 pool 704------------------------------envoy 1.01c sidecar 0.85cwrk hottest 0.67c mean 0.25c------------------------------mean request 5.31 ms before 2.81 envoy 0.91 sidecar 1.14 worker 0.46 resume 1.09 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 49.0 pool 704------------------------------envoy 0.96c sidecar 0.91cwrk hottest 0.70c mean 0.23c------------------------------mean request 2.93 ms before 0.99 envoy 0.68 sidecar 0.83 worker 0.43 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4491 /sachieved 4491 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 51.0 pool 704------------------------------envoy 1.03c sidecar 0.96cwrk hottest 0.82c mean 0.26c------------------------------mean request 3.06 ms before 1.08 envoy 0.65 sidecar 0.82 worker 0.51 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 44.0 pool 704------------------------------envoy 1.16c sidecar 1.08cwrk hottest 0.84c mean 0.28c------------------------------mean request 3.11 ms before 1.07 envoy 0.61 sidecar 0.82 worker 0.61 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 53.0 pool 704------------------------------envoy 1.08c sidecar 1.09cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.39 ms before 1.16 envoy 0.62 sidecar 0.82 worker 0.79 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5362 /sachieved 5362 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 154 pool 704------------------------------envoy 1.24c sidecar 1.13cwrk hottest 0.83c mean 0.31c------------------------------mean request 3.23 ms before 1.08 envoy 0.64 sidecar 0.84 worker 0.67 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 5.1 msin-flight 1867 pool 1881------------------------------envoy 1.32c sidecar 1.27cwrk hottest 0.83c mean 0.37c------------------------------mean request 7.34 ms before 1.36 envoy 1.23 sidecar 2.79 worker 1.96 resume 2.75 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 4.9 msin-flight 100 pool 1881------------------------------envoy 1.46c sidecar 1.29cwrk hottest 0.84c mean 0.37c------------------------------mean request 3.25 ms before 1.11 envoy 0.67 sidecar 0.83 worker 0.64 resume 0.79 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6758 /sachieved 6758 /s (100.0% ok)p50 3.4 ms p95 8.2 msin-flight 103 pool 1881------------------------------envoy 1.60c sidecar 1.40cwrk hottest 0.92c mean 0.40c------------------------------mean request 4.09 ms before 1.55 envoy 0.79 sidecar 0.85 worker 0.89 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 106 pool 1881------------------------------envoy 1.75c sidecar 1.44cwrk hottest 0.85c mean 0.44c------------------------------mean request 3.61 ms before 1.36 envoy 0.70 sidecar 0.86 worker 0.69 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7116 /sachieved 7116 /s (100.0% ok)p50 3.4 ms p95 14.4 msin-flight 1611 pool 1881------------------------------envoy 1.69c sidecar 1.43cwrk hottest 0.90c mean 0.42c------------------------------mean request 8.76 ms before 2.25 envoy 1.03 sidecar 3.35 worker 2.13 resume 3.01 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 4.6 ms p95 16.4 msin-flight 150 pool 1881------------------------------envoy 1.74c sidecar 1.43cwrk hottest 0.97c mean 0.44c------------------------------mean request 6.39 ms before 1.79 envoy 1.44 sidecar 0.89 worker 2.27 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 4.3 ms p95 18.2 msin-flight 2220 pool 2252------------------------------envoy 1.79c sidecar 1.51cwrk hottest 0.95c mean 0.44c------------------------------mean request 11.66 ms before 5.04 envoy 1.00 sidecar 2.95 worker 2.67 resume 2.91 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8001 /s (100.0% ok)p50 3.0 ms p95 5.7 msin-flight 210 pool 2252------------------------------envoy 1.94c sidecar 1.65cwrk hottest 0.73c mean 0.51c------------------------------mean request 3.54 ms before 1.18 envoy 0.68 sidecar 0.98 worker 0.70 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8253 /sachieved 8253 /s (100.0% ok)p50 3.2 ms p95 6.4 msin-flight 219 pool 2252------------------------------envoy 2.10c sidecar 1.69cwrk hottest 0.80c mean 0.54c------------------------------mean request 3.71 ms before 1.28 envoy 0.69 sidecar 0.96 worker 0.78 resume 0.91 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.2 ms p95 8.4 msin-flight 727 pool 2252------------------------------envoy 2.11c sidecar 1.81cwrk hottest 0.80c mean 0.57c------------------------------mean request 4.39 ms before 1.55 envoy 0.74 sidecar 1.21 worker 0.89 resume 1.16 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.3 ms p95 7.8 msin-flight 532 pool 2252------------------------------envoy 2.22c sidecar 1.79cwrk hottest 0.85c mean 0.55c------------------------------mean request 4.05 ms before 1.60 envoy 0.53 sidecar 1.03 worker 0.89 resume 0.98 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9338 /sachieved 9332 /s (100.0% ok)p50 3.6 ms p95 18.8 msin-flight 225 pool 2252------------------------------envoy 2.34c sidecar 1.75cwrk hottest 0.92c mean 0.58c------------------------------mean request 6.36 ms before 3.08 envoy 1.04 sidecar 0.96 worker 1.28 resume 0.92 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 4.0 ms p95 19.9 msin-flight 1033 pool 2252------------------------------envoy 2.28c sidecar 1.90cwrk hottest 0.93c mean 0.61c------------------------------mean request 6.90 ms before 3.21 envoy 1.03 sidecar 1.00 worker 1.67 resume 0.95 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (98.9% ok)p50 3.7 ms p95 78.4 msin-flight 2933 pool 3285------------------------------envoy 2.52c sidecar 1.90cwrk hottest 0.85c mean 0.63c------------------------------mean request 17.32 ms before 4.47 envoy 3.20 sidecar 4.59 worker 5.06 resume 4.58 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 3.7 ms p95 10.9 msin-flight 212 pool 3285------------------------------envoy 2.39c sidecar 1.90cwrk hottest 0.87c mean 0.60c------------------------------mean request 4.75 ms before 1.96 envoy 0.74 sidecar 0.90 worker 1.16 resume 0.86 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 5.3 ms p95 305.1 msin-flight 5540 pool 6436------------------------------envoy 2.64c sidecar 1.87cwrk hottest 0.80c mean 0.66c------------------------------mean request 52.23 ms before 29.39 envoy 8.65 sidecar 1.38 worker 12.81 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.2 ms p95 159.1 msin-flight 2403 pool 6436------------------------------envoy 2.73c sidecar 2.01cwrk hottest 0.92c mean 0.68c------------------------------mean request 30.18 ms before 18.11 envoy 4.11 sidecar 1.36 worker 6.59 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.0 ms p95 16.4 msin-flight 659 pool 6436------------------------------envoy 2.80c sidecar 2.07cwrk hottest 0.92c mean 0.70c------------------------------mean request 6.04 ms before 1.92 envoy 1.23 sidecar 0.91 worker 1.98 resume 0.86 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.3 ms p95 14.0 msin-flight 324 pool 6436------------------------------envoy 3.03c sidecar 2.20cwrk hottest 0.91c mean 0.72c------------------------------mean request 5.81 ms before 2.37 envoy 1.00 sidecar 0.88 worker 1.57 resume 0.83 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.1 ms p95 12.2 msin-flight 350 pool 6436------------------------------envoy 3.03c sidecar 2.26cwrk hottest 0.90c mean 0.76c------------------------------mean request 5.46 ms before 2.19 envoy 0.94 sidecar 0.85 worker 1.48 resume 0.80 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.5 ms p95 15.3 msin-flight 543 pool 6436------------------------------envoy 3.18c sidecar 2.31cwrk hottest 0.86c mean 0.80c------------------------------mean request 6.22 ms before 2.76 envoy 0.94 sidecar 0.96 worker 1.55 resume 0.92 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.9 ms p95 17.1 msin-flight 550 pool 6436------------------------------envoy 3.24c sidecar 2.39cwrk hottest 0.90c mean 0.81c------------------------------mean request 6.87 ms before 2.78 envoy 1.15 sidecar 0.99 worker 1.95 resume 0.95 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.8 ms p95 16.2 msin-flight 2118 pool 6436------------------------------envoy 3.26c sidecar 2.39cwrk hottest 0.87c mean 0.82c------------------------------mean request 6.61 ms before 2.56 envoy 1.24 sidecar 0.90 worker 1.91 resume 0.85 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 5.8 ms p95 29.9 msin-flight 2410 pool 2587------------------------------envoy 3.25c sidecar 2.40cwrk hottest 0.89c mean 0.82c------------------------------mean request 12.02 ms before 3.81 envoy 2.92 sidecar 0.93 worker 4.35 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.1 ms p95 51.4 msin-flight 2423 pool 2714------------------------------envoy 3.39c sidecar 2.50cwrk hottest 0.90c mean 0.83c------------------------------mean request 13.65 ms before 3.82 envoy 2.87 sidecar 1.93 worker 5.04 resume 1.88 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 6.9 ms p95 18.6 msin-flight 492 pool 2714------------------------------envoy 3.44c sidecar 2.41cwrk hottest 0.90c mean 0.84c------------------------------mean request 8.48 ms before 2.88 envoy 1.74 sidecar 0.94 worker 2.92 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.4 ms p95 25.4 msin-flight 511 pool 2714------------------------------envoy 3.41c sidecar 2.45cwrk hottest 0.89c mean 0.85c------------------------------mean request 10.12 ms before 3.62 envoy 2.03 sidecar 0.97 worker 3.50 resume 0.93 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 10.6 ms p95 28.3 msin-flight 414 pool 2714------------------------------envoy 3.45c sidecar 2.51cwrk hottest 0.90c mean 0.86c------------------------------mean request 12.62 ms before 4.68 envoy 2.54 sidecar 1.00 worker 4.41 resume 0.95 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 9.8 ms p95 44.5 msin-flight 2294 pool 2714------------------------------envoy 3.47c sidecar 2.52cwrk hottest 0.90c mean 0.86c------------------------------mean request 15.59 ms before 4.75 envoy 3.45 sidecar 1.63 worker 5.76 resume 1.58 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (99.6% ok)p50 16.1 ms p95 96.4 msin-flight 3565 pool 3690------------------------------envoy 3.43c sidecar 2.49cwrk hottest 0.89c mean 0.87c------------------------------mean request 27.16 ms before 10.58 envoy 5.05 sidecar 2.46 worker 9.07 resume 2.41 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 21.6 ms p95 77.4 msin-flight 1131 pool 3690------------------------------envoy 3.52c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 28.65 ms before 9.47 envoy 6.52 sidecar 1.23 worker 11.44 resume 1.18 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 20.3 ms p95 60.1 msin-flight 1072 pool 3690------------------------------envoy 3.53c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 24.95 ms before 10.84 envoy 4.66 sidecar 1.15 worker 8.30 resume 1.10 (control plane) +rung 16 17k qps setpoint [warmup]------------------------------offered 17k /sachieved 16k /s (100.0% ok)p50 40.0 ms p95 934.2 msin-flight 9399 pool 11k------------------------------envoy 3.51c sidecar 2.43cwrk hottest 0.90c mean 0.88c------------------------------mean request 224.00 ms before -136.73 envoy 277.87 sidecar 2.29 worker 80.57 resume 2.17 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.8% ok)p50 137.8 ms p95 1400.4 msin-flight 13k pool 14k------------------------------envoy 3.49c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 617.96 ms before -445.83 envoy 943.82 sidecar 2.69 worker 117.28 resume 2.51 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.9% ok)p50 88.9 ms p95 1634.3 msin-flight 16k pool 17k------------------------------envoy 3.53c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 747.00 ms before -632.55 envoy 1249.91 sidecar 2.94 worker 126.70 resume 2.76 (control plane) +rung 17 18k qps setpoint [warmup]------------------------------offered 18k /sachieved 16k /s (95.5% ok)p50 846.1 ms p95 2730.7 msin-flight 36k pool 35k------------------------------envoy 3.48c sidecar 2.37cwrk hottest 0.89c mean 0.88c------------------------------mean request 1164.59 ms before -711.52 envoy 1744.49 sidecar 4.67 worker 126.95 resume 3.64 (control plane) +rung -1 0 qps setpoint------------------------------offered 267 /sachieved 267 /s (100.0% ok)p50 2.9 ms p95 5.8 msin-flight 20.0 pool 23.0------------------------------envoy 0.08c sidecar 0.09cwrk hottest 0.13c mean 0.07c------------------------------mean request 3.39 ms before 0.95 envoy 0.73 sidecar 0.79 worker 0.92 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.5 msin-flight 5.0 pool 23.0------------------------------envoy 0.11c sidecar 0.26cwrk hottest 0.11c mean 0.06c------------------------------mean request 2.88 ms before 1.06 envoy 0.78 sidecar 0.79 worker 0.25 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.4 msin-flight 5.0 pool 23.0------------------------------envoy 0.26c sidecar 0.26cwrk hottest 0.12c mean 0.06c------------------------------mean request 2.86 ms before 1.03 envoy 0.80 sidecar 0.79 worker 0.23 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1412 /sachieved 1412 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 24.0------------------------------envoy 0.24c sidecar 0.34cwrk hottest 0.27c mean 0.09c------------------------------mean request 2.81 ms before 1.00 envoy 0.76 sidecar 0.80 worker 0.25 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.51c sidecar 0.49cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.92 envoy 0.85 sidecar 0.83 worker 0.18 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 10.0 pool 24.0------------------------------envoy 0.51c sidecar 0.51cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.95 envoy 0.86 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2270 /sachieved 2270 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.52c sidecar 0.52cwrk hottest 0.35c mean 0.15c------------------------------mean request 2.76 ms before 0.94 envoy 0.84 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 17.0 pool 24.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.35c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.16 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 22.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.36c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.17 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3170 /sachieved 3170 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 24.0 pool 25.0------------------------------envoy 0.81c sidecar 0.73cwrk hottest 0.35c mean 0.20c------------------------------mean request 2.75 ms before 0.87 envoy 0.84 sidecar 0.84 worker 0.21 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 4.2 msin-flight 532 pool 704------------------------------envoy 1.01c sidecar 0.85cwrk hottest 0.67c mean 0.25c------------------------------mean request 5.31 ms before 2.81 envoy 0.91 sidecar 1.14 worker 0.46 resume 1.09 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 49.0 pool 704------------------------------envoy 0.96c sidecar 0.91cwrk hottest 0.70c mean 0.23c------------------------------mean request 2.93 ms before 0.99 envoy 0.68 sidecar 0.83 worker 0.43 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4491 /sachieved 4491 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 51.0 pool 704------------------------------envoy 1.03c sidecar 0.96cwrk hottest 0.82c mean 0.26c------------------------------mean request 3.06 ms before 1.08 envoy 0.65 sidecar 0.82 worker 0.51 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 44.0 pool 704------------------------------envoy 1.16c sidecar 1.08cwrk hottest 0.84c mean 0.28c------------------------------mean request 3.11 ms before 1.07 envoy 0.61 sidecar 0.82 worker 0.61 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 53.0 pool 704------------------------------envoy 1.08c sidecar 1.09cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.39 ms before 1.16 envoy 0.62 sidecar 0.82 worker 0.79 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5362 /sachieved 5362 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 154 pool 704------------------------------envoy 1.24c sidecar 1.13cwrk hottest 0.83c mean 0.31c------------------------------mean request 3.23 ms before 1.08 envoy 0.64 sidecar 0.84 worker 0.67 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 5.1 msin-flight 1867 pool 1881------------------------------envoy 1.32c sidecar 1.27cwrk hottest 0.83c mean 0.37c------------------------------mean request 7.34 ms before 1.36 envoy 1.23 sidecar 2.79 worker 1.96 resume 2.75 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 4.9 msin-flight 100 pool 1881------------------------------envoy 1.46c sidecar 1.29cwrk hottest 0.84c mean 0.37c------------------------------mean request 3.25 ms before 1.11 envoy 0.67 sidecar 0.83 worker 0.64 resume 0.79 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6758 /sachieved 6758 /s (100.0% ok)p50 3.4 ms p95 8.2 msin-flight 103 pool 1881------------------------------envoy 1.60c sidecar 1.40cwrk hottest 0.92c mean 0.40c------------------------------mean request 4.09 ms before 1.55 envoy 0.79 sidecar 0.85 worker 0.89 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 106 pool 1881------------------------------envoy 1.75c sidecar 1.44cwrk hottest 0.85c mean 0.44c------------------------------mean request 3.61 ms before 1.36 envoy 0.70 sidecar 0.86 worker 0.69 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7116 /sachieved 7116 /s (100.0% ok)p50 3.4 ms p95 14.4 msin-flight 1611 pool 1881------------------------------envoy 1.69c sidecar 1.43cwrk hottest 0.90c mean 0.42c------------------------------mean request 8.76 ms before 2.25 envoy 1.03 sidecar 3.35 worker 2.13 resume 3.01 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 4.6 ms p95 16.4 msin-flight 150 pool 1881------------------------------envoy 1.74c sidecar 1.43cwrk hottest 0.97c mean 0.44c------------------------------mean request 6.39 ms before 1.79 envoy 1.44 sidecar 0.89 worker 2.27 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 4.3 ms p95 18.2 msin-flight 2220 pool 2252------------------------------envoy 1.79c sidecar 1.51cwrk hottest 0.95c mean 0.44c------------------------------mean request 11.66 ms before 5.04 envoy 1.00 sidecar 2.95 worker 2.67 resume 2.91 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8001 /s (100.0% ok)p50 3.0 ms p95 5.7 msin-flight 210 pool 2252------------------------------envoy 1.94c sidecar 1.65cwrk hottest 0.73c mean 0.51c------------------------------mean request 3.54 ms before 1.18 envoy 0.68 sidecar 0.98 worker 0.70 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8253 /sachieved 8253 /s (100.0% ok)p50 3.2 ms p95 6.4 msin-flight 219 pool 2252------------------------------envoy 2.10c sidecar 1.69cwrk hottest 0.80c mean 0.54c------------------------------mean request 3.71 ms before 1.28 envoy 0.69 sidecar 0.96 worker 0.78 resume 0.91 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.2 ms p95 8.4 msin-flight 727 pool 2252------------------------------envoy 2.11c sidecar 1.81cwrk hottest 0.80c mean 0.57c------------------------------mean request 4.39 ms before 1.55 envoy 0.74 sidecar 1.21 worker 0.89 resume 1.16 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.3 ms p95 7.8 msin-flight 532 pool 2252------------------------------envoy 2.22c sidecar 1.79cwrk hottest 0.85c mean 0.55c------------------------------mean request 4.05 ms before 1.60 envoy 0.53 sidecar 1.03 worker 0.89 resume 0.98 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9338 /sachieved 9332 /s (100.0% ok)p50 3.6 ms p95 18.8 msin-flight 225 pool 2252------------------------------envoy 2.34c sidecar 1.75cwrk hottest 0.92c mean 0.58c------------------------------mean request 6.36 ms before 3.08 envoy 1.04 sidecar 0.96 worker 1.28 resume 0.92 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 4.0 ms p95 19.9 msin-flight 1033 pool 2252------------------------------envoy 2.28c sidecar 1.90cwrk hottest 0.93c mean 0.61c------------------------------mean request 6.90 ms before 3.21 envoy 1.03 sidecar 1.00 worker 1.67 resume 0.95 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (98.9% ok)p50 3.7 ms p95 78.4 msin-flight 2933 pool 3285------------------------------envoy 2.52c sidecar 1.90cwrk hottest 0.85c mean 0.63c------------------------------mean request 17.32 ms before 4.47 envoy 3.20 sidecar 4.59 worker 5.06 resume 4.58 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 3.7 ms p95 10.9 msin-flight 212 pool 3285------------------------------envoy 2.39c sidecar 1.90cwrk hottest 0.87c mean 0.60c------------------------------mean request 4.75 ms before 1.96 envoy 0.74 sidecar 0.90 worker 1.16 resume 0.86 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 5.3 ms p95 305.1 msin-flight 5540 pool 6436------------------------------envoy 2.64c sidecar 1.87cwrk hottest 0.80c mean 0.66c------------------------------mean request 52.23 ms before 29.39 envoy 8.65 sidecar 1.38 worker 12.81 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.2 ms p95 159.1 msin-flight 2403 pool 6436------------------------------envoy 2.73c sidecar 2.01cwrk hottest 0.92c mean 0.68c------------------------------mean request 30.18 ms before 18.11 envoy 4.11 sidecar 1.36 worker 6.59 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.0 ms p95 16.4 msin-flight 659 pool 6436------------------------------envoy 2.80c sidecar 2.07cwrk hottest 0.92c mean 0.70c------------------------------mean request 6.04 ms before 1.92 envoy 1.23 sidecar 0.91 worker 1.98 resume 0.86 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.3 ms p95 14.0 msin-flight 324 pool 6436------------------------------envoy 3.03c sidecar 2.20cwrk hottest 0.91c mean 0.72c------------------------------mean request 5.81 ms before 2.37 envoy 1.00 sidecar 0.88 worker 1.57 resume 0.83 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.1 ms p95 12.2 msin-flight 350 pool 6436------------------------------envoy 3.03c sidecar 2.26cwrk hottest 0.90c mean 0.76c------------------------------mean request 5.46 ms before 2.19 envoy 0.94 sidecar 0.85 worker 1.48 resume 0.80 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.5 ms p95 15.3 msin-flight 543 pool 6436------------------------------envoy 3.18c sidecar 2.31cwrk hottest 0.86c mean 0.80c------------------------------mean request 6.22 ms before 2.76 envoy 0.94 sidecar 0.96 worker 1.55 resume 0.92 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.9 ms p95 17.1 msin-flight 550 pool 6436------------------------------envoy 3.24c sidecar 2.39cwrk hottest 0.90c mean 0.81c------------------------------mean request 6.87 ms before 2.78 envoy 1.15 sidecar 0.99 worker 1.95 resume 0.95 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.8 ms p95 16.2 msin-flight 2118 pool 6436------------------------------envoy 3.26c sidecar 2.39cwrk hottest 0.87c mean 0.82c------------------------------mean request 6.61 ms before 2.56 envoy 1.24 sidecar 0.90 worker 1.91 resume 0.85 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 5.8 ms p95 29.9 msin-flight 2410 pool 2587------------------------------envoy 3.25c sidecar 2.40cwrk hottest 0.89c mean 0.82c------------------------------mean request 12.02 ms before 3.81 envoy 2.92 sidecar 0.93 worker 4.35 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.1 ms p95 51.4 msin-flight 2423 pool 2714------------------------------envoy 3.39c sidecar 2.50cwrk hottest 0.90c mean 0.83c------------------------------mean request 13.65 ms before 3.82 envoy 2.87 sidecar 1.93 worker 5.04 resume 1.88 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 6.9 ms p95 18.6 msin-flight 492 pool 2714------------------------------envoy 3.44c sidecar 2.41cwrk hottest 0.90c mean 0.84c------------------------------mean request 8.48 ms before 2.88 envoy 1.74 sidecar 0.94 worker 2.92 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.4 ms p95 25.4 msin-flight 511 pool 2714------------------------------envoy 3.41c sidecar 2.45cwrk hottest 0.89c mean 0.85c------------------------------mean request 10.12 ms before 3.62 envoy 2.03 sidecar 0.97 worker 3.50 resume 0.93 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 10.6 ms p95 28.3 msin-flight 414 pool 2714------------------------------envoy 3.45c sidecar 2.51cwrk hottest 0.90c mean 0.86c------------------------------mean request 12.62 ms before 4.68 envoy 2.54 sidecar 1.00 worker 4.41 resume 0.95 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 9.8 ms p95 44.5 msin-flight 2294 pool 2714------------------------------envoy 3.47c sidecar 2.52cwrk hottest 0.90c mean 0.86c------------------------------mean request 15.59 ms before 4.75 envoy 3.45 sidecar 1.63 worker 5.76 resume 1.58 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (99.6% ok)p50 16.1 ms p95 96.4 msin-flight 3565 pool 3690------------------------------envoy 3.43c sidecar 2.49cwrk hottest 0.89c mean 0.87c------------------------------mean request 27.16 ms before 10.58 envoy 5.05 sidecar 2.46 worker 9.07 resume 2.41 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 21.6 ms p95 77.4 msin-flight 1131 pool 3690------------------------------envoy 3.52c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 28.65 ms before 9.47 envoy 6.52 sidecar 1.23 worker 11.44 resume 1.18 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 20.3 ms p95 60.1 msin-flight 1072 pool 3690------------------------------envoy 3.53c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 24.95 ms before 10.84 envoy 4.66 sidecar 1.15 worker 8.30 resume 1.10 (control plane) +rung 16 17k qps setpoint [warmup]------------------------------offered 17k /sachieved 16k /s (100.0% ok)p50 40.0 ms p95 934.2 msin-flight 9399 pool 11k------------------------------envoy 3.51c sidecar 2.43cwrk hottest 0.90c mean 0.88c------------------------------mean request 224.00 ms before -136.73 envoy 277.87 sidecar 2.29 worker 80.57 resume 2.17 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.8% ok)p50 137.8 ms p95 1400.4 msin-flight 13k pool 14k------------------------------envoy 3.49c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 617.96 ms before -445.83 envoy 943.82 sidecar 2.69 worker 117.28 resume 2.51 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.9% ok)p50 88.9 ms p95 1634.3 msin-flight 16k pool 17k------------------------------envoy 3.53c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 747.00 ms before -632.55 envoy 1249.91 sidecar 2.94 worker 126.70 resume 2.76 (control plane) +rung 17 18k qps setpoint [warmup]------------------------------offered 18k /sachieved 16k /s (95.5% ok)p50 846.1 ms p95 2730.7 msin-flight 36k pool 35k------------------------------envoy 3.48c sidecar 2.37cwrk hottest 0.89c mean 0.88c------------------------------mean request 1164.59 ms before -711.52 envoy 1744.49 sidecar 4.67 worker 126.95 resume 3.64 (control plane) +rung -1 0 qps setpoint------------------------------offered 267 /sachieved 267 /s (100.0% ok)p50 2.9 ms p95 5.8 msin-flight 20.0 pool 23.0------------------------------envoy 0.08c sidecar 0.09cwrk hottest 0.13c mean 0.07c------------------------------mean request 3.39 ms before 0.95 envoy 0.73 sidecar 0.79 worker 0.92 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.5 msin-flight 5.0 pool 23.0------------------------------envoy 0.11c sidecar 0.26cwrk hottest 0.11c mean 0.06c------------------------------mean request 2.88 ms before 1.06 envoy 0.78 sidecar 0.79 worker 0.25 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.4 msin-flight 5.0 pool 23.0------------------------------envoy 0.26c sidecar 0.26cwrk hottest 0.12c mean 0.06c------------------------------mean request 2.86 ms before 1.03 envoy 0.80 sidecar 0.79 worker 0.23 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1412 /sachieved 1412 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 24.0------------------------------envoy 0.24c sidecar 0.34cwrk hottest 0.27c mean 0.09c------------------------------mean request 2.81 ms before 1.00 envoy 0.76 sidecar 0.80 worker 0.25 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.51c sidecar 0.49cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.92 envoy 0.85 sidecar 0.83 worker 0.18 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 10.0 pool 24.0------------------------------envoy 0.51c sidecar 0.51cwrk hottest 0.29c mean 0.13c------------------------------mean request 2.78 ms before 0.95 envoy 0.86 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2270 /sachieved 2270 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 12.0 pool 24.0------------------------------envoy 0.52c sidecar 0.52cwrk hottest 0.35c mean 0.15c------------------------------mean request 2.76 ms before 0.94 envoy 0.84 sidecar 0.82 worker 0.15 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 17.0 pool 24.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.35c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.16 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 14.0 pool 22.0------------------------------envoy 0.77c sidecar 0.74cwrk hottest 0.36c mean 0.19c------------------------------mean request 2.75 ms before 0.91 envoy 0.84 sidecar 0.83 worker 0.17 resume 0.79 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3170 /sachieved 3170 /s (100.0% ok)p50 2.7 ms p95 3.4 msin-flight 24.0 pool 25.0------------------------------envoy 0.81c sidecar 0.73cwrk hottest 0.35c mean 0.20c------------------------------mean request 2.75 ms before 0.87 envoy 0.84 sidecar 0.84 worker 0.21 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 4.2 msin-flight 532 pool 704------------------------------envoy 1.01c sidecar 0.85cwrk hottest 0.67c mean 0.25c------------------------------mean request 5.31 ms before 2.81 envoy 0.91 sidecar 1.14 worker 0.46 resume 1.09 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 49.0 pool 704------------------------------envoy 0.96c sidecar 0.91cwrk hottest 0.70c mean 0.23c------------------------------mean request 2.93 ms before 0.99 envoy 0.68 sidecar 0.83 worker 0.43 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4491 /sachieved 4491 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 51.0 pool 704------------------------------envoy 1.03c sidecar 0.96cwrk hottest 0.82c mean 0.26c------------------------------mean request 3.06 ms before 1.08 envoy 0.65 sidecar 0.82 worker 0.51 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 44.0 pool 704------------------------------envoy 1.16c sidecar 1.08cwrk hottest 0.84c mean 0.28c------------------------------mean request 3.11 ms before 1.07 envoy 0.61 sidecar 0.82 worker 0.61 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.2 ms p95 5.0 msin-flight 53.0 pool 704------------------------------envoy 1.08c sidecar 1.09cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.39 ms before 1.16 envoy 0.62 sidecar 0.82 worker 0.79 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5362 /sachieved 5362 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 154 pool 704------------------------------envoy 1.24c sidecar 1.13cwrk hottest 0.83c mean 0.31c------------------------------mean request 3.23 ms before 1.08 envoy 0.64 sidecar 0.84 worker 0.67 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 5.1 msin-flight 1867 pool 1881------------------------------envoy 1.32c sidecar 1.27cwrk hottest 0.83c mean 0.37c------------------------------mean request 7.34 ms before 1.36 envoy 1.23 sidecar 2.79 worker 1.96 resume 2.75 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 4.9 msin-flight 100 pool 1881------------------------------envoy 1.46c sidecar 1.29cwrk hottest 0.84c mean 0.37c------------------------------mean request 3.25 ms before 1.11 envoy 0.67 sidecar 0.83 worker 0.64 resume 0.79 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6758 /sachieved 6758 /s (100.0% ok)p50 3.4 ms p95 8.2 msin-flight 103 pool 1881------------------------------envoy 1.60c sidecar 1.40cwrk hottest 0.92c mean 0.40c------------------------------mean request 4.09 ms before 1.55 envoy 0.79 sidecar 0.85 worker 0.89 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 106 pool 1881------------------------------envoy 1.75c sidecar 1.44cwrk hottest 0.85c mean 0.44c------------------------------mean request 3.61 ms before 1.36 envoy 0.70 sidecar 0.86 worker 0.69 resume 0.81 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7116 /sachieved 7116 /s (100.0% ok)p50 3.4 ms p95 14.4 msin-flight 1611 pool 1881------------------------------envoy 1.69c sidecar 1.43cwrk hottest 0.90c mean 0.42c------------------------------mean request 8.76 ms before 2.25 envoy 1.03 sidecar 3.35 worker 2.13 resume 3.01 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 8000 /s (100.0% ok)p50 4.6 ms p95 16.4 msin-flight 150 pool 1881------------------------------envoy 1.74c sidecar 1.43cwrk hottest 0.97c mean 0.44c------------------------------mean request 6.39 ms before 1.79 envoy 1.44 sidecar 0.89 worker 2.27 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7999 /s (100.0% ok)p50 4.3 ms p95 18.2 msin-flight 2220 pool 2252------------------------------envoy 1.79c sidecar 1.51cwrk hottest 0.95c mean 0.44c------------------------------mean request 11.66 ms before 5.04 envoy 1.00 sidecar 2.95 worker 2.67 resume 2.91 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8001 /s (100.0% ok)p50 3.0 ms p95 5.7 msin-flight 210 pool 2252------------------------------envoy 1.94c sidecar 1.65cwrk hottest 0.73c mean 0.51c------------------------------mean request 3.54 ms before 1.18 envoy 0.68 sidecar 0.98 worker 0.70 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8253 /sachieved 8253 /s (100.0% ok)p50 3.2 ms p95 6.4 msin-flight 219 pool 2252------------------------------envoy 2.10c sidecar 1.69cwrk hottest 0.80c mean 0.54c------------------------------mean request 3.71 ms before 1.28 envoy 0.69 sidecar 0.96 worker 0.78 resume 0.91 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.2 ms p95 8.4 msin-flight 727 pool 2252------------------------------envoy 2.11c sidecar 1.81cwrk hottest 0.80c mean 0.57c------------------------------mean request 4.39 ms before 1.55 envoy 0.74 sidecar 1.21 worker 0.89 resume 1.16 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9000 /s (100.0% ok)p50 3.3 ms p95 7.8 msin-flight 532 pool 2252------------------------------envoy 2.22c sidecar 1.79cwrk hottest 0.85c mean 0.55c------------------------------mean request 4.05 ms before 1.60 envoy 0.53 sidecar 1.03 worker 0.89 resume 0.98 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9338 /sachieved 9332 /s (100.0% ok)p50 3.6 ms p95 18.8 msin-flight 225 pool 2252------------------------------envoy 2.34c sidecar 1.75cwrk hottest 0.92c mean 0.58c------------------------------mean request 6.36 ms before 3.08 envoy 1.04 sidecar 0.96 worker 1.28 resume 0.92 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 4.0 ms p95 19.9 msin-flight 1033 pool 2252------------------------------envoy 2.28c sidecar 1.90cwrk hottest 0.93c mean 0.61c------------------------------mean request 6.90 ms before 3.21 envoy 1.03 sidecar 1.00 worker 1.67 resume 0.95 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (98.9% ok)p50 3.7 ms p95 78.4 msin-flight 2933 pool 3285------------------------------envoy 2.52c sidecar 1.90cwrk hottest 0.85c mean 0.63c------------------------------mean request 17.32 ms before 4.47 envoy 3.20 sidecar 4.59 worker 5.06 resume 4.58 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 3.7 ms p95 10.9 msin-flight 212 pool 3285------------------------------envoy 2.39c sidecar 1.90cwrk hottest 0.87c mean 0.60c------------------------------mean request 4.75 ms before 1.96 envoy 0.74 sidecar 0.90 worker 1.16 resume 0.86 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 5.3 ms p95 305.1 msin-flight 5540 pool 6436------------------------------envoy 2.64c sidecar 1.87cwrk hottest 0.80c mean 0.66c------------------------------mean request 52.23 ms before 29.39 envoy 8.65 sidecar 1.38 worker 12.81 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.2 ms p95 159.1 msin-flight 2403 pool 6436------------------------------envoy 2.73c sidecar 2.01cwrk hottest 0.92c mean 0.68c------------------------------mean request 30.18 ms before 18.11 envoy 4.11 sidecar 1.36 worker 6.59 resume 1.31 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 4.0 ms p95 16.4 msin-flight 659 pool 6436------------------------------envoy 2.80c sidecar 2.07cwrk hottest 0.92c mean 0.70c------------------------------mean request 6.04 ms before 1.92 envoy 1.23 sidecar 0.91 worker 1.98 resume 0.86 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.3 ms p95 14.0 msin-flight 324 pool 6436------------------------------envoy 3.03c sidecar 2.20cwrk hottest 0.91c mean 0.72c------------------------------mean request 5.81 ms before 2.37 envoy 1.00 sidecar 0.88 worker 1.57 resume 0.83 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 4.1 ms p95 12.2 msin-flight 350 pool 6436------------------------------envoy 3.03c sidecar 2.26cwrk hottest 0.90c mean 0.76c------------------------------mean request 5.46 ms before 2.19 envoy 0.94 sidecar 0.85 worker 1.48 resume 0.80 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.5 ms p95 15.3 msin-flight 543 pool 6436------------------------------envoy 3.18c sidecar 2.31cwrk hottest 0.86c mean 0.80c------------------------------mean request 6.22 ms before 2.76 envoy 0.94 sidecar 0.96 worker 1.55 resume 0.92 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.9 ms p95 17.1 msin-flight 550 pool 6436------------------------------envoy 3.24c sidecar 2.39cwrk hottest 0.90c mean 0.81c------------------------------mean request 6.87 ms before 2.78 envoy 1.15 sidecar 0.99 worker 1.95 resume 0.95 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 4.8 ms p95 16.2 msin-flight 2118 pool 6436------------------------------envoy 3.26c sidecar 2.39cwrk hottest 0.87c mean 0.82c------------------------------mean request 6.61 ms before 2.56 envoy 1.24 sidecar 0.90 worker 1.91 resume 0.85 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 5.8 ms p95 29.9 msin-flight 2410 pool 2587------------------------------envoy 3.25c sidecar 2.40cwrk hottest 0.89c mean 0.82c------------------------------mean request 12.02 ms before 3.81 envoy 2.92 sidecar 0.93 worker 4.35 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.1 ms p95 51.4 msin-flight 2423 pool 2714------------------------------envoy 3.39c sidecar 2.50cwrk hottest 0.90c mean 0.83c------------------------------mean request 13.65 ms before 3.82 envoy 2.87 sidecar 1.93 worker 5.04 resume 1.88 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 6.9 ms p95 18.6 msin-flight 492 pool 2714------------------------------envoy 3.44c sidecar 2.41cwrk hottest 0.90c mean 0.84c------------------------------mean request 8.48 ms before 2.88 envoy 1.74 sidecar 0.94 worker 2.92 resume 0.89 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 7.4 ms p95 25.4 msin-flight 511 pool 2714------------------------------envoy 3.41c sidecar 2.45cwrk hottest 0.89c mean 0.85c------------------------------mean request 10.12 ms before 3.62 envoy 2.03 sidecar 0.97 worker 3.50 resume 0.93 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 10.6 ms p95 28.3 msin-flight 414 pool 2714------------------------------envoy 3.45c sidecar 2.51cwrk hottest 0.90c mean 0.86c------------------------------mean request 12.62 ms before 4.68 envoy 2.54 sidecar 1.00 worker 4.41 resume 0.95 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (100.0% ok)p50 9.8 ms p95 44.5 msin-flight 2294 pool 2714------------------------------envoy 3.47c sidecar 2.52cwrk hottest 0.90c mean 0.86c------------------------------mean request 15.59 ms before 4.75 envoy 3.45 sidecar 1.63 worker 5.76 resume 1.58 (control plane) +rung 14 15k qps setpoint------------------------------offered 15k /sachieved 15k /s (99.6% ok)p50 16.1 ms p95 96.4 msin-flight 3565 pool 3690------------------------------envoy 3.43c sidecar 2.49cwrk hottest 0.89c mean 0.87c------------------------------mean request 27.16 ms before 10.58 envoy 5.05 sidecar 2.46 worker 9.07 resume 2.41 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 21.6 ms p95 77.4 msin-flight 1131 pool 3690------------------------------envoy 3.52c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 28.65 ms before 9.47 envoy 6.52 sidecar 1.23 worker 11.44 resume 1.18 (control plane) +rung 15 16k qps setpoint------------------------------offered 16k /sachieved 16k /s (100.0% ok)p50 20.3 ms p95 60.1 msin-flight 1072 pool 3690------------------------------envoy 3.53c sidecar 2.43cwrk hottest 0.89c mean 0.88c------------------------------mean request 24.95 ms before 10.84 envoy 4.66 sidecar 1.15 worker 8.30 resume 1.10 (control plane) +rung 16 17k qps setpoint [warmup]------------------------------offered 17k /sachieved 16k /s (100.0% ok)p50 40.0 ms p95 934.2 msin-flight 9399 pool 11k------------------------------envoy 3.51c sidecar 2.43cwrk hottest 0.90c mean 0.88c------------------------------mean request 224.00 ms before -136.73 envoy 277.87 sidecar 2.29 worker 80.57 resume 2.17 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.8% ok)p50 137.8 ms p95 1400.4 msin-flight 13k pool 14k------------------------------envoy 3.49c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 617.96 ms before -445.83 envoy 943.82 sidecar 2.69 worker 117.28 resume 2.51 (control plane) +rung 16 17k qps setpoint------------------------------offered 17k /sachieved 17k /s (99.9% ok)p50 88.9 ms p95 1634.3 msin-flight 16k pool 17k------------------------------envoy 3.53c sidecar 2.20cwrk hottest 0.89c mean 0.88c------------------------------mean request 747.00 ms before -632.55 envoy 1249.91 sidecar 2.94 worker 126.70 resume 2.76 (control plane) +rung 17 18k qps setpoint [warmup]------------------------------offered 18k /sachieved 16k /s (95.5% ok)p50 846.1 ms p95 2730.7 msin-flight 36k pool 35k------------------------------envoy 3.48c sidecar 2.37cwrk hottest 0.89c mean 0.88c------------------------------mean request 1164.59 ms before -711.52 envoy 1744.49 sidecar 4.67 worker 126.95 resume 3.64 (control plane) +
+ + + +atenet-router capacity — 8c/2t +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 + +0 + +10k + +20k + +30k +requests / s + + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + +0 + +2.0 + +4.0 + +6.0 + +8.0 +cores + + + + + + + + +envoy CPU limit, 8 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + +0 + +500.0M + +1.0G + +1.5G +bytes + +50.0 + +100 + +150 + +200 + +250 + +300 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 146 /sachieved 146 /s (100.0% ok)p50 3.2 ms p95 12.1 msin-flight 33.0 pool 33.0------------------------------envoy 0.09c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.18cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.03 ms before 0.85 envoy 0.61 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.52cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.01 ms before 1.03 envoy 0.64 sidecar 0.80 worker 0.54 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 6.0 pool 33.0------------------------------envoy 0.51c sidecar 0.52cwrk hottest 0.37c mean 0.25c------------------------------mean request 3.00 ms before 1.03 envoy 0.69 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1603 /sachieved 1603 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 25.0 pool 33.0------------------------------envoy 0.76c sidecar 0.63cwrk hottest 0.48c mean 0.36c------------------------------mean request 3.02 ms before 1.05 envoy 0.70 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 137 pool 137------------------------------envoy 0.91c sidecar 0.89cwrk hottest 0.57c mean 0.45c------------------------------mean request 3.30 ms before 1.15 envoy 0.74 sidecar 0.83 worker 0.58 resume 0.79 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.3 ms p95 4.6 msin-flight 28.0 pool 137------------------------------envoy 0.80c sidecar 0.88cwrk hottest 0.80c mean 0.40c------------------------------mean request 3.40 ms before 1.32 envoy 0.54 sidecar 0.82 worker 0.73 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2635 /sachieved 2635 /s (100.0% ok)p50 3.2 ms p95 4.8 msin-flight 73.0 pool 137------------------------------envoy 1.15c sidecar 0.98cwrk hottest 0.79c mean 0.59c------------------------------mean request 3.46 ms before 1.29 envoy 0.58 sidecar 0.85 worker 0.74 resume 0.81 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 22.0 pool 137------------------------------envoy 1.25c sidecar 1.18cwrk hottest 0.78c mean 0.63c------------------------------mean request 3.24 ms before 1.18 envoy 0.56 sidecar 0.80 worker 0.69 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3134 /sachieved 3133 /s (100.0% ok)p50 3.2 ms p95 5.1 msin-flight 220 pool 220------------------------------envoy 1.33c sidecar 1.15cwrk hottest 0.83c mean 0.65c------------------------------mean request 3.73 ms before 1.44 envoy 0.64 sidecar 0.81 worker 0.84 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 110 pool 220------------------------------envoy 1.57c sidecar 1.33cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.61 ms before 1.35 envoy 0.53 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 34.0 pool 220------------------------------envoy 1.60c sidecar 1.43cwrk hottest 0.82c mean 0.80c------------------------------mean request 3.60 ms before 1.38 envoy 0.53 sidecar 0.80 worker 0.89 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 30.0 pool 220------------------------------envoy 1.59c sidecar 0.00cwrk hottest 0.83c mean 0.80c------------------------------mean request 3.56 ms before 1.35 envoy 0.50 sidecar 0.82 worker 0.90 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4679 /sachieved 4590 /s (100.0% ok)p50 5.4 ms p95 683.9 msin-flight 3836 pool 3836------------------------------envoy 1.76c sidecar 1.45cwrk hottest 0.93c mean 0.88c------------------------------mean request 149.66 ms before 135.26 envoy 6.45 sidecar 2.53 worker 5.41 resume 1.72 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5090 /s (100.0% ok)p50 6.0 ms p95 247.3 msin-flight 1400 pool 3836------------------------------envoy 1.85c sidecar 1.29cwrk hottest 0.95c mean 0.93c------------------------------mean request 40.36 ms before -74.10 envoy 89.82 sidecar 1.00 worker 23.64 resume 0.96 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.9 ms p95 17.2 msin-flight 378 pool 3836------------------------------envoy 1.87c sidecar 1.51cwrk hottest 0.95c mean 0.93c------------------------------mean request 7.49 ms before 3.12 envoy 1.25 sidecar 0.91 worker 2.21 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5970 /sachieved 5669 /s (99.9% ok)p50 250.2 ms p95 808.8 msin-flight 5712 pool 5721------------------------------envoy 1.92c sidecar 1.37cwrk hottest 0.96c mean 0.96c------------------------------mean request 336.74 ms before 164.56 envoy 119.57 sidecar 2.31 worker 50.31 resume 2.23 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5444 /s (99.4% ok)p50 1373.5 ms p95 2360.3 msin-flight 16k pool 16k------------------------------envoy 1.93c sidecar 1.06cwrk hottest 0.97c mean 0.96c------------------------------mean request 1574.13 ms before 195.98 envoy 1297.37 sidecar 2.90 worker 77.89 resume 2.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6368 /sachieved 6030 /s (75.6% ok)p50 3131.1 ms p95 5534.4 msin-flight 21k pool 21k------------------------------envoy 1.93c sidecar 1.01cwrk hottest 0.97c mean 0.96c------------------------------mean request 2869.22 ms before 138.77 envoy 2641.40 sidecar 3.01 worker 86.04 resume 2.90 (control plane) +rung -1 0 qps setpoint------------------------------offered 146 /sachieved 146 /s (100.0% ok)p50 3.2 ms p95 12.1 msin-flight 33.0 pool 33.0------------------------------envoy 0.09c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.18cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.03 ms before 0.85 envoy 0.61 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.52cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.01 ms before 1.03 envoy 0.64 sidecar 0.80 worker 0.54 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 6.0 pool 33.0------------------------------envoy 0.51c sidecar 0.52cwrk hottest 0.37c mean 0.25c------------------------------mean request 3.00 ms before 1.03 envoy 0.69 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1603 /sachieved 1603 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 25.0 pool 33.0------------------------------envoy 0.76c sidecar 0.63cwrk hottest 0.48c mean 0.36c------------------------------mean request 3.02 ms before 1.05 envoy 0.70 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 137 pool 137------------------------------envoy 0.91c sidecar 0.89cwrk hottest 0.57c mean 0.45c------------------------------mean request 3.30 ms before 1.15 envoy 0.74 sidecar 0.83 worker 0.58 resume 0.79 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.3 ms p95 4.6 msin-flight 28.0 pool 137------------------------------envoy 0.80c sidecar 0.88cwrk hottest 0.80c mean 0.40c------------------------------mean request 3.40 ms before 1.32 envoy 0.54 sidecar 0.82 worker 0.73 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2635 /sachieved 2635 /s (100.0% ok)p50 3.2 ms p95 4.8 msin-flight 73.0 pool 137------------------------------envoy 1.15c sidecar 0.98cwrk hottest 0.79c mean 0.59c------------------------------mean request 3.46 ms before 1.29 envoy 0.58 sidecar 0.85 worker 0.74 resume 0.81 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 22.0 pool 137------------------------------envoy 1.25c sidecar 1.18cwrk hottest 0.78c mean 0.63c------------------------------mean request 3.24 ms before 1.18 envoy 0.56 sidecar 0.80 worker 0.69 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3134 /sachieved 3133 /s (100.0% ok)p50 3.2 ms p95 5.1 msin-flight 220 pool 220------------------------------envoy 1.33c sidecar 1.15cwrk hottest 0.83c mean 0.65c------------------------------mean request 3.73 ms before 1.44 envoy 0.64 sidecar 0.81 worker 0.84 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 110 pool 220------------------------------envoy 1.57c sidecar 1.33cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.61 ms before 1.35 envoy 0.53 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 34.0 pool 220------------------------------envoy 1.60c sidecar 1.43cwrk hottest 0.82c mean 0.80c------------------------------mean request 3.60 ms before 1.38 envoy 0.53 sidecar 0.80 worker 0.89 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 30.0 pool 220------------------------------envoy 1.59c sidecar 0.00cwrk hottest 0.83c mean 0.80c------------------------------mean request 3.56 ms before 1.35 envoy 0.50 sidecar 0.82 worker 0.90 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4679 /sachieved 4590 /s (100.0% ok)p50 5.4 ms p95 683.9 msin-flight 3836 pool 3836------------------------------envoy 1.76c sidecar 1.45cwrk hottest 0.93c mean 0.88c------------------------------mean request 149.66 ms before 135.26 envoy 6.45 sidecar 2.53 worker 5.41 resume 1.72 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5090 /s (100.0% ok)p50 6.0 ms p95 247.3 msin-flight 1400 pool 3836------------------------------envoy 1.85c sidecar 1.29cwrk hottest 0.95c mean 0.93c------------------------------mean request 40.36 ms before -74.10 envoy 89.82 sidecar 1.00 worker 23.64 resume 0.96 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.9 ms p95 17.2 msin-flight 378 pool 3836------------------------------envoy 1.87c sidecar 1.51cwrk hottest 0.95c mean 0.93c------------------------------mean request 7.49 ms before 3.12 envoy 1.25 sidecar 0.91 worker 2.21 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5970 /sachieved 5669 /s (99.9% ok)p50 250.2 ms p95 808.8 msin-flight 5712 pool 5721------------------------------envoy 1.92c sidecar 1.37cwrk hottest 0.96c mean 0.96c------------------------------mean request 336.74 ms before 164.56 envoy 119.57 sidecar 2.31 worker 50.31 resume 2.23 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5444 /s (99.4% ok)p50 1373.5 ms p95 2360.3 msin-flight 16k pool 16k------------------------------envoy 1.93c sidecar 1.06cwrk hottest 0.97c mean 0.96c------------------------------mean request 1574.13 ms before 195.98 envoy 1297.37 sidecar 2.90 worker 77.89 resume 2.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6368 /sachieved 6030 /s (75.6% ok)p50 3131.1 ms p95 5534.4 msin-flight 21k pool 21k------------------------------envoy 1.93c sidecar 1.01cwrk hottest 0.97c mean 0.96c------------------------------mean request 2869.22 ms before 138.77 envoy 2641.40 sidecar 3.01 worker 86.04 resume 2.90 (control plane) +rung -1 0 qps setpoint------------------------------offered 146 /sachieved 146 /s (100.0% ok)p50 3.2 ms p95 12.1 msin-flight 33.0 pool 33.0------------------------------envoy 0.09c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.18cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.03 ms before 0.85 envoy 0.61 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.52cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.01 ms before 1.03 envoy 0.64 sidecar 0.80 worker 0.54 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 6.0 pool 33.0------------------------------envoy 0.51c sidecar 0.52cwrk hottest 0.37c mean 0.25c------------------------------mean request 3.00 ms before 1.03 envoy 0.69 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1603 /sachieved 1603 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 25.0 pool 33.0------------------------------envoy 0.76c sidecar 0.63cwrk hottest 0.48c mean 0.36c------------------------------mean request 3.02 ms before 1.05 envoy 0.70 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 137 pool 137------------------------------envoy 0.91c sidecar 0.89cwrk hottest 0.57c mean 0.45c------------------------------mean request 3.30 ms before 1.15 envoy 0.74 sidecar 0.83 worker 0.58 resume 0.79 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.3 ms p95 4.6 msin-flight 28.0 pool 137------------------------------envoy 0.80c sidecar 0.88cwrk hottest 0.80c mean 0.40c------------------------------mean request 3.40 ms before 1.32 envoy 0.54 sidecar 0.82 worker 0.73 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2635 /sachieved 2635 /s (100.0% ok)p50 3.2 ms p95 4.8 msin-flight 73.0 pool 137------------------------------envoy 1.15c sidecar 0.98cwrk hottest 0.79c mean 0.59c------------------------------mean request 3.46 ms before 1.29 envoy 0.58 sidecar 0.85 worker 0.74 resume 0.81 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 22.0 pool 137------------------------------envoy 1.25c sidecar 1.18cwrk hottest 0.78c mean 0.63c------------------------------mean request 3.24 ms before 1.18 envoy 0.56 sidecar 0.80 worker 0.69 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3134 /sachieved 3133 /s (100.0% ok)p50 3.2 ms p95 5.1 msin-flight 220 pool 220------------------------------envoy 1.33c sidecar 1.15cwrk hottest 0.83c mean 0.65c------------------------------mean request 3.73 ms before 1.44 envoy 0.64 sidecar 0.81 worker 0.84 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 110 pool 220------------------------------envoy 1.57c sidecar 1.33cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.61 ms before 1.35 envoy 0.53 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 34.0 pool 220------------------------------envoy 1.60c sidecar 1.43cwrk hottest 0.82c mean 0.80c------------------------------mean request 3.60 ms before 1.38 envoy 0.53 sidecar 0.80 worker 0.89 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 30.0 pool 220------------------------------envoy 1.59c sidecar 0.00cwrk hottest 0.83c mean 0.80c------------------------------mean request 3.56 ms before 1.35 envoy 0.50 sidecar 0.82 worker 0.90 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4679 /sachieved 4590 /s (100.0% ok)p50 5.4 ms p95 683.9 msin-flight 3836 pool 3836------------------------------envoy 1.76c sidecar 1.45cwrk hottest 0.93c mean 0.88c------------------------------mean request 149.66 ms before 135.26 envoy 6.45 sidecar 2.53 worker 5.41 resume 1.72 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5090 /s (100.0% ok)p50 6.0 ms p95 247.3 msin-flight 1400 pool 3836------------------------------envoy 1.85c sidecar 1.29cwrk hottest 0.95c mean 0.93c------------------------------mean request 40.36 ms before -74.10 envoy 89.82 sidecar 1.00 worker 23.64 resume 0.96 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.9 ms p95 17.2 msin-flight 378 pool 3836------------------------------envoy 1.87c sidecar 1.51cwrk hottest 0.95c mean 0.93c------------------------------mean request 7.49 ms before 3.12 envoy 1.25 sidecar 0.91 worker 2.21 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5970 /sachieved 5669 /s (99.9% ok)p50 250.2 ms p95 808.8 msin-flight 5712 pool 5721------------------------------envoy 1.92c sidecar 1.37cwrk hottest 0.96c mean 0.96c------------------------------mean request 336.74 ms before 164.56 envoy 119.57 sidecar 2.31 worker 50.31 resume 2.23 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5444 /s (99.4% ok)p50 1373.5 ms p95 2360.3 msin-flight 16k pool 16k------------------------------envoy 1.93c sidecar 1.06cwrk hottest 0.97c mean 0.96c------------------------------mean request 1574.13 ms before 195.98 envoy 1297.37 sidecar 2.90 worker 77.89 resume 2.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6368 /sachieved 6030 /s (75.6% ok)p50 3131.1 ms p95 5534.4 msin-flight 21k pool 21k------------------------------envoy 1.93c sidecar 1.01cwrk hottest 0.97c mean 0.96c------------------------------mean request 2869.22 ms before 138.77 envoy 2641.40 sidecar 3.01 worker 86.04 resume 2.90 (control plane) +rung -1 0 qps setpoint------------------------------offered 146 /sachieved 146 /s (100.0% ok)p50 3.2 ms p95 12.1 msin-flight 33.0 pool 33.0------------------------------envoy 0.09c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.18cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.03 ms before 0.85 envoy 0.61 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.52cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.01 ms before 1.03 envoy 0.64 sidecar 0.80 worker 0.54 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 6.0 pool 33.0------------------------------envoy 0.51c sidecar 0.52cwrk hottest 0.37c mean 0.25c------------------------------mean request 3.00 ms before 1.03 envoy 0.69 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1603 /sachieved 1603 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 25.0 pool 33.0------------------------------envoy 0.76c sidecar 0.63cwrk hottest 0.48c mean 0.36c------------------------------mean request 3.02 ms before 1.05 envoy 0.70 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 137 pool 137------------------------------envoy 0.91c sidecar 0.89cwrk hottest 0.57c mean 0.45c------------------------------mean request 3.30 ms before 1.15 envoy 0.74 sidecar 0.83 worker 0.58 resume 0.79 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.3 ms p95 4.6 msin-flight 28.0 pool 137------------------------------envoy 0.80c sidecar 0.88cwrk hottest 0.80c mean 0.40c------------------------------mean request 3.40 ms before 1.32 envoy 0.54 sidecar 0.82 worker 0.73 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2635 /sachieved 2635 /s (100.0% ok)p50 3.2 ms p95 4.8 msin-flight 73.0 pool 137------------------------------envoy 1.15c sidecar 0.98cwrk hottest 0.79c mean 0.59c------------------------------mean request 3.46 ms before 1.29 envoy 0.58 sidecar 0.85 worker 0.74 resume 0.81 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 22.0 pool 137------------------------------envoy 1.25c sidecar 1.18cwrk hottest 0.78c mean 0.63c------------------------------mean request 3.24 ms before 1.18 envoy 0.56 sidecar 0.80 worker 0.69 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3134 /sachieved 3133 /s (100.0% ok)p50 3.2 ms p95 5.1 msin-flight 220 pool 220------------------------------envoy 1.33c sidecar 1.15cwrk hottest 0.83c mean 0.65c------------------------------mean request 3.73 ms before 1.44 envoy 0.64 sidecar 0.81 worker 0.84 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 110 pool 220------------------------------envoy 1.57c sidecar 1.33cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.61 ms before 1.35 envoy 0.53 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 34.0 pool 220------------------------------envoy 1.60c sidecar 1.43cwrk hottest 0.82c mean 0.80c------------------------------mean request 3.60 ms before 1.38 envoy 0.53 sidecar 0.80 worker 0.89 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 30.0 pool 220------------------------------envoy 1.59c sidecar 0.00cwrk hottest 0.83c mean 0.80c------------------------------mean request 3.56 ms before 1.35 envoy 0.50 sidecar 0.82 worker 0.90 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4679 /sachieved 4590 /s (100.0% ok)p50 5.4 ms p95 683.9 msin-flight 3836 pool 3836------------------------------envoy 1.76c sidecar 1.45cwrk hottest 0.93c mean 0.88c------------------------------mean request 149.66 ms before 135.26 envoy 6.45 sidecar 2.53 worker 5.41 resume 1.72 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5090 /s (100.0% ok)p50 6.0 ms p95 247.3 msin-flight 1400 pool 3836------------------------------envoy 1.85c sidecar 1.29cwrk hottest 0.95c mean 0.93c------------------------------mean request 40.36 ms before -74.10 envoy 89.82 sidecar 1.00 worker 23.64 resume 0.96 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.9 ms p95 17.2 msin-flight 378 pool 3836------------------------------envoy 1.87c sidecar 1.51cwrk hottest 0.95c mean 0.93c------------------------------mean request 7.49 ms before 3.12 envoy 1.25 sidecar 0.91 worker 2.21 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5970 /sachieved 5669 /s (99.9% ok)p50 250.2 ms p95 808.8 msin-flight 5712 pool 5721------------------------------envoy 1.92c sidecar 1.37cwrk hottest 0.96c mean 0.96c------------------------------mean request 336.74 ms before 164.56 envoy 119.57 sidecar 2.31 worker 50.31 resume 2.23 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5444 /s (99.4% ok)p50 1373.5 ms p95 2360.3 msin-flight 16k pool 16k------------------------------envoy 1.93c sidecar 1.06cwrk hottest 0.97c mean 0.96c------------------------------mean request 1574.13 ms before 195.98 envoy 1297.37 sidecar 2.90 worker 77.89 resume 2.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6368 /sachieved 6030 /s (75.6% ok)p50 3131.1 ms p95 5534.4 msin-flight 21k pool 21k------------------------------envoy 1.93c sidecar 1.01cwrk hottest 0.97c mean 0.96c------------------------------mean request 2869.22 ms before 138.77 envoy 2641.40 sidecar 3.01 worker 86.04 resume 2.90 (control plane) +rung -1 0 qps setpoint------------------------------offered 146 /sachieved 146 /s (100.0% ok)p50 3.2 ms p95 12.1 msin-flight 33.0 pool 33.0------------------------------envoy 0.09c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.18cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.03 ms before 0.85 envoy 0.61 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.52cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.01 ms before 1.03 envoy 0.64 sidecar 0.80 worker 0.54 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 6.0 pool 33.0------------------------------envoy 0.51c sidecar 0.52cwrk hottest 0.37c mean 0.25c------------------------------mean request 3.00 ms before 1.03 envoy 0.69 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1603 /sachieved 1603 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 25.0 pool 33.0------------------------------envoy 0.76c sidecar 0.63cwrk hottest 0.48c mean 0.36c------------------------------mean request 3.02 ms before 1.05 envoy 0.70 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 137 pool 137------------------------------envoy 0.91c sidecar 0.89cwrk hottest 0.57c mean 0.45c------------------------------mean request 3.30 ms before 1.15 envoy 0.74 sidecar 0.83 worker 0.58 resume 0.79 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.3 ms p95 4.6 msin-flight 28.0 pool 137------------------------------envoy 0.80c sidecar 0.88cwrk hottest 0.80c mean 0.40c------------------------------mean request 3.40 ms before 1.32 envoy 0.54 sidecar 0.82 worker 0.73 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2635 /sachieved 2635 /s (100.0% ok)p50 3.2 ms p95 4.8 msin-flight 73.0 pool 137------------------------------envoy 1.15c sidecar 0.98cwrk hottest 0.79c mean 0.59c------------------------------mean request 3.46 ms before 1.29 envoy 0.58 sidecar 0.85 worker 0.74 resume 0.81 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 22.0 pool 137------------------------------envoy 1.25c sidecar 1.18cwrk hottest 0.78c mean 0.63c------------------------------mean request 3.24 ms before 1.18 envoy 0.56 sidecar 0.80 worker 0.69 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3134 /sachieved 3133 /s (100.0% ok)p50 3.2 ms p95 5.1 msin-flight 220 pool 220------------------------------envoy 1.33c sidecar 1.15cwrk hottest 0.83c mean 0.65c------------------------------mean request 3.73 ms before 1.44 envoy 0.64 sidecar 0.81 worker 0.84 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 110 pool 220------------------------------envoy 1.57c sidecar 1.33cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.61 ms before 1.35 envoy 0.53 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 34.0 pool 220------------------------------envoy 1.60c sidecar 1.43cwrk hottest 0.82c mean 0.80c------------------------------mean request 3.60 ms before 1.38 envoy 0.53 sidecar 0.80 worker 0.89 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 30.0 pool 220------------------------------envoy 1.59c sidecar 0.00cwrk hottest 0.83c mean 0.80c------------------------------mean request 3.56 ms before 1.35 envoy 0.50 sidecar 0.82 worker 0.90 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4679 /sachieved 4590 /s (100.0% ok)p50 5.4 ms p95 683.9 msin-flight 3836 pool 3836------------------------------envoy 1.76c sidecar 1.45cwrk hottest 0.93c mean 0.88c------------------------------mean request 149.66 ms before 135.26 envoy 6.45 sidecar 2.53 worker 5.41 resume 1.72 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5090 /s (100.0% ok)p50 6.0 ms p95 247.3 msin-flight 1400 pool 3836------------------------------envoy 1.85c sidecar 1.29cwrk hottest 0.95c mean 0.93c------------------------------mean request 40.36 ms before -74.10 envoy 89.82 sidecar 1.00 worker 23.64 resume 0.96 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.9 ms p95 17.2 msin-flight 378 pool 3836------------------------------envoy 1.87c sidecar 1.51cwrk hottest 0.95c mean 0.93c------------------------------mean request 7.49 ms before 3.12 envoy 1.25 sidecar 0.91 worker 2.21 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5970 /sachieved 5669 /s (99.9% ok)p50 250.2 ms p95 808.8 msin-flight 5712 pool 5721------------------------------envoy 1.92c sidecar 1.37cwrk hottest 0.96c mean 0.96c------------------------------mean request 336.74 ms before 164.56 envoy 119.57 sidecar 2.31 worker 50.31 resume 2.23 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5444 /s (99.4% ok)p50 1373.5 ms p95 2360.3 msin-flight 16k pool 16k------------------------------envoy 1.93c sidecar 1.06cwrk hottest 0.97c mean 0.96c------------------------------mean request 1574.13 ms before 195.98 envoy 1297.37 sidecar 2.90 worker 77.89 resume 2.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6368 /sachieved 6030 /s (75.6% ok)p50 3131.1 ms p95 5534.4 msin-flight 21k pool 21k------------------------------envoy 1.93c sidecar 1.01cwrk hottest 0.97c mean 0.96c------------------------------mean request 2869.22 ms before 138.77 envoy 2641.40 sidecar 3.01 worker 86.04 resume 2.90 (control plane) +rung -1 0 qps setpoint------------------------------offered 146 /sachieved 146 /s (100.0% ok)p50 3.2 ms p95 12.1 msin-flight 33.0 pool 33.0------------------------------envoy 0.09c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.18cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.03 ms before 0.85 envoy 0.61 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 33.0------------------------------envoy 0.48c sidecar 0.52cwrk hottest 0.38c mean 0.24c------------------------------mean request 3.01 ms before 1.03 envoy 0.64 sidecar 0.80 worker 0.54 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 6.0 pool 33.0------------------------------envoy 0.51c sidecar 0.52cwrk hottest 0.37c mean 0.25c------------------------------mean request 3.00 ms before 1.03 envoy 0.69 sidecar 0.80 worker 0.47 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1603 /sachieved 1603 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 25.0 pool 33.0------------------------------envoy 0.76c sidecar 0.63cwrk hottest 0.48c mean 0.36c------------------------------mean request 3.02 ms before 1.05 envoy 0.70 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 137 pool 137------------------------------envoy 0.91c sidecar 0.89cwrk hottest 0.57c mean 0.45c------------------------------mean request 3.30 ms before 1.15 envoy 0.74 sidecar 0.83 worker 0.58 resume 0.79 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.3 ms p95 4.6 msin-flight 28.0 pool 137------------------------------envoy 0.80c sidecar 0.88cwrk hottest 0.80c mean 0.40c------------------------------mean request 3.40 ms before 1.32 envoy 0.54 sidecar 0.82 worker 0.73 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2635 /sachieved 2635 /s (100.0% ok)p50 3.2 ms p95 4.8 msin-flight 73.0 pool 137------------------------------envoy 1.15c sidecar 0.98cwrk hottest 0.79c mean 0.59c------------------------------mean request 3.46 ms before 1.29 envoy 0.58 sidecar 0.85 worker 0.74 resume 0.81 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 22.0 pool 137------------------------------envoy 1.25c sidecar 1.18cwrk hottest 0.78c mean 0.63c------------------------------mean request 3.24 ms before 1.18 envoy 0.56 sidecar 0.80 worker 0.69 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3134 /sachieved 3133 /s (100.0% ok)p50 3.2 ms p95 5.1 msin-flight 220 pool 220------------------------------envoy 1.33c sidecar 1.15cwrk hottest 0.83c mean 0.65c------------------------------mean request 3.73 ms before 1.44 envoy 0.64 sidecar 0.81 worker 0.84 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 110 pool 220------------------------------envoy 1.57c sidecar 1.33cwrk hottest 0.85c mean 0.79c------------------------------mean request 3.61 ms before 1.35 envoy 0.53 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.1 msin-flight 34.0 pool 220------------------------------envoy 1.60c sidecar 1.43cwrk hottest 0.82c mean 0.80c------------------------------mean request 3.60 ms before 1.38 envoy 0.53 sidecar 0.80 worker 0.89 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 30.0 pool 220------------------------------envoy 1.59c sidecar 0.00cwrk hottest 0.83c mean 0.80c------------------------------mean request 3.56 ms before 1.35 envoy 0.50 sidecar 0.82 worker 0.90 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4679 /sachieved 4590 /s (100.0% ok)p50 5.4 ms p95 683.9 msin-flight 3836 pool 3836------------------------------envoy 1.76c sidecar 1.45cwrk hottest 0.93c mean 0.88c------------------------------mean request 149.66 ms before 135.26 envoy 6.45 sidecar 2.53 worker 5.41 resume 1.72 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5090 /s (100.0% ok)p50 6.0 ms p95 247.3 msin-flight 1400 pool 3836------------------------------envoy 1.85c sidecar 1.29cwrk hottest 0.95c mean 0.93c------------------------------mean request 40.36 ms before -74.10 envoy 89.82 sidecar 1.00 worker 23.64 resume 0.96 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 5.9 ms p95 17.2 msin-flight 378 pool 3836------------------------------envoy 1.87c sidecar 1.51cwrk hottest 0.95c mean 0.93c------------------------------mean request 7.49 ms before 3.12 envoy 1.25 sidecar 0.91 worker 2.21 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5970 /sachieved 5669 /s (99.9% ok)p50 250.2 ms p95 808.8 msin-flight 5712 pool 5721------------------------------envoy 1.92c sidecar 1.37cwrk hottest 0.96c mean 0.96c------------------------------mean request 336.74 ms before 164.56 envoy 119.57 sidecar 2.31 worker 50.31 resume 2.23 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5444 /s (99.4% ok)p50 1373.5 ms p95 2360.3 msin-flight 16k pool 16k------------------------------envoy 1.93c sidecar 1.06cwrk hottest 0.97c mean 0.96c------------------------------mean request 1574.13 ms before 195.98 envoy 1297.37 sidecar 2.90 worker 77.89 resume 2.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6368 /sachieved 6030 /s (75.6% ok)p50 3131.1 ms p95 5534.4 msin-flight 21k pool 21k------------------------------envoy 1.93c sidecar 1.01cwrk hottest 0.97c mean 0.96c------------------------------mean request 2869.22 ms before 138.77 envoy 2641.40 sidecar 3.01 worker 86.04 resume 2.90 (control plane) +
+ + + +atenet-router capacity — 8c/4t +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 +R9 + +0 + +10k + +20k + +30k +requests / s + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + + +0 + +2.0 + +4.0 + +6.0 + +8.0 +cores + + + + + + +envoy CPU limit, 8 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + + +0 + +500.0M + +1.0G + +1.5G +bytes + +100 + +200 + +300 + +400 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 191 /sachieved 191 /s (100.0% ok)p50 3.1 ms p95 9.0 msin-flight 35.0 pool 34.0------------------------------envoy 0.13c sidecar 0.01c------------------------------mean request 4.05 ms before 1.39 envoy 0.67 sidecar 0.79 worker 1.21 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.50c sidecar 0.48cwrk hottest 0.36c mean 0.13c------------------------------mean request 3.02 ms before 1.04 envoy 0.63 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 34.0------------------------------envoy 0.52c sidecar 0.50cwrk hottest 0.32c mean 0.13c------------------------------mean request 2.96 ms before 1.01 envoy 0.71 sidecar 0.79 worker 0.45 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1299 /sachieved 1299 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 571 pool 571------------------------------envoy 0.60c sidecar 0.49cwrk hottest 0.72c mean 0.16c------------------------------mean request 13.00 ms before 9.33 envoy 1.31 sidecar 0.90 worker 1.46 resume 0.86 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 24.0 pool 571------------------------------envoy 0.88c sidecar 0.79cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.23 ms before 1.19 envoy 0.52 sidecar 0.81 worker 0.72 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 21.0 pool 571------------------------------envoy 0.86c sidecar 0.82cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.08 ms before 1.12 envoy 0.57 sidecar 0.80 worker 0.59 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2756 /sachieved 2754 /s (100.0% ok)p50 3.6 ms p95 7.5 msin-flight 55.0 pool 571------------------------------envoy 1.14c sidecar 0.92cwrk hottest 0.93c mean 0.29c------------------------------mean request 4.09 ms before 1.69 envoy 0.67 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3002 /s (100.0% ok)p50 3.7 ms p95 7.8 msin-flight 92.0 pool 571------------------------------envoy 1.18c sidecar 1.05cwrk hottest 0.91c mean 0.29c------------------------------mean request 4.35 ms before 1.70 envoy 0.72 sidecar 0.80 worker 1.13 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.7 ms p95 7.3 msin-flight 70.0 pool 571------------------------------envoy 1.21c sidecar 1.04cwrk hottest 0.91c mean 0.30c------------------------------mean request 4.14 ms before 1.64 envoy 0.66 sidecar 0.81 worker 1.03 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3964 /sachieved 3962 /s (100.0% ok)p50 5.9 ms p95 22.2 msin-flight 180 pool 571------------------------------envoy 1.53c sidecar 1.12cwrk hottest 0.97c mean 0.39c------------------------------mean request 8.71 ms before 3.66 envoy 1.59 sidecar 0.84 worker 2.63 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 4.6 ms p95 14.8 msin-flight 97.0 pool 180------------------------------envoy 1.54c sidecar 1.25cwrk hottest 0.96c mean 0.38c------------------------------mean request 6.02 ms before 2.20 envoy 1.13 sidecar 0.83 worker 1.85 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4273 /sachieved 4271 /s (100.0% ok)p50 3.6 ms p95 16.9 msin-flight 188 pool 188------------------------------envoy 1.81c sidecar 1.36cwrk hottest 0.95c mean 0.45c------------------------------mean request 5.91 ms before 2.51 envoy 1.00 sidecar 0.88 worker 1.52 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5002 /s (100.0% ok)p50 4.4 ms p95 18.1 msin-flight 84.0 pool 188------------------------------envoy 2.07c sidecar 1.60cwrk hottest 0.95c mean 0.52c------------------------------mean request 6.76 ms before 2.55 envoy 1.31 sidecar 0.84 worker 2.06 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.7 ms p95 9.5 msin-flight 72.0 pool 188------------------------------envoy 2.14c sidecar 0.00cwrk hottest 0.94c mean 0.53c------------------------------mean request 4.66 ms before 1.90 envoy 0.84 sidecar 0.82 worker 1.10 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5152 /sachieved 5149 /s (100.0% ok)p50 4.6 ms p95 28.9 msin-flight 159 pool 188------------------------------envoy 2.09c sidecar 1.60cwrk hottest 0.95c mean 0.51c------------------------------mean request 8.27 ms before 4.41 envoy 1.17 sidecar 0.87 worker 1.82 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 3.8 ms p95 252.5 msin-flight 2427 pool 2427------------------------------envoy 2.69c sidecar 1.62cwrk hottest 0.87c mean 0.66c------------------------------mean request 34.41 ms before 14.18 envoy 8.08 sidecar 1.05 worker 11.11 resume 1.00 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.5 ms p95 6.1 msin-flight 74.0 pool 2427------------------------------envoy 2.76c sidecar 1.88cwrk hottest 0.84c mean 0.69c------------------------------mean request 3.83 ms before 1.43 envoy 0.65 sidecar 0.84 worker 0.91 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6340 /sachieved 6340 /s (100.0% ok)p50 3.6 ms p95 7.4 msin-flight 83.0 pool 2427------------------------------envoy 2.85c sidecar 1.93cwrk hottest 0.86c mean 0.72c------------------------------mean request 4.04 ms before 1.61 envoy 0.66 sidecar 0.82 worker 0.95 resume 0.79 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7001 /s (100.0% ok)p50 4.1 ms p95 9.7 msin-flight 223 pool 2427------------------------------envoy 3.06c sidecar 2.02cwrk hottest 0.88c mean 0.76c------------------------------mean request 4.91 ms before 1.91 envoy 0.82 sidecar 0.88 worker 1.30 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6998 /s (100.0% ok)p50 4.8 ms p95 13.0 msin-flight 609 pool 2427------------------------------envoy 3.00c sidecar 2.12cwrk hottest 0.87c mean 0.75c------------------------------mean request 6.55 ms before 2.44 envoy 1.19 sidecar 1.03 worker 1.89 resume 0.99 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6996 /s (100.0% ok)p50 4.0 ms p95 13.4 msin-flight 1073 pool 2427------------------------------envoy 3.12c sidecar 0.00cwrk hottest 0.81c mean 0.78c------------------------------mean request 7.21 ms before 4.29 envoy 0.19 sidecar 1.46 worker 1.27 resume 1.42 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7624 /sachieved 7617 /s (100.0% ok)p50 9.5 ms p95 34.1 msin-flight 339 pool 2427------------------------------envoy 3.25c sidecar 2.09cwrk hottest 0.90c mean 0.81c------------------------------mean request 12.80 ms before 4.11 envoy 3.21 sidecar 0.98 worker 4.50 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (100.0% ok)p50 11.3 ms p95 33.5 msin-flight 324 pool 2427------------------------------envoy 3.29c sidecar 2.17cwrk hottest 0.89c mean 0.82c------------------------------mean request 13.82 ms before 5.66 envoy 2.61 sidecar 0.97 worker 4.58 resume 0.93 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7995 /s (100.0% ok)p50 9.0 ms p95 28.6 msin-flight 869 pool 1767------------------------------envoy 3.22c sidecar 2.15cwrk hottest 0.89c mean 0.80c------------------------------mean request 12.11 ms before 4.62 envoy 2.30 sidecar 1.11 worker 4.08 resume 1.07 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8674 /sachieved 8666 /s (100.0% ok)p50 39.0 ms p95 145.1 msin-flight 1388 pool 1389------------------------------envoy 3.37c sidecar 2.18cwrk hottest 0.89c mean 0.85c------------------------------mean request 51.75 ms before 21.12 envoy 10.67 sidecar 1.34 worker 18.62 resume 1.30 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9012 /s (100.0% ok)p50 33.6 ms p95 76.1 msin-flight 820 pool 1389------------------------------envoy 3.45c sidecar 2.05cwrk hottest 0.87c mean 0.86c------------------------------mean request 36.05 ms before 14.37 envoy 7.31 sidecar 1.21 worker 13.16 resume 1.17 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8826 /s (99.8% ok)p50 37.6 ms p95 276.9 msin-flight 3058 pool 3058------------------------------envoy 3.44c sidecar 0.00cwrk hottest 0.88c mean 0.86c------------------------------mean request 63.03 ms before 14.95 envoy 20.13 sidecar 2.34 worker 25.60 resume 2.29 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9854 /sachieved 8516 /s (99.1% ok)p50 1083.8 ms p95 2043.5 msin-flight 21k pool 21k------------------------------envoy 3.52c sidecar 1.98cwrk hottest 0.89c mean 0.88c------------------------------mean request 1118.51 ms before 286.98 envoy 730.18 sidecar 2.92 worker 98.44 resume 2.77 (control plane) +rung -1 0 qps setpoint------------------------------offered 191 /sachieved 191 /s (100.0% ok)p50 3.1 ms p95 9.0 msin-flight 35.0 pool 34.0------------------------------envoy 0.13c sidecar 0.01c------------------------------mean request 4.05 ms before 1.39 envoy 0.67 sidecar 0.79 worker 1.21 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.50c sidecar 0.48cwrk hottest 0.36c mean 0.13c------------------------------mean request 3.02 ms before 1.04 envoy 0.63 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 34.0------------------------------envoy 0.52c sidecar 0.50cwrk hottest 0.32c mean 0.13c------------------------------mean request 2.96 ms before 1.01 envoy 0.71 sidecar 0.79 worker 0.45 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1299 /sachieved 1299 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 571 pool 571------------------------------envoy 0.60c sidecar 0.49cwrk hottest 0.72c mean 0.16c------------------------------mean request 13.00 ms before 9.33 envoy 1.31 sidecar 0.90 worker 1.46 resume 0.86 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 24.0 pool 571------------------------------envoy 0.88c sidecar 0.79cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.23 ms before 1.19 envoy 0.52 sidecar 0.81 worker 0.72 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 21.0 pool 571------------------------------envoy 0.86c sidecar 0.82cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.08 ms before 1.12 envoy 0.57 sidecar 0.80 worker 0.59 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2756 /sachieved 2754 /s (100.0% ok)p50 3.6 ms p95 7.5 msin-flight 55.0 pool 571------------------------------envoy 1.14c sidecar 0.92cwrk hottest 0.93c mean 0.29c------------------------------mean request 4.09 ms before 1.69 envoy 0.67 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3002 /s (100.0% ok)p50 3.7 ms p95 7.8 msin-flight 92.0 pool 571------------------------------envoy 1.18c sidecar 1.05cwrk hottest 0.91c mean 0.29c------------------------------mean request 4.35 ms before 1.70 envoy 0.72 sidecar 0.80 worker 1.13 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.7 ms p95 7.3 msin-flight 70.0 pool 571------------------------------envoy 1.21c sidecar 1.04cwrk hottest 0.91c mean 0.30c------------------------------mean request 4.14 ms before 1.64 envoy 0.66 sidecar 0.81 worker 1.03 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3964 /sachieved 3962 /s (100.0% ok)p50 5.9 ms p95 22.2 msin-flight 180 pool 571------------------------------envoy 1.53c sidecar 1.12cwrk hottest 0.97c mean 0.39c------------------------------mean request 8.71 ms before 3.66 envoy 1.59 sidecar 0.84 worker 2.63 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 4.6 ms p95 14.8 msin-flight 97.0 pool 180------------------------------envoy 1.54c sidecar 1.25cwrk hottest 0.96c mean 0.38c------------------------------mean request 6.02 ms before 2.20 envoy 1.13 sidecar 0.83 worker 1.85 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4273 /sachieved 4271 /s (100.0% ok)p50 3.6 ms p95 16.9 msin-flight 188 pool 188------------------------------envoy 1.81c sidecar 1.36cwrk hottest 0.95c mean 0.45c------------------------------mean request 5.91 ms before 2.51 envoy 1.00 sidecar 0.88 worker 1.52 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5002 /s (100.0% ok)p50 4.4 ms p95 18.1 msin-flight 84.0 pool 188------------------------------envoy 2.07c sidecar 1.60cwrk hottest 0.95c mean 0.52c------------------------------mean request 6.76 ms before 2.55 envoy 1.31 sidecar 0.84 worker 2.06 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.7 ms p95 9.5 msin-flight 72.0 pool 188------------------------------envoy 2.14c sidecar 0.00cwrk hottest 0.94c mean 0.53c------------------------------mean request 4.66 ms before 1.90 envoy 0.84 sidecar 0.82 worker 1.10 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5152 /sachieved 5149 /s (100.0% ok)p50 4.6 ms p95 28.9 msin-flight 159 pool 188------------------------------envoy 2.09c sidecar 1.60cwrk hottest 0.95c mean 0.51c------------------------------mean request 8.27 ms before 4.41 envoy 1.17 sidecar 0.87 worker 1.82 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 3.8 ms p95 252.5 msin-flight 2427 pool 2427------------------------------envoy 2.69c sidecar 1.62cwrk hottest 0.87c mean 0.66c------------------------------mean request 34.41 ms before 14.18 envoy 8.08 sidecar 1.05 worker 11.11 resume 1.00 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.5 ms p95 6.1 msin-flight 74.0 pool 2427------------------------------envoy 2.76c sidecar 1.88cwrk hottest 0.84c mean 0.69c------------------------------mean request 3.83 ms before 1.43 envoy 0.65 sidecar 0.84 worker 0.91 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6340 /sachieved 6340 /s (100.0% ok)p50 3.6 ms p95 7.4 msin-flight 83.0 pool 2427------------------------------envoy 2.85c sidecar 1.93cwrk hottest 0.86c mean 0.72c------------------------------mean request 4.04 ms before 1.61 envoy 0.66 sidecar 0.82 worker 0.95 resume 0.79 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7001 /s (100.0% ok)p50 4.1 ms p95 9.7 msin-flight 223 pool 2427------------------------------envoy 3.06c sidecar 2.02cwrk hottest 0.88c mean 0.76c------------------------------mean request 4.91 ms before 1.91 envoy 0.82 sidecar 0.88 worker 1.30 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6998 /s (100.0% ok)p50 4.8 ms p95 13.0 msin-flight 609 pool 2427------------------------------envoy 3.00c sidecar 2.12cwrk hottest 0.87c mean 0.75c------------------------------mean request 6.55 ms before 2.44 envoy 1.19 sidecar 1.03 worker 1.89 resume 0.99 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6996 /s (100.0% ok)p50 4.0 ms p95 13.4 msin-flight 1073 pool 2427------------------------------envoy 3.12c sidecar 0.00cwrk hottest 0.81c mean 0.78c------------------------------mean request 7.21 ms before 4.29 envoy 0.19 sidecar 1.46 worker 1.27 resume 1.42 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7624 /sachieved 7617 /s (100.0% ok)p50 9.5 ms p95 34.1 msin-flight 339 pool 2427------------------------------envoy 3.25c sidecar 2.09cwrk hottest 0.90c mean 0.81c------------------------------mean request 12.80 ms before 4.11 envoy 3.21 sidecar 0.98 worker 4.50 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (100.0% ok)p50 11.3 ms p95 33.5 msin-flight 324 pool 2427------------------------------envoy 3.29c sidecar 2.17cwrk hottest 0.89c mean 0.82c------------------------------mean request 13.82 ms before 5.66 envoy 2.61 sidecar 0.97 worker 4.58 resume 0.93 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7995 /s (100.0% ok)p50 9.0 ms p95 28.6 msin-flight 869 pool 1767------------------------------envoy 3.22c sidecar 2.15cwrk hottest 0.89c mean 0.80c------------------------------mean request 12.11 ms before 4.62 envoy 2.30 sidecar 1.11 worker 4.08 resume 1.07 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8674 /sachieved 8666 /s (100.0% ok)p50 39.0 ms p95 145.1 msin-flight 1388 pool 1389------------------------------envoy 3.37c sidecar 2.18cwrk hottest 0.89c mean 0.85c------------------------------mean request 51.75 ms before 21.12 envoy 10.67 sidecar 1.34 worker 18.62 resume 1.30 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9012 /s (100.0% ok)p50 33.6 ms p95 76.1 msin-flight 820 pool 1389------------------------------envoy 3.45c sidecar 2.05cwrk hottest 0.87c mean 0.86c------------------------------mean request 36.05 ms before 14.37 envoy 7.31 sidecar 1.21 worker 13.16 resume 1.17 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8826 /s (99.8% ok)p50 37.6 ms p95 276.9 msin-flight 3058 pool 3058------------------------------envoy 3.44c sidecar 0.00cwrk hottest 0.88c mean 0.86c------------------------------mean request 63.03 ms before 14.95 envoy 20.13 sidecar 2.34 worker 25.60 resume 2.29 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9854 /sachieved 8516 /s (99.1% ok)p50 1083.8 ms p95 2043.5 msin-flight 21k pool 21k------------------------------envoy 3.52c sidecar 1.98cwrk hottest 0.89c mean 0.88c------------------------------mean request 1118.51 ms before 286.98 envoy 730.18 sidecar 2.92 worker 98.44 resume 2.77 (control plane) +rung -1 0 qps setpoint------------------------------offered 191 /sachieved 191 /s (100.0% ok)p50 3.1 ms p95 9.0 msin-flight 35.0 pool 34.0------------------------------envoy 0.13c sidecar 0.01c------------------------------mean request 4.05 ms before 1.39 envoy 0.67 sidecar 0.79 worker 1.21 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.50c sidecar 0.48cwrk hottest 0.36c mean 0.13c------------------------------mean request 3.02 ms before 1.04 envoy 0.63 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 34.0------------------------------envoy 0.52c sidecar 0.50cwrk hottest 0.32c mean 0.13c------------------------------mean request 2.96 ms before 1.01 envoy 0.71 sidecar 0.79 worker 0.45 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1299 /sachieved 1299 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 571 pool 571------------------------------envoy 0.60c sidecar 0.49cwrk hottest 0.72c mean 0.16c------------------------------mean request 13.00 ms before 9.33 envoy 1.31 sidecar 0.90 worker 1.46 resume 0.86 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 24.0 pool 571------------------------------envoy 0.88c sidecar 0.79cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.23 ms before 1.19 envoy 0.52 sidecar 0.81 worker 0.72 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 21.0 pool 571------------------------------envoy 0.86c sidecar 0.82cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.08 ms before 1.12 envoy 0.57 sidecar 0.80 worker 0.59 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2756 /sachieved 2754 /s (100.0% ok)p50 3.6 ms p95 7.5 msin-flight 55.0 pool 571------------------------------envoy 1.14c sidecar 0.92cwrk hottest 0.93c mean 0.29c------------------------------mean request 4.09 ms before 1.69 envoy 0.67 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3002 /s (100.0% ok)p50 3.7 ms p95 7.8 msin-flight 92.0 pool 571------------------------------envoy 1.18c sidecar 1.05cwrk hottest 0.91c mean 0.29c------------------------------mean request 4.35 ms before 1.70 envoy 0.72 sidecar 0.80 worker 1.13 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.7 ms p95 7.3 msin-flight 70.0 pool 571------------------------------envoy 1.21c sidecar 1.04cwrk hottest 0.91c mean 0.30c------------------------------mean request 4.14 ms before 1.64 envoy 0.66 sidecar 0.81 worker 1.03 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3964 /sachieved 3962 /s (100.0% ok)p50 5.9 ms p95 22.2 msin-flight 180 pool 571------------------------------envoy 1.53c sidecar 1.12cwrk hottest 0.97c mean 0.39c------------------------------mean request 8.71 ms before 3.66 envoy 1.59 sidecar 0.84 worker 2.63 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 4.6 ms p95 14.8 msin-flight 97.0 pool 180------------------------------envoy 1.54c sidecar 1.25cwrk hottest 0.96c mean 0.38c------------------------------mean request 6.02 ms before 2.20 envoy 1.13 sidecar 0.83 worker 1.85 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4273 /sachieved 4271 /s (100.0% ok)p50 3.6 ms p95 16.9 msin-flight 188 pool 188------------------------------envoy 1.81c sidecar 1.36cwrk hottest 0.95c mean 0.45c------------------------------mean request 5.91 ms before 2.51 envoy 1.00 sidecar 0.88 worker 1.52 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5002 /s (100.0% ok)p50 4.4 ms p95 18.1 msin-flight 84.0 pool 188------------------------------envoy 2.07c sidecar 1.60cwrk hottest 0.95c mean 0.52c------------------------------mean request 6.76 ms before 2.55 envoy 1.31 sidecar 0.84 worker 2.06 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.7 ms p95 9.5 msin-flight 72.0 pool 188------------------------------envoy 2.14c sidecar 0.00cwrk hottest 0.94c mean 0.53c------------------------------mean request 4.66 ms before 1.90 envoy 0.84 sidecar 0.82 worker 1.10 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5152 /sachieved 5149 /s (100.0% ok)p50 4.6 ms p95 28.9 msin-flight 159 pool 188------------------------------envoy 2.09c sidecar 1.60cwrk hottest 0.95c mean 0.51c------------------------------mean request 8.27 ms before 4.41 envoy 1.17 sidecar 0.87 worker 1.82 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 3.8 ms p95 252.5 msin-flight 2427 pool 2427------------------------------envoy 2.69c sidecar 1.62cwrk hottest 0.87c mean 0.66c------------------------------mean request 34.41 ms before 14.18 envoy 8.08 sidecar 1.05 worker 11.11 resume 1.00 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.5 ms p95 6.1 msin-flight 74.0 pool 2427------------------------------envoy 2.76c sidecar 1.88cwrk hottest 0.84c mean 0.69c------------------------------mean request 3.83 ms before 1.43 envoy 0.65 sidecar 0.84 worker 0.91 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6340 /sachieved 6340 /s (100.0% ok)p50 3.6 ms p95 7.4 msin-flight 83.0 pool 2427------------------------------envoy 2.85c sidecar 1.93cwrk hottest 0.86c mean 0.72c------------------------------mean request 4.04 ms before 1.61 envoy 0.66 sidecar 0.82 worker 0.95 resume 0.79 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7001 /s (100.0% ok)p50 4.1 ms p95 9.7 msin-flight 223 pool 2427------------------------------envoy 3.06c sidecar 2.02cwrk hottest 0.88c mean 0.76c------------------------------mean request 4.91 ms before 1.91 envoy 0.82 sidecar 0.88 worker 1.30 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6998 /s (100.0% ok)p50 4.8 ms p95 13.0 msin-flight 609 pool 2427------------------------------envoy 3.00c sidecar 2.12cwrk hottest 0.87c mean 0.75c------------------------------mean request 6.55 ms before 2.44 envoy 1.19 sidecar 1.03 worker 1.89 resume 0.99 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6996 /s (100.0% ok)p50 4.0 ms p95 13.4 msin-flight 1073 pool 2427------------------------------envoy 3.12c sidecar 0.00cwrk hottest 0.81c mean 0.78c------------------------------mean request 7.21 ms before 4.29 envoy 0.19 sidecar 1.46 worker 1.27 resume 1.42 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7624 /sachieved 7617 /s (100.0% ok)p50 9.5 ms p95 34.1 msin-flight 339 pool 2427------------------------------envoy 3.25c sidecar 2.09cwrk hottest 0.90c mean 0.81c------------------------------mean request 12.80 ms before 4.11 envoy 3.21 sidecar 0.98 worker 4.50 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (100.0% ok)p50 11.3 ms p95 33.5 msin-flight 324 pool 2427------------------------------envoy 3.29c sidecar 2.17cwrk hottest 0.89c mean 0.82c------------------------------mean request 13.82 ms before 5.66 envoy 2.61 sidecar 0.97 worker 4.58 resume 0.93 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7995 /s (100.0% ok)p50 9.0 ms p95 28.6 msin-flight 869 pool 1767------------------------------envoy 3.22c sidecar 2.15cwrk hottest 0.89c mean 0.80c------------------------------mean request 12.11 ms before 4.62 envoy 2.30 sidecar 1.11 worker 4.08 resume 1.07 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8674 /sachieved 8666 /s (100.0% ok)p50 39.0 ms p95 145.1 msin-flight 1388 pool 1389------------------------------envoy 3.37c sidecar 2.18cwrk hottest 0.89c mean 0.85c------------------------------mean request 51.75 ms before 21.12 envoy 10.67 sidecar 1.34 worker 18.62 resume 1.30 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9012 /s (100.0% ok)p50 33.6 ms p95 76.1 msin-flight 820 pool 1389------------------------------envoy 3.45c sidecar 2.05cwrk hottest 0.87c mean 0.86c------------------------------mean request 36.05 ms before 14.37 envoy 7.31 sidecar 1.21 worker 13.16 resume 1.17 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8826 /s (99.8% ok)p50 37.6 ms p95 276.9 msin-flight 3058 pool 3058------------------------------envoy 3.44c sidecar 0.00cwrk hottest 0.88c mean 0.86c------------------------------mean request 63.03 ms before 14.95 envoy 20.13 sidecar 2.34 worker 25.60 resume 2.29 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9854 /sachieved 8516 /s (99.1% ok)p50 1083.8 ms p95 2043.5 msin-flight 21k pool 21k------------------------------envoy 3.52c sidecar 1.98cwrk hottest 0.89c mean 0.88c------------------------------mean request 1118.51 ms before 286.98 envoy 730.18 sidecar 2.92 worker 98.44 resume 2.77 (control plane) +rung -1 0 qps setpoint------------------------------offered 191 /sachieved 191 /s (100.0% ok)p50 3.1 ms p95 9.0 msin-flight 35.0 pool 34.0------------------------------envoy 0.13c sidecar 0.01c------------------------------mean request 4.05 ms before 1.39 envoy 0.67 sidecar 0.79 worker 1.21 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.50c sidecar 0.48cwrk hottest 0.36c mean 0.13c------------------------------mean request 3.02 ms before 1.04 envoy 0.63 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 34.0------------------------------envoy 0.52c sidecar 0.50cwrk hottest 0.32c mean 0.13c------------------------------mean request 2.96 ms before 1.01 envoy 0.71 sidecar 0.79 worker 0.45 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1299 /sachieved 1299 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 571 pool 571------------------------------envoy 0.60c sidecar 0.49cwrk hottest 0.72c mean 0.16c------------------------------mean request 13.00 ms before 9.33 envoy 1.31 sidecar 0.90 worker 1.46 resume 0.86 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 24.0 pool 571------------------------------envoy 0.88c sidecar 0.79cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.23 ms before 1.19 envoy 0.52 sidecar 0.81 worker 0.72 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 21.0 pool 571------------------------------envoy 0.86c sidecar 0.82cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.08 ms before 1.12 envoy 0.57 sidecar 0.80 worker 0.59 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2756 /sachieved 2754 /s (100.0% ok)p50 3.6 ms p95 7.5 msin-flight 55.0 pool 571------------------------------envoy 1.14c sidecar 0.92cwrk hottest 0.93c mean 0.29c------------------------------mean request 4.09 ms before 1.69 envoy 0.67 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3002 /s (100.0% ok)p50 3.7 ms p95 7.8 msin-flight 92.0 pool 571------------------------------envoy 1.18c sidecar 1.05cwrk hottest 0.91c mean 0.29c------------------------------mean request 4.35 ms before 1.70 envoy 0.72 sidecar 0.80 worker 1.13 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.7 ms p95 7.3 msin-flight 70.0 pool 571------------------------------envoy 1.21c sidecar 1.04cwrk hottest 0.91c mean 0.30c------------------------------mean request 4.14 ms before 1.64 envoy 0.66 sidecar 0.81 worker 1.03 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3964 /sachieved 3962 /s (100.0% ok)p50 5.9 ms p95 22.2 msin-flight 180 pool 571------------------------------envoy 1.53c sidecar 1.12cwrk hottest 0.97c mean 0.39c------------------------------mean request 8.71 ms before 3.66 envoy 1.59 sidecar 0.84 worker 2.63 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 4.6 ms p95 14.8 msin-flight 97.0 pool 180------------------------------envoy 1.54c sidecar 1.25cwrk hottest 0.96c mean 0.38c------------------------------mean request 6.02 ms before 2.20 envoy 1.13 sidecar 0.83 worker 1.85 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4273 /sachieved 4271 /s (100.0% ok)p50 3.6 ms p95 16.9 msin-flight 188 pool 188------------------------------envoy 1.81c sidecar 1.36cwrk hottest 0.95c mean 0.45c------------------------------mean request 5.91 ms before 2.51 envoy 1.00 sidecar 0.88 worker 1.52 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5002 /s (100.0% ok)p50 4.4 ms p95 18.1 msin-flight 84.0 pool 188------------------------------envoy 2.07c sidecar 1.60cwrk hottest 0.95c mean 0.52c------------------------------mean request 6.76 ms before 2.55 envoy 1.31 sidecar 0.84 worker 2.06 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.7 ms p95 9.5 msin-flight 72.0 pool 188------------------------------envoy 2.14c sidecar 0.00cwrk hottest 0.94c mean 0.53c------------------------------mean request 4.66 ms before 1.90 envoy 0.84 sidecar 0.82 worker 1.10 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5152 /sachieved 5149 /s (100.0% ok)p50 4.6 ms p95 28.9 msin-flight 159 pool 188------------------------------envoy 2.09c sidecar 1.60cwrk hottest 0.95c mean 0.51c------------------------------mean request 8.27 ms before 4.41 envoy 1.17 sidecar 0.87 worker 1.82 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 3.8 ms p95 252.5 msin-flight 2427 pool 2427------------------------------envoy 2.69c sidecar 1.62cwrk hottest 0.87c mean 0.66c------------------------------mean request 34.41 ms before 14.18 envoy 8.08 sidecar 1.05 worker 11.11 resume 1.00 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.5 ms p95 6.1 msin-flight 74.0 pool 2427------------------------------envoy 2.76c sidecar 1.88cwrk hottest 0.84c mean 0.69c------------------------------mean request 3.83 ms before 1.43 envoy 0.65 sidecar 0.84 worker 0.91 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6340 /sachieved 6340 /s (100.0% ok)p50 3.6 ms p95 7.4 msin-flight 83.0 pool 2427------------------------------envoy 2.85c sidecar 1.93cwrk hottest 0.86c mean 0.72c------------------------------mean request 4.04 ms before 1.61 envoy 0.66 sidecar 0.82 worker 0.95 resume 0.79 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7001 /s (100.0% ok)p50 4.1 ms p95 9.7 msin-flight 223 pool 2427------------------------------envoy 3.06c sidecar 2.02cwrk hottest 0.88c mean 0.76c------------------------------mean request 4.91 ms before 1.91 envoy 0.82 sidecar 0.88 worker 1.30 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6998 /s (100.0% ok)p50 4.8 ms p95 13.0 msin-flight 609 pool 2427------------------------------envoy 3.00c sidecar 2.12cwrk hottest 0.87c mean 0.75c------------------------------mean request 6.55 ms before 2.44 envoy 1.19 sidecar 1.03 worker 1.89 resume 0.99 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6996 /s (100.0% ok)p50 4.0 ms p95 13.4 msin-flight 1073 pool 2427------------------------------envoy 3.12c sidecar 0.00cwrk hottest 0.81c mean 0.78c------------------------------mean request 7.21 ms before 4.29 envoy 0.19 sidecar 1.46 worker 1.27 resume 1.42 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7624 /sachieved 7617 /s (100.0% ok)p50 9.5 ms p95 34.1 msin-flight 339 pool 2427------------------------------envoy 3.25c sidecar 2.09cwrk hottest 0.90c mean 0.81c------------------------------mean request 12.80 ms before 4.11 envoy 3.21 sidecar 0.98 worker 4.50 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (100.0% ok)p50 11.3 ms p95 33.5 msin-flight 324 pool 2427------------------------------envoy 3.29c sidecar 2.17cwrk hottest 0.89c mean 0.82c------------------------------mean request 13.82 ms before 5.66 envoy 2.61 sidecar 0.97 worker 4.58 resume 0.93 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7995 /s (100.0% ok)p50 9.0 ms p95 28.6 msin-flight 869 pool 1767------------------------------envoy 3.22c sidecar 2.15cwrk hottest 0.89c mean 0.80c------------------------------mean request 12.11 ms before 4.62 envoy 2.30 sidecar 1.11 worker 4.08 resume 1.07 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8674 /sachieved 8666 /s (100.0% ok)p50 39.0 ms p95 145.1 msin-flight 1388 pool 1389------------------------------envoy 3.37c sidecar 2.18cwrk hottest 0.89c mean 0.85c------------------------------mean request 51.75 ms before 21.12 envoy 10.67 sidecar 1.34 worker 18.62 resume 1.30 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9012 /s (100.0% ok)p50 33.6 ms p95 76.1 msin-flight 820 pool 1389------------------------------envoy 3.45c sidecar 2.05cwrk hottest 0.87c mean 0.86c------------------------------mean request 36.05 ms before 14.37 envoy 7.31 sidecar 1.21 worker 13.16 resume 1.17 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8826 /s (99.8% ok)p50 37.6 ms p95 276.9 msin-flight 3058 pool 3058------------------------------envoy 3.44c sidecar 0.00cwrk hottest 0.88c mean 0.86c------------------------------mean request 63.03 ms before 14.95 envoy 20.13 sidecar 2.34 worker 25.60 resume 2.29 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9854 /sachieved 8516 /s (99.1% ok)p50 1083.8 ms p95 2043.5 msin-flight 21k pool 21k------------------------------envoy 3.52c sidecar 1.98cwrk hottest 0.89c mean 0.88c------------------------------mean request 1118.51 ms before 286.98 envoy 730.18 sidecar 2.92 worker 98.44 resume 2.77 (control plane) +rung -1 0 qps setpoint------------------------------offered 191 /sachieved 191 /s (100.0% ok)p50 3.1 ms p95 9.0 msin-flight 35.0 pool 34.0------------------------------envoy 0.13c sidecar 0.01c------------------------------mean request 4.05 ms before 1.39 envoy 0.67 sidecar 0.79 worker 1.21 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.50c sidecar 0.48cwrk hottest 0.36c mean 0.13c------------------------------mean request 3.02 ms before 1.04 envoy 0.63 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 34.0------------------------------envoy 0.52c sidecar 0.50cwrk hottest 0.32c mean 0.13c------------------------------mean request 2.96 ms before 1.01 envoy 0.71 sidecar 0.79 worker 0.45 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1299 /sachieved 1299 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 571 pool 571------------------------------envoy 0.60c sidecar 0.49cwrk hottest 0.72c mean 0.16c------------------------------mean request 13.00 ms before 9.33 envoy 1.31 sidecar 0.90 worker 1.46 resume 0.86 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 24.0 pool 571------------------------------envoy 0.88c sidecar 0.79cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.23 ms before 1.19 envoy 0.52 sidecar 0.81 worker 0.72 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 21.0 pool 571------------------------------envoy 0.86c sidecar 0.82cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.08 ms before 1.12 envoy 0.57 sidecar 0.80 worker 0.59 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2756 /sachieved 2754 /s (100.0% ok)p50 3.6 ms p95 7.5 msin-flight 55.0 pool 571------------------------------envoy 1.14c sidecar 0.92cwrk hottest 0.93c mean 0.29c------------------------------mean request 4.09 ms before 1.69 envoy 0.67 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3002 /s (100.0% ok)p50 3.7 ms p95 7.8 msin-flight 92.0 pool 571------------------------------envoy 1.18c sidecar 1.05cwrk hottest 0.91c mean 0.29c------------------------------mean request 4.35 ms before 1.70 envoy 0.72 sidecar 0.80 worker 1.13 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.7 ms p95 7.3 msin-flight 70.0 pool 571------------------------------envoy 1.21c sidecar 1.04cwrk hottest 0.91c mean 0.30c------------------------------mean request 4.14 ms before 1.64 envoy 0.66 sidecar 0.81 worker 1.03 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3964 /sachieved 3962 /s (100.0% ok)p50 5.9 ms p95 22.2 msin-flight 180 pool 571------------------------------envoy 1.53c sidecar 1.12cwrk hottest 0.97c mean 0.39c------------------------------mean request 8.71 ms before 3.66 envoy 1.59 sidecar 0.84 worker 2.63 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 4.6 ms p95 14.8 msin-flight 97.0 pool 180------------------------------envoy 1.54c sidecar 1.25cwrk hottest 0.96c mean 0.38c------------------------------mean request 6.02 ms before 2.20 envoy 1.13 sidecar 0.83 worker 1.85 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4273 /sachieved 4271 /s (100.0% ok)p50 3.6 ms p95 16.9 msin-flight 188 pool 188------------------------------envoy 1.81c sidecar 1.36cwrk hottest 0.95c mean 0.45c------------------------------mean request 5.91 ms before 2.51 envoy 1.00 sidecar 0.88 worker 1.52 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5002 /s (100.0% ok)p50 4.4 ms p95 18.1 msin-flight 84.0 pool 188------------------------------envoy 2.07c sidecar 1.60cwrk hottest 0.95c mean 0.52c------------------------------mean request 6.76 ms before 2.55 envoy 1.31 sidecar 0.84 worker 2.06 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.7 ms p95 9.5 msin-flight 72.0 pool 188------------------------------envoy 2.14c sidecar 0.00cwrk hottest 0.94c mean 0.53c------------------------------mean request 4.66 ms before 1.90 envoy 0.84 sidecar 0.82 worker 1.10 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5152 /sachieved 5149 /s (100.0% ok)p50 4.6 ms p95 28.9 msin-flight 159 pool 188------------------------------envoy 2.09c sidecar 1.60cwrk hottest 0.95c mean 0.51c------------------------------mean request 8.27 ms before 4.41 envoy 1.17 sidecar 0.87 worker 1.82 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 3.8 ms p95 252.5 msin-flight 2427 pool 2427------------------------------envoy 2.69c sidecar 1.62cwrk hottest 0.87c mean 0.66c------------------------------mean request 34.41 ms before 14.18 envoy 8.08 sidecar 1.05 worker 11.11 resume 1.00 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.5 ms p95 6.1 msin-flight 74.0 pool 2427------------------------------envoy 2.76c sidecar 1.88cwrk hottest 0.84c mean 0.69c------------------------------mean request 3.83 ms before 1.43 envoy 0.65 sidecar 0.84 worker 0.91 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6340 /sachieved 6340 /s (100.0% ok)p50 3.6 ms p95 7.4 msin-flight 83.0 pool 2427------------------------------envoy 2.85c sidecar 1.93cwrk hottest 0.86c mean 0.72c------------------------------mean request 4.04 ms before 1.61 envoy 0.66 sidecar 0.82 worker 0.95 resume 0.79 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7001 /s (100.0% ok)p50 4.1 ms p95 9.7 msin-flight 223 pool 2427------------------------------envoy 3.06c sidecar 2.02cwrk hottest 0.88c mean 0.76c------------------------------mean request 4.91 ms before 1.91 envoy 0.82 sidecar 0.88 worker 1.30 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6998 /s (100.0% ok)p50 4.8 ms p95 13.0 msin-flight 609 pool 2427------------------------------envoy 3.00c sidecar 2.12cwrk hottest 0.87c mean 0.75c------------------------------mean request 6.55 ms before 2.44 envoy 1.19 sidecar 1.03 worker 1.89 resume 0.99 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6996 /s (100.0% ok)p50 4.0 ms p95 13.4 msin-flight 1073 pool 2427------------------------------envoy 3.12c sidecar 0.00cwrk hottest 0.81c mean 0.78c------------------------------mean request 7.21 ms before 4.29 envoy 0.19 sidecar 1.46 worker 1.27 resume 1.42 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7624 /sachieved 7617 /s (100.0% ok)p50 9.5 ms p95 34.1 msin-flight 339 pool 2427------------------------------envoy 3.25c sidecar 2.09cwrk hottest 0.90c mean 0.81c------------------------------mean request 12.80 ms before 4.11 envoy 3.21 sidecar 0.98 worker 4.50 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (100.0% ok)p50 11.3 ms p95 33.5 msin-flight 324 pool 2427------------------------------envoy 3.29c sidecar 2.17cwrk hottest 0.89c mean 0.82c------------------------------mean request 13.82 ms before 5.66 envoy 2.61 sidecar 0.97 worker 4.58 resume 0.93 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7995 /s (100.0% ok)p50 9.0 ms p95 28.6 msin-flight 869 pool 1767------------------------------envoy 3.22c sidecar 2.15cwrk hottest 0.89c mean 0.80c------------------------------mean request 12.11 ms before 4.62 envoy 2.30 sidecar 1.11 worker 4.08 resume 1.07 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8674 /sachieved 8666 /s (100.0% ok)p50 39.0 ms p95 145.1 msin-flight 1388 pool 1389------------------------------envoy 3.37c sidecar 2.18cwrk hottest 0.89c mean 0.85c------------------------------mean request 51.75 ms before 21.12 envoy 10.67 sidecar 1.34 worker 18.62 resume 1.30 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9012 /s (100.0% ok)p50 33.6 ms p95 76.1 msin-flight 820 pool 1389------------------------------envoy 3.45c sidecar 2.05cwrk hottest 0.87c mean 0.86c------------------------------mean request 36.05 ms before 14.37 envoy 7.31 sidecar 1.21 worker 13.16 resume 1.17 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8826 /s (99.8% ok)p50 37.6 ms p95 276.9 msin-flight 3058 pool 3058------------------------------envoy 3.44c sidecar 0.00cwrk hottest 0.88c mean 0.86c------------------------------mean request 63.03 ms before 14.95 envoy 20.13 sidecar 2.34 worker 25.60 resume 2.29 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9854 /sachieved 8516 /s (99.1% ok)p50 1083.8 ms p95 2043.5 msin-flight 21k pool 21k------------------------------envoy 3.52c sidecar 1.98cwrk hottest 0.89c mean 0.88c------------------------------mean request 1118.51 ms before 286.98 envoy 730.18 sidecar 2.92 worker 98.44 resume 2.77 (control plane) +rung -1 0 qps setpoint------------------------------offered 191 /sachieved 191 /s (100.0% ok)p50 3.1 ms p95 9.0 msin-flight 35.0 pool 34.0------------------------------envoy 0.13c sidecar 0.01c------------------------------mean request 4.05 ms before 1.39 envoy 0.67 sidecar 0.79 worker 1.21 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.50c sidecar 0.48cwrk hottest 0.36c mean 0.13c------------------------------mean request 3.02 ms before 1.04 envoy 0.63 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 34.0------------------------------envoy 0.52c sidecar 0.50cwrk hottest 0.32c mean 0.13c------------------------------mean request 2.96 ms before 1.01 envoy 0.71 sidecar 0.79 worker 0.45 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1299 /sachieved 1299 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 571 pool 571------------------------------envoy 0.60c sidecar 0.49cwrk hottest 0.72c mean 0.16c------------------------------mean request 13.00 ms before 9.33 envoy 1.31 sidecar 0.90 worker 1.46 resume 0.86 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 24.0 pool 571------------------------------envoy 0.88c sidecar 0.79cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.23 ms before 1.19 envoy 0.52 sidecar 0.81 worker 0.72 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 21.0 pool 571------------------------------envoy 0.86c sidecar 0.82cwrk hottest 0.72c mean 0.22c------------------------------mean request 3.08 ms before 1.12 envoy 0.57 sidecar 0.80 worker 0.59 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2756 /sachieved 2754 /s (100.0% ok)p50 3.6 ms p95 7.5 msin-flight 55.0 pool 571------------------------------envoy 1.14c sidecar 0.92cwrk hottest 0.93c mean 0.29c------------------------------mean request 4.09 ms before 1.69 envoy 0.67 sidecar 0.81 worker 0.93 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3002 /s (100.0% ok)p50 3.7 ms p95 7.8 msin-flight 92.0 pool 571------------------------------envoy 1.18c sidecar 1.05cwrk hottest 0.91c mean 0.29c------------------------------mean request 4.35 ms before 1.70 envoy 0.72 sidecar 0.80 worker 1.13 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.7 ms p95 7.3 msin-flight 70.0 pool 571------------------------------envoy 1.21c sidecar 1.04cwrk hottest 0.91c mean 0.30c------------------------------mean request 4.14 ms before 1.64 envoy 0.66 sidecar 0.81 worker 1.03 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3964 /sachieved 3962 /s (100.0% ok)p50 5.9 ms p95 22.2 msin-flight 180 pool 571------------------------------envoy 1.53c sidecar 1.12cwrk hottest 0.97c mean 0.39c------------------------------mean request 8.71 ms before 3.66 envoy 1.59 sidecar 0.84 worker 2.63 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 4.6 ms p95 14.8 msin-flight 97.0 pool 180------------------------------envoy 1.54c sidecar 1.25cwrk hottest 0.96c mean 0.38c------------------------------mean request 6.02 ms before 2.20 envoy 1.13 sidecar 0.83 worker 1.85 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4273 /sachieved 4271 /s (100.0% ok)p50 3.6 ms p95 16.9 msin-flight 188 pool 188------------------------------envoy 1.81c sidecar 1.36cwrk hottest 0.95c mean 0.45c------------------------------mean request 5.91 ms before 2.51 envoy 1.00 sidecar 0.88 worker 1.52 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5002 /s (100.0% ok)p50 4.4 ms p95 18.1 msin-flight 84.0 pool 188------------------------------envoy 2.07c sidecar 1.60cwrk hottest 0.95c mean 0.52c------------------------------mean request 6.76 ms before 2.55 envoy 1.31 sidecar 0.84 worker 2.06 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.7 ms p95 9.5 msin-flight 72.0 pool 188------------------------------envoy 2.14c sidecar 0.00cwrk hottest 0.94c mean 0.53c------------------------------mean request 4.66 ms before 1.90 envoy 0.84 sidecar 0.82 worker 1.10 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5152 /sachieved 5149 /s (100.0% ok)p50 4.6 ms p95 28.9 msin-flight 159 pool 188------------------------------envoy 2.09c sidecar 1.60cwrk hottest 0.95c mean 0.51c------------------------------mean request 8.27 ms before 4.41 envoy 1.17 sidecar 0.87 worker 1.82 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 3.8 ms p95 252.5 msin-flight 2427 pool 2427------------------------------envoy 2.69c sidecar 1.62cwrk hottest 0.87c mean 0.66c------------------------------mean request 34.41 ms before 14.18 envoy 8.08 sidecar 1.05 worker 11.11 resume 1.00 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.5 ms p95 6.1 msin-flight 74.0 pool 2427------------------------------envoy 2.76c sidecar 1.88cwrk hottest 0.84c mean 0.69c------------------------------mean request 3.83 ms before 1.43 envoy 0.65 sidecar 0.84 worker 0.91 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6340 /sachieved 6340 /s (100.0% ok)p50 3.6 ms p95 7.4 msin-flight 83.0 pool 2427------------------------------envoy 2.85c sidecar 1.93cwrk hottest 0.86c mean 0.72c------------------------------mean request 4.04 ms before 1.61 envoy 0.66 sidecar 0.82 worker 0.95 resume 0.79 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7001 /s (100.0% ok)p50 4.1 ms p95 9.7 msin-flight 223 pool 2427------------------------------envoy 3.06c sidecar 2.02cwrk hottest 0.88c mean 0.76c------------------------------mean request 4.91 ms before 1.91 envoy 0.82 sidecar 0.88 worker 1.30 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6998 /s (100.0% ok)p50 4.8 ms p95 13.0 msin-flight 609 pool 2427------------------------------envoy 3.00c sidecar 2.12cwrk hottest 0.87c mean 0.75c------------------------------mean request 6.55 ms before 2.44 envoy 1.19 sidecar 1.03 worker 1.89 resume 0.99 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6996 /s (100.0% ok)p50 4.0 ms p95 13.4 msin-flight 1073 pool 2427------------------------------envoy 3.12c sidecar 0.00cwrk hottest 0.81c mean 0.78c------------------------------mean request 7.21 ms before 4.29 envoy 0.19 sidecar 1.46 worker 1.27 resume 1.42 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7624 /sachieved 7617 /s (100.0% ok)p50 9.5 ms p95 34.1 msin-flight 339 pool 2427------------------------------envoy 3.25c sidecar 2.09cwrk hottest 0.90c mean 0.81c------------------------------mean request 12.80 ms before 4.11 envoy 3.21 sidecar 0.98 worker 4.50 resume 0.94 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (100.0% ok)p50 11.3 ms p95 33.5 msin-flight 324 pool 2427------------------------------envoy 3.29c sidecar 2.17cwrk hottest 0.89c mean 0.82c------------------------------mean request 13.82 ms before 5.66 envoy 2.61 sidecar 0.97 worker 4.58 resume 0.93 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7995 /s (100.0% ok)p50 9.0 ms p95 28.6 msin-flight 869 pool 1767------------------------------envoy 3.22c sidecar 2.15cwrk hottest 0.89c mean 0.80c------------------------------mean request 12.11 ms before 4.62 envoy 2.30 sidecar 1.11 worker 4.08 resume 1.07 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8674 /sachieved 8666 /s (100.0% ok)p50 39.0 ms p95 145.1 msin-flight 1388 pool 1389------------------------------envoy 3.37c sidecar 2.18cwrk hottest 0.89c mean 0.85c------------------------------mean request 51.75 ms before 21.12 envoy 10.67 sidecar 1.34 worker 18.62 resume 1.30 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9012 /s (100.0% ok)p50 33.6 ms p95 76.1 msin-flight 820 pool 1389------------------------------envoy 3.45c sidecar 2.05cwrk hottest 0.87c mean 0.86c------------------------------mean request 36.05 ms before 14.37 envoy 7.31 sidecar 1.21 worker 13.16 resume 1.17 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8826 /s (99.8% ok)p50 37.6 ms p95 276.9 msin-flight 3058 pool 3058------------------------------envoy 3.44c sidecar 0.00cwrk hottest 0.88c mean 0.86c------------------------------mean request 63.03 ms before 14.95 envoy 20.13 sidecar 2.34 worker 25.60 resume 2.29 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9854 /sachieved 8516 /s (99.1% ok)p50 1083.8 ms p95 2043.5 msin-flight 21k pool 21k------------------------------envoy 3.52c sidecar 1.98cwrk hottest 0.89c mean 0.88c------------------------------mean request 1118.51 ms before 286.98 envoy 730.18 sidecar 2.92 worker 98.44 resume 2.77 (control plane) +
+ + + +atenet-router capacity — 8c/8t +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 +R9 +R10 +R11 +R12 + +0 + +10k + +20k + +30k +requests / s + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + + + + + +0 + +2.0 + +4.0 + +6.0 + +8.0 +cores + + + + + + + +envoy CPU limit, 8 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + + + + + +0 + +500.0M + +1.0G + +1.5G +bytes + +100 + +200 + +300 + +400 + +500 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 83.9 /sachieved 83.7 /s (100.0% ok)p50 3.6 ms p95 11.2 msin-flight 24.0 pool 24.0------------------------------envoy 0.07c sidecar 0.00c------------------------------mean request 4.86 ms before 1.55 envoy 0.66 sidecar 0.77 worker 1.87 resume 0.73 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 2.95 ms before 1.05 envoy 0.67 sidecar 0.77 worker 0.45 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 305 pool 304------------------------------envoy 0.53c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 6.97 ms before 5.03 envoy 0.04 sidecar 1.42 worker 0.47 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 304------------------------------envoy 0.51c sidecar 0.50cwrk hottest 0.28c mean 0.07c------------------------------mean request 2.92 ms before 0.19 envoy 1.41 sidecar 0.79 worker 0.53 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1969 /sachieved 1969 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 116 pool 304------------------------------envoy 0.91c sidecar 0.73cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.25 ms before 1.25 envoy 0.73 sidecar 0.81 worker 0.46 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 17.0 pool 304------------------------------envoy 0.88c sidecar 0.86cwrk hottest 0.72c mean 0.11c------------------------------mean request 3.08 ms before 1.11 envoy 0.63 sidecar 0.80 worker 0.53 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2200 /sachieved 2200 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 27.0 pool 304------------------------------envoy 0.91c sidecar 0.81cwrk hottest 0.69c mean 0.11c------------------------------mean request 3.06 ms before 1.13 envoy 0.55 sidecar 0.80 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 30.0 pool 304------------------------------envoy 1.33c sidecar 0.96cwrk hottest 0.78c mean 0.17c------------------------------mean request 3.16 ms before 1.17 envoy 0.62 sidecar 0.79 worker 0.57 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 21.0 pool 304------------------------------envoy 1.26c sidecar 1.15cwrk hottest 0.79c mean 0.16c------------------------------mean request 3.24 ms before 1.23 envoy 0.59 sidecar 0.80 worker 0.62 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3054 /sachieved 3054 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 43.0 pool 304------------------------------envoy 1.29c sidecar 1.14cwrk hottest 0.82c mean 0.16c------------------------------mean request 3.29 ms before 1.24 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 47.0 pool 116------------------------------envoy 1.60c sidecar 1.31cwrk hottest 0.87c mean 0.20c------------------------------mean request 3.55 ms before 1.32 envoy 0.60 sidecar 0.79 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 39.0 pool 116------------------------------envoy 1.55c sidecar 1.38cwrk hottest 0.85c mean 0.19c------------------------------mean request 3.46 ms before 1.31 envoy 0.53 sidecar 0.80 worker 0.82 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4190 /sachieved 4189 /s (100.0% ok)p50 3.4 ms p95 6.4 msin-flight 52.0 pool 52.0------------------------------envoy 1.69c sidecar 1.35cwrk hottest 0.91c mean 0.20c------------------------------mean request 3.71 ms before 1.37 envoy 0.59 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.2 ms p95 6.1 msin-flight 52.0 pool 52.0------------------------------envoy 2.18c sidecar 1.52cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.55 ms before 1.32 envoy 0.68 sidecar 0.80 worker 0.74 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.1 ms p95 5.5 msin-flight 50.0 pool 54.0------------------------------envoy 2.22c sidecar 1.72cwrk hottest 0.89c mean 0.28c------------------------------mean request 3.39 ms before 1.21 envoy 0.69 sidecar 0.80 worker 0.69 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5151 /sachieved 5150 /s (100.0% ok)p50 3.3 ms p95 6.9 msin-flight 55.0 pool 61.0------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.91c mean 0.28c------------------------------mean request 3.78 ms before 1.38 envoy 0.70 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 7.0 msin-flight 127 pool 126------------------------------envoy 2.84c sidecar 1.95cwrk hottest 0.87c mean 0.36c------------------------------mean request 3.57 ms before 1.28 envoy 0.75 sidecar 0.83 worker 0.71 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.1 ms p95 8.8 msin-flight 74.0 pool 126------------------------------envoy 2.95c sidecar 0.00cwrk hottest 0.89c mean 0.37c------------------------------mean request 4.00 ms before 1.47 envoy 0.88 sidecar 0.84 worker 0.81 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 6.1 msin-flight 117 pool 126------------------------------envoy 2.82c sidecar 2.10cwrk hottest 0.86c mean 0.35c------------------------------mean request 3.50 ms before 1.21 envoy 0.76 sidecar 0.86 worker 0.68 resume 0.82 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6804 /sachieved 6803 /s (99.1% ok)p50 3.4 ms p95 9.1 msin-flight 1386 pool 1385------------------------------envoy 3.17c sidecar 2.11cwrk hottest 0.87c mean 0.40c------------------------------mean request 7.57 ms before 2.04 envoy 0.99 sidecar 2.96 worker 1.57 resume 2.95 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.2% ok)p50 3.3 ms p95 8.1 msin-flight 1349 pool 1385------------------------------envoy 3.29c sidecar 2.27cwrk hottest 0.81c mean 0.41c------------------------------mean request 7.89 ms before 1.56 envoy 1.38 sidecar 3.02 worker 1.93 resume 3.01 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.5% ok)p50 3.3 ms p95 7.5 msin-flight 1384 pool 1385------------------------------envoy 3.32c sidecar 0.00cwrk hottest 0.82c mean 0.41c------------------------------mean request 6.94 ms before 2.13 envoy 0.77 sidecar 2.52 worker 1.52 resume 2.49 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7494 /sachieved 7494 /s (100.0% ok)p50 3.5 ms p95 9.6 msin-flight 348 pool 1385------------------------------envoy 3.56c sidecar 2.28cwrk hottest 0.87c mean 0.44c------------------------------mean request 4.62 ms before 1.65 envoy 0.88 sidecar 0.93 worker 1.16 resume 0.90 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 4.4 ms p95 14.4 msin-flight 167 pool 1385------------------------------envoy 3.64c sidecar 2.44cwrk hottest 0.88c mean 0.46c------------------------------mean request 5.95 ms before 2.31 envoy 1.13 sidecar 0.90 worker 1.61 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 4.0 ms p95 13.6 msin-flight 153 pool 1385------------------------------envoy 3.69c sidecar 0.00cwrk hottest 0.87c mean 0.46c------------------------------mean request 5.55 ms before 2.10 envoy 1.07 sidecar 0.88 worker 1.50 resume 0.84 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8510 /sachieved 8486 /s (100.0% ok)p50 4.8 ms p95 19.5 msin-flight 6250 pool 6250------------------------------envoy 3.85c sidecar 2.46cwrk hottest 0.89c mean 0.48c------------------------------mean request 7.66 ms before 3.33 envoy 1.11 sidecar 1.14 worker 2.08 resume 1.02 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9036 /s (100.0% ok)p50 17.8 ms p95 661.4 msin-flight 6461 pool 6461------------------------------envoy 4.44c sidecar 0.00cwrk hottest 0.79c mean 0.54c------------------------------mean request 170.90 ms before 55.23 envoy 57.91 sidecar 1.81 worker 55.96 resume 1.45 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8994 /s (100.0% ok)p50 5.2 ms p95 38.1 msin-flight 320 pool 6461------------------------------envoy 4.10c sidecar 2.50cwrk hottest 0.87c mean 0.52c------------------------------mean request 11.11 ms before 3.42 envoy 2.72 sidecar 0.93 worker 4.04 resume 0.89 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9633 /sachieved 9631 /s (100.0% ok)p50 6.0 ms p95 62.5 msin-flight 467 pool 6461------------------------------envoy 4.35c sidecar 2.67cwrk hottest 0.87c mean 0.54c------------------------------mean request 14.97 ms before 6.16 envoy 2.91 sidecar 0.97 worker 4.93 resume 0.93 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 6.2 ms p95 43.3 msin-flight 578 pool 6461------------------------------envoy 4.54c sidecar 2.77cwrk hottest 0.84c mean 0.57c------------------------------mean request 12.94 ms before 5.30 envoy 2.48 sidecar 1.01 worker 4.14 resume 0.97 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 5.7 ms p95 41.1 msin-flight 445 pool 6461------------------------------envoy 4.55c sidecar 0.00cwrk hottest 0.83c mean 0.57c------------------------------mean request 10.90 ms before 3.88 envoy 2.37 sidecar 0.88 worker 3.77 resume 0.84 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 8.4 ms p95 62.4 msin-flight 855 pool 6461------------------------------envoy 4.72c sidecar 2.80cwrk hottest 0.83c mean 0.59c------------------------------mean request 17.81 ms before 7.45 envoy 3.53 sidecar 0.96 worker 5.87 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.2 ms p95 71.5 msin-flight 587 pool 6461------------------------------envoy 5.10c sidecar 2.91cwrk hottest 0.80c mean 0.64c------------------------------mean request 22.11 ms before 8.40 envoy 4.75 sidecar 0.99 worker 7.97 resume 0.95 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.8 ms p95 69.3 msin-flight 741 pool 2962------------------------------envoy 5.19c sidecar 3.00cwrk hottest 0.80c mean 0.65c------------------------------mean request 21.68 ms before 9.27 envoy 4.28 sidecar 0.99 worker 7.15 resume 0.94 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 18.6 ms p95 118.3 msin-flight 783 pool 854------------------------------envoy 5.30c sidecar 0.00cwrk hottest 0.78c mean 0.67c------------------------------mean request 33.30 ms before 14.46 envoy 6.61 sidecar 1.07 worker 11.15 resume 1.03 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 23.5 ms p95 118.8 msin-flight 1842 pool 1843------------------------------envoy 5.35c sidecar 2.98cwrk hottest 0.77c mean 0.68c------------------------------mean request 40.85 ms before 18.26 envoy 7.76 sidecar 1.51 worker 13.32 resume 1.47 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.8% ok)p50 22.1 ms p95 168.9 msin-flight 2426 pool 2426------------------------------envoy 5.64c sidecar 3.02cwrk hottest 0.76c mean 0.70c------------------------------mean request 46.31 ms before 16.01 envoy 10.95 sidecar 1.80 worker 17.55 resume 1.75 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 14.1 ms p95 239.5 msin-flight 6597 pool 6698------------------------------envoy 5.85c sidecar 2.95cwrk hottest 0.75c mean 0.73c------------------------------mean request 49.43 ms before 17.17 envoy 12.08 sidecar 1.40 worker 18.78 resume 1.34 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 996.9 ms p95 1514.0 msin-flight 21k pool 21k------------------------------envoy 5.88c sidecar 2.43cwrk hottest 0.74c mean 0.66c------------------------------mean request 983.18 ms before 233.22 envoy 607.22 sidecar 2.96 worker 139.78 resume 2.86 (control plane) +rung -1 0 qps setpoint------------------------------offered 83.9 /sachieved 83.7 /s (100.0% ok)p50 3.6 ms p95 11.2 msin-flight 24.0 pool 24.0------------------------------envoy 0.07c sidecar 0.00c------------------------------mean request 4.86 ms before 1.55 envoy 0.66 sidecar 0.77 worker 1.87 resume 0.73 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 2.95 ms before 1.05 envoy 0.67 sidecar 0.77 worker 0.45 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 305 pool 304------------------------------envoy 0.53c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 6.97 ms before 5.03 envoy 0.04 sidecar 1.42 worker 0.47 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 304------------------------------envoy 0.51c sidecar 0.50cwrk hottest 0.28c mean 0.07c------------------------------mean request 2.92 ms before 0.19 envoy 1.41 sidecar 0.79 worker 0.53 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1969 /sachieved 1969 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 116 pool 304------------------------------envoy 0.91c sidecar 0.73cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.25 ms before 1.25 envoy 0.73 sidecar 0.81 worker 0.46 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 17.0 pool 304------------------------------envoy 0.88c sidecar 0.86cwrk hottest 0.72c mean 0.11c------------------------------mean request 3.08 ms before 1.11 envoy 0.63 sidecar 0.80 worker 0.53 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2200 /sachieved 2200 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 27.0 pool 304------------------------------envoy 0.91c sidecar 0.81cwrk hottest 0.69c mean 0.11c------------------------------mean request 3.06 ms before 1.13 envoy 0.55 sidecar 0.80 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 30.0 pool 304------------------------------envoy 1.33c sidecar 0.96cwrk hottest 0.78c mean 0.17c------------------------------mean request 3.16 ms before 1.17 envoy 0.62 sidecar 0.79 worker 0.57 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 21.0 pool 304------------------------------envoy 1.26c sidecar 1.15cwrk hottest 0.79c mean 0.16c------------------------------mean request 3.24 ms before 1.23 envoy 0.59 sidecar 0.80 worker 0.62 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3054 /sachieved 3054 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 43.0 pool 304------------------------------envoy 1.29c sidecar 1.14cwrk hottest 0.82c mean 0.16c------------------------------mean request 3.29 ms before 1.24 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 47.0 pool 116------------------------------envoy 1.60c sidecar 1.31cwrk hottest 0.87c mean 0.20c------------------------------mean request 3.55 ms before 1.32 envoy 0.60 sidecar 0.79 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 39.0 pool 116------------------------------envoy 1.55c sidecar 1.38cwrk hottest 0.85c mean 0.19c------------------------------mean request 3.46 ms before 1.31 envoy 0.53 sidecar 0.80 worker 0.82 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4190 /sachieved 4189 /s (100.0% ok)p50 3.4 ms p95 6.4 msin-flight 52.0 pool 52.0------------------------------envoy 1.69c sidecar 1.35cwrk hottest 0.91c mean 0.20c------------------------------mean request 3.71 ms before 1.37 envoy 0.59 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.2 ms p95 6.1 msin-flight 52.0 pool 52.0------------------------------envoy 2.18c sidecar 1.52cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.55 ms before 1.32 envoy 0.68 sidecar 0.80 worker 0.74 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.1 ms p95 5.5 msin-flight 50.0 pool 54.0------------------------------envoy 2.22c sidecar 1.72cwrk hottest 0.89c mean 0.28c------------------------------mean request 3.39 ms before 1.21 envoy 0.69 sidecar 0.80 worker 0.69 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5151 /sachieved 5150 /s (100.0% ok)p50 3.3 ms p95 6.9 msin-flight 55.0 pool 61.0------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.91c mean 0.28c------------------------------mean request 3.78 ms before 1.38 envoy 0.70 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 7.0 msin-flight 127 pool 126------------------------------envoy 2.84c sidecar 1.95cwrk hottest 0.87c mean 0.36c------------------------------mean request 3.57 ms before 1.28 envoy 0.75 sidecar 0.83 worker 0.71 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.1 ms p95 8.8 msin-flight 74.0 pool 126------------------------------envoy 2.95c sidecar 0.00cwrk hottest 0.89c mean 0.37c------------------------------mean request 4.00 ms before 1.47 envoy 0.88 sidecar 0.84 worker 0.81 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 6.1 msin-flight 117 pool 126------------------------------envoy 2.82c sidecar 2.10cwrk hottest 0.86c mean 0.35c------------------------------mean request 3.50 ms before 1.21 envoy 0.76 sidecar 0.86 worker 0.68 resume 0.82 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6804 /sachieved 6803 /s (99.1% ok)p50 3.4 ms p95 9.1 msin-flight 1386 pool 1385------------------------------envoy 3.17c sidecar 2.11cwrk hottest 0.87c mean 0.40c------------------------------mean request 7.57 ms before 2.04 envoy 0.99 sidecar 2.96 worker 1.57 resume 2.95 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.2% ok)p50 3.3 ms p95 8.1 msin-flight 1349 pool 1385------------------------------envoy 3.29c sidecar 2.27cwrk hottest 0.81c mean 0.41c------------------------------mean request 7.89 ms before 1.56 envoy 1.38 sidecar 3.02 worker 1.93 resume 3.01 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.5% ok)p50 3.3 ms p95 7.5 msin-flight 1384 pool 1385------------------------------envoy 3.32c sidecar 0.00cwrk hottest 0.82c mean 0.41c------------------------------mean request 6.94 ms before 2.13 envoy 0.77 sidecar 2.52 worker 1.52 resume 2.49 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7494 /sachieved 7494 /s (100.0% ok)p50 3.5 ms p95 9.6 msin-flight 348 pool 1385------------------------------envoy 3.56c sidecar 2.28cwrk hottest 0.87c mean 0.44c------------------------------mean request 4.62 ms before 1.65 envoy 0.88 sidecar 0.93 worker 1.16 resume 0.90 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 4.4 ms p95 14.4 msin-flight 167 pool 1385------------------------------envoy 3.64c sidecar 2.44cwrk hottest 0.88c mean 0.46c------------------------------mean request 5.95 ms before 2.31 envoy 1.13 sidecar 0.90 worker 1.61 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 4.0 ms p95 13.6 msin-flight 153 pool 1385------------------------------envoy 3.69c sidecar 0.00cwrk hottest 0.87c mean 0.46c------------------------------mean request 5.55 ms before 2.10 envoy 1.07 sidecar 0.88 worker 1.50 resume 0.84 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8510 /sachieved 8486 /s (100.0% ok)p50 4.8 ms p95 19.5 msin-flight 6250 pool 6250------------------------------envoy 3.85c sidecar 2.46cwrk hottest 0.89c mean 0.48c------------------------------mean request 7.66 ms before 3.33 envoy 1.11 sidecar 1.14 worker 2.08 resume 1.02 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9036 /s (100.0% ok)p50 17.8 ms p95 661.4 msin-flight 6461 pool 6461------------------------------envoy 4.44c sidecar 0.00cwrk hottest 0.79c mean 0.54c------------------------------mean request 170.90 ms before 55.23 envoy 57.91 sidecar 1.81 worker 55.96 resume 1.45 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8994 /s (100.0% ok)p50 5.2 ms p95 38.1 msin-flight 320 pool 6461------------------------------envoy 4.10c sidecar 2.50cwrk hottest 0.87c mean 0.52c------------------------------mean request 11.11 ms before 3.42 envoy 2.72 sidecar 0.93 worker 4.04 resume 0.89 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9633 /sachieved 9631 /s (100.0% ok)p50 6.0 ms p95 62.5 msin-flight 467 pool 6461------------------------------envoy 4.35c sidecar 2.67cwrk hottest 0.87c mean 0.54c------------------------------mean request 14.97 ms before 6.16 envoy 2.91 sidecar 0.97 worker 4.93 resume 0.93 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 6.2 ms p95 43.3 msin-flight 578 pool 6461------------------------------envoy 4.54c sidecar 2.77cwrk hottest 0.84c mean 0.57c------------------------------mean request 12.94 ms before 5.30 envoy 2.48 sidecar 1.01 worker 4.14 resume 0.97 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 5.7 ms p95 41.1 msin-flight 445 pool 6461------------------------------envoy 4.55c sidecar 0.00cwrk hottest 0.83c mean 0.57c------------------------------mean request 10.90 ms before 3.88 envoy 2.37 sidecar 0.88 worker 3.77 resume 0.84 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 8.4 ms p95 62.4 msin-flight 855 pool 6461------------------------------envoy 4.72c sidecar 2.80cwrk hottest 0.83c mean 0.59c------------------------------mean request 17.81 ms before 7.45 envoy 3.53 sidecar 0.96 worker 5.87 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.2 ms p95 71.5 msin-flight 587 pool 6461------------------------------envoy 5.10c sidecar 2.91cwrk hottest 0.80c mean 0.64c------------------------------mean request 22.11 ms before 8.40 envoy 4.75 sidecar 0.99 worker 7.97 resume 0.95 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.8 ms p95 69.3 msin-flight 741 pool 2962------------------------------envoy 5.19c sidecar 3.00cwrk hottest 0.80c mean 0.65c------------------------------mean request 21.68 ms before 9.27 envoy 4.28 sidecar 0.99 worker 7.15 resume 0.94 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 18.6 ms p95 118.3 msin-flight 783 pool 854------------------------------envoy 5.30c sidecar 0.00cwrk hottest 0.78c mean 0.67c------------------------------mean request 33.30 ms before 14.46 envoy 6.61 sidecar 1.07 worker 11.15 resume 1.03 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 23.5 ms p95 118.8 msin-flight 1842 pool 1843------------------------------envoy 5.35c sidecar 2.98cwrk hottest 0.77c mean 0.68c------------------------------mean request 40.85 ms before 18.26 envoy 7.76 sidecar 1.51 worker 13.32 resume 1.47 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.8% ok)p50 22.1 ms p95 168.9 msin-flight 2426 pool 2426------------------------------envoy 5.64c sidecar 3.02cwrk hottest 0.76c mean 0.70c------------------------------mean request 46.31 ms before 16.01 envoy 10.95 sidecar 1.80 worker 17.55 resume 1.75 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 14.1 ms p95 239.5 msin-flight 6597 pool 6698------------------------------envoy 5.85c sidecar 2.95cwrk hottest 0.75c mean 0.73c------------------------------mean request 49.43 ms before 17.17 envoy 12.08 sidecar 1.40 worker 18.78 resume 1.34 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 996.9 ms p95 1514.0 msin-flight 21k pool 21k------------------------------envoy 5.88c sidecar 2.43cwrk hottest 0.74c mean 0.66c------------------------------mean request 983.18 ms before 233.22 envoy 607.22 sidecar 2.96 worker 139.78 resume 2.86 (control plane) +rung -1 0 qps setpoint------------------------------offered 83.9 /sachieved 83.7 /s (100.0% ok)p50 3.6 ms p95 11.2 msin-flight 24.0 pool 24.0------------------------------envoy 0.07c sidecar 0.00c------------------------------mean request 4.86 ms before 1.55 envoy 0.66 sidecar 0.77 worker 1.87 resume 0.73 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 2.95 ms before 1.05 envoy 0.67 sidecar 0.77 worker 0.45 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 305 pool 304------------------------------envoy 0.53c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 6.97 ms before 5.03 envoy 0.04 sidecar 1.42 worker 0.47 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 304------------------------------envoy 0.51c sidecar 0.50cwrk hottest 0.28c mean 0.07c------------------------------mean request 2.92 ms before 0.19 envoy 1.41 sidecar 0.79 worker 0.53 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1969 /sachieved 1969 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 116 pool 304------------------------------envoy 0.91c sidecar 0.73cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.25 ms before 1.25 envoy 0.73 sidecar 0.81 worker 0.46 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 17.0 pool 304------------------------------envoy 0.88c sidecar 0.86cwrk hottest 0.72c mean 0.11c------------------------------mean request 3.08 ms before 1.11 envoy 0.63 sidecar 0.80 worker 0.53 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2200 /sachieved 2200 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 27.0 pool 304------------------------------envoy 0.91c sidecar 0.81cwrk hottest 0.69c mean 0.11c------------------------------mean request 3.06 ms before 1.13 envoy 0.55 sidecar 0.80 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 30.0 pool 304------------------------------envoy 1.33c sidecar 0.96cwrk hottest 0.78c mean 0.17c------------------------------mean request 3.16 ms before 1.17 envoy 0.62 sidecar 0.79 worker 0.57 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 21.0 pool 304------------------------------envoy 1.26c sidecar 1.15cwrk hottest 0.79c mean 0.16c------------------------------mean request 3.24 ms before 1.23 envoy 0.59 sidecar 0.80 worker 0.62 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3054 /sachieved 3054 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 43.0 pool 304------------------------------envoy 1.29c sidecar 1.14cwrk hottest 0.82c mean 0.16c------------------------------mean request 3.29 ms before 1.24 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 47.0 pool 116------------------------------envoy 1.60c sidecar 1.31cwrk hottest 0.87c mean 0.20c------------------------------mean request 3.55 ms before 1.32 envoy 0.60 sidecar 0.79 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 39.0 pool 116------------------------------envoy 1.55c sidecar 1.38cwrk hottest 0.85c mean 0.19c------------------------------mean request 3.46 ms before 1.31 envoy 0.53 sidecar 0.80 worker 0.82 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4190 /sachieved 4189 /s (100.0% ok)p50 3.4 ms p95 6.4 msin-flight 52.0 pool 52.0------------------------------envoy 1.69c sidecar 1.35cwrk hottest 0.91c mean 0.20c------------------------------mean request 3.71 ms before 1.37 envoy 0.59 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.2 ms p95 6.1 msin-flight 52.0 pool 52.0------------------------------envoy 2.18c sidecar 1.52cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.55 ms before 1.32 envoy 0.68 sidecar 0.80 worker 0.74 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.1 ms p95 5.5 msin-flight 50.0 pool 54.0------------------------------envoy 2.22c sidecar 1.72cwrk hottest 0.89c mean 0.28c------------------------------mean request 3.39 ms before 1.21 envoy 0.69 sidecar 0.80 worker 0.69 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5151 /sachieved 5150 /s (100.0% ok)p50 3.3 ms p95 6.9 msin-flight 55.0 pool 61.0------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.91c mean 0.28c------------------------------mean request 3.78 ms before 1.38 envoy 0.70 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 7.0 msin-flight 127 pool 126------------------------------envoy 2.84c sidecar 1.95cwrk hottest 0.87c mean 0.36c------------------------------mean request 3.57 ms before 1.28 envoy 0.75 sidecar 0.83 worker 0.71 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.1 ms p95 8.8 msin-flight 74.0 pool 126------------------------------envoy 2.95c sidecar 0.00cwrk hottest 0.89c mean 0.37c------------------------------mean request 4.00 ms before 1.47 envoy 0.88 sidecar 0.84 worker 0.81 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 6.1 msin-flight 117 pool 126------------------------------envoy 2.82c sidecar 2.10cwrk hottest 0.86c mean 0.35c------------------------------mean request 3.50 ms before 1.21 envoy 0.76 sidecar 0.86 worker 0.68 resume 0.82 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6804 /sachieved 6803 /s (99.1% ok)p50 3.4 ms p95 9.1 msin-flight 1386 pool 1385------------------------------envoy 3.17c sidecar 2.11cwrk hottest 0.87c mean 0.40c------------------------------mean request 7.57 ms before 2.04 envoy 0.99 sidecar 2.96 worker 1.57 resume 2.95 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.2% ok)p50 3.3 ms p95 8.1 msin-flight 1349 pool 1385------------------------------envoy 3.29c sidecar 2.27cwrk hottest 0.81c mean 0.41c------------------------------mean request 7.89 ms before 1.56 envoy 1.38 sidecar 3.02 worker 1.93 resume 3.01 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.5% ok)p50 3.3 ms p95 7.5 msin-flight 1384 pool 1385------------------------------envoy 3.32c sidecar 0.00cwrk hottest 0.82c mean 0.41c------------------------------mean request 6.94 ms before 2.13 envoy 0.77 sidecar 2.52 worker 1.52 resume 2.49 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7494 /sachieved 7494 /s (100.0% ok)p50 3.5 ms p95 9.6 msin-flight 348 pool 1385------------------------------envoy 3.56c sidecar 2.28cwrk hottest 0.87c mean 0.44c------------------------------mean request 4.62 ms before 1.65 envoy 0.88 sidecar 0.93 worker 1.16 resume 0.90 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 4.4 ms p95 14.4 msin-flight 167 pool 1385------------------------------envoy 3.64c sidecar 2.44cwrk hottest 0.88c mean 0.46c------------------------------mean request 5.95 ms before 2.31 envoy 1.13 sidecar 0.90 worker 1.61 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 4.0 ms p95 13.6 msin-flight 153 pool 1385------------------------------envoy 3.69c sidecar 0.00cwrk hottest 0.87c mean 0.46c------------------------------mean request 5.55 ms before 2.10 envoy 1.07 sidecar 0.88 worker 1.50 resume 0.84 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8510 /sachieved 8486 /s (100.0% ok)p50 4.8 ms p95 19.5 msin-flight 6250 pool 6250------------------------------envoy 3.85c sidecar 2.46cwrk hottest 0.89c mean 0.48c------------------------------mean request 7.66 ms before 3.33 envoy 1.11 sidecar 1.14 worker 2.08 resume 1.02 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9036 /s (100.0% ok)p50 17.8 ms p95 661.4 msin-flight 6461 pool 6461------------------------------envoy 4.44c sidecar 0.00cwrk hottest 0.79c mean 0.54c------------------------------mean request 170.90 ms before 55.23 envoy 57.91 sidecar 1.81 worker 55.96 resume 1.45 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8994 /s (100.0% ok)p50 5.2 ms p95 38.1 msin-flight 320 pool 6461------------------------------envoy 4.10c sidecar 2.50cwrk hottest 0.87c mean 0.52c------------------------------mean request 11.11 ms before 3.42 envoy 2.72 sidecar 0.93 worker 4.04 resume 0.89 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9633 /sachieved 9631 /s (100.0% ok)p50 6.0 ms p95 62.5 msin-flight 467 pool 6461------------------------------envoy 4.35c sidecar 2.67cwrk hottest 0.87c mean 0.54c------------------------------mean request 14.97 ms before 6.16 envoy 2.91 sidecar 0.97 worker 4.93 resume 0.93 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 6.2 ms p95 43.3 msin-flight 578 pool 6461------------------------------envoy 4.54c sidecar 2.77cwrk hottest 0.84c mean 0.57c------------------------------mean request 12.94 ms before 5.30 envoy 2.48 sidecar 1.01 worker 4.14 resume 0.97 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 5.7 ms p95 41.1 msin-flight 445 pool 6461------------------------------envoy 4.55c sidecar 0.00cwrk hottest 0.83c mean 0.57c------------------------------mean request 10.90 ms before 3.88 envoy 2.37 sidecar 0.88 worker 3.77 resume 0.84 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 8.4 ms p95 62.4 msin-flight 855 pool 6461------------------------------envoy 4.72c sidecar 2.80cwrk hottest 0.83c mean 0.59c------------------------------mean request 17.81 ms before 7.45 envoy 3.53 sidecar 0.96 worker 5.87 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.2 ms p95 71.5 msin-flight 587 pool 6461------------------------------envoy 5.10c sidecar 2.91cwrk hottest 0.80c mean 0.64c------------------------------mean request 22.11 ms before 8.40 envoy 4.75 sidecar 0.99 worker 7.97 resume 0.95 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.8 ms p95 69.3 msin-flight 741 pool 2962------------------------------envoy 5.19c sidecar 3.00cwrk hottest 0.80c mean 0.65c------------------------------mean request 21.68 ms before 9.27 envoy 4.28 sidecar 0.99 worker 7.15 resume 0.94 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 18.6 ms p95 118.3 msin-flight 783 pool 854------------------------------envoy 5.30c sidecar 0.00cwrk hottest 0.78c mean 0.67c------------------------------mean request 33.30 ms before 14.46 envoy 6.61 sidecar 1.07 worker 11.15 resume 1.03 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 23.5 ms p95 118.8 msin-flight 1842 pool 1843------------------------------envoy 5.35c sidecar 2.98cwrk hottest 0.77c mean 0.68c------------------------------mean request 40.85 ms before 18.26 envoy 7.76 sidecar 1.51 worker 13.32 resume 1.47 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.8% ok)p50 22.1 ms p95 168.9 msin-flight 2426 pool 2426------------------------------envoy 5.64c sidecar 3.02cwrk hottest 0.76c mean 0.70c------------------------------mean request 46.31 ms before 16.01 envoy 10.95 sidecar 1.80 worker 17.55 resume 1.75 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 14.1 ms p95 239.5 msin-flight 6597 pool 6698------------------------------envoy 5.85c sidecar 2.95cwrk hottest 0.75c mean 0.73c------------------------------mean request 49.43 ms before 17.17 envoy 12.08 sidecar 1.40 worker 18.78 resume 1.34 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 996.9 ms p95 1514.0 msin-flight 21k pool 21k------------------------------envoy 5.88c sidecar 2.43cwrk hottest 0.74c mean 0.66c------------------------------mean request 983.18 ms before 233.22 envoy 607.22 sidecar 2.96 worker 139.78 resume 2.86 (control plane) +rung -1 0 qps setpoint------------------------------offered 83.9 /sachieved 83.7 /s (100.0% ok)p50 3.6 ms p95 11.2 msin-flight 24.0 pool 24.0------------------------------envoy 0.07c sidecar 0.00c------------------------------mean request 4.86 ms before 1.55 envoy 0.66 sidecar 0.77 worker 1.87 resume 0.73 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 2.95 ms before 1.05 envoy 0.67 sidecar 0.77 worker 0.45 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 305 pool 304------------------------------envoy 0.53c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 6.97 ms before 5.03 envoy 0.04 sidecar 1.42 worker 0.47 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 304------------------------------envoy 0.51c sidecar 0.50cwrk hottest 0.28c mean 0.07c------------------------------mean request 2.92 ms before 0.19 envoy 1.41 sidecar 0.79 worker 0.53 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1969 /sachieved 1969 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 116 pool 304------------------------------envoy 0.91c sidecar 0.73cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.25 ms before 1.25 envoy 0.73 sidecar 0.81 worker 0.46 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 17.0 pool 304------------------------------envoy 0.88c sidecar 0.86cwrk hottest 0.72c mean 0.11c------------------------------mean request 3.08 ms before 1.11 envoy 0.63 sidecar 0.80 worker 0.53 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2200 /sachieved 2200 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 27.0 pool 304------------------------------envoy 0.91c sidecar 0.81cwrk hottest 0.69c mean 0.11c------------------------------mean request 3.06 ms before 1.13 envoy 0.55 sidecar 0.80 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 30.0 pool 304------------------------------envoy 1.33c sidecar 0.96cwrk hottest 0.78c mean 0.17c------------------------------mean request 3.16 ms before 1.17 envoy 0.62 sidecar 0.79 worker 0.57 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 21.0 pool 304------------------------------envoy 1.26c sidecar 1.15cwrk hottest 0.79c mean 0.16c------------------------------mean request 3.24 ms before 1.23 envoy 0.59 sidecar 0.80 worker 0.62 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3054 /sachieved 3054 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 43.0 pool 304------------------------------envoy 1.29c sidecar 1.14cwrk hottest 0.82c mean 0.16c------------------------------mean request 3.29 ms before 1.24 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 47.0 pool 116------------------------------envoy 1.60c sidecar 1.31cwrk hottest 0.87c mean 0.20c------------------------------mean request 3.55 ms before 1.32 envoy 0.60 sidecar 0.79 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 39.0 pool 116------------------------------envoy 1.55c sidecar 1.38cwrk hottest 0.85c mean 0.19c------------------------------mean request 3.46 ms before 1.31 envoy 0.53 sidecar 0.80 worker 0.82 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4190 /sachieved 4189 /s (100.0% ok)p50 3.4 ms p95 6.4 msin-flight 52.0 pool 52.0------------------------------envoy 1.69c sidecar 1.35cwrk hottest 0.91c mean 0.20c------------------------------mean request 3.71 ms before 1.37 envoy 0.59 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.2 ms p95 6.1 msin-flight 52.0 pool 52.0------------------------------envoy 2.18c sidecar 1.52cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.55 ms before 1.32 envoy 0.68 sidecar 0.80 worker 0.74 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.1 ms p95 5.5 msin-flight 50.0 pool 54.0------------------------------envoy 2.22c sidecar 1.72cwrk hottest 0.89c mean 0.28c------------------------------mean request 3.39 ms before 1.21 envoy 0.69 sidecar 0.80 worker 0.69 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5151 /sachieved 5150 /s (100.0% ok)p50 3.3 ms p95 6.9 msin-flight 55.0 pool 61.0------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.91c mean 0.28c------------------------------mean request 3.78 ms before 1.38 envoy 0.70 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 7.0 msin-flight 127 pool 126------------------------------envoy 2.84c sidecar 1.95cwrk hottest 0.87c mean 0.36c------------------------------mean request 3.57 ms before 1.28 envoy 0.75 sidecar 0.83 worker 0.71 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.1 ms p95 8.8 msin-flight 74.0 pool 126------------------------------envoy 2.95c sidecar 0.00cwrk hottest 0.89c mean 0.37c------------------------------mean request 4.00 ms before 1.47 envoy 0.88 sidecar 0.84 worker 0.81 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 6.1 msin-flight 117 pool 126------------------------------envoy 2.82c sidecar 2.10cwrk hottest 0.86c mean 0.35c------------------------------mean request 3.50 ms before 1.21 envoy 0.76 sidecar 0.86 worker 0.68 resume 0.82 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6804 /sachieved 6803 /s (99.1% ok)p50 3.4 ms p95 9.1 msin-flight 1386 pool 1385------------------------------envoy 3.17c sidecar 2.11cwrk hottest 0.87c mean 0.40c------------------------------mean request 7.57 ms before 2.04 envoy 0.99 sidecar 2.96 worker 1.57 resume 2.95 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.2% ok)p50 3.3 ms p95 8.1 msin-flight 1349 pool 1385------------------------------envoy 3.29c sidecar 2.27cwrk hottest 0.81c mean 0.41c------------------------------mean request 7.89 ms before 1.56 envoy 1.38 sidecar 3.02 worker 1.93 resume 3.01 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.5% ok)p50 3.3 ms p95 7.5 msin-flight 1384 pool 1385------------------------------envoy 3.32c sidecar 0.00cwrk hottest 0.82c mean 0.41c------------------------------mean request 6.94 ms before 2.13 envoy 0.77 sidecar 2.52 worker 1.52 resume 2.49 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7494 /sachieved 7494 /s (100.0% ok)p50 3.5 ms p95 9.6 msin-flight 348 pool 1385------------------------------envoy 3.56c sidecar 2.28cwrk hottest 0.87c mean 0.44c------------------------------mean request 4.62 ms before 1.65 envoy 0.88 sidecar 0.93 worker 1.16 resume 0.90 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 4.4 ms p95 14.4 msin-flight 167 pool 1385------------------------------envoy 3.64c sidecar 2.44cwrk hottest 0.88c mean 0.46c------------------------------mean request 5.95 ms before 2.31 envoy 1.13 sidecar 0.90 worker 1.61 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 4.0 ms p95 13.6 msin-flight 153 pool 1385------------------------------envoy 3.69c sidecar 0.00cwrk hottest 0.87c mean 0.46c------------------------------mean request 5.55 ms before 2.10 envoy 1.07 sidecar 0.88 worker 1.50 resume 0.84 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8510 /sachieved 8486 /s (100.0% ok)p50 4.8 ms p95 19.5 msin-flight 6250 pool 6250------------------------------envoy 3.85c sidecar 2.46cwrk hottest 0.89c mean 0.48c------------------------------mean request 7.66 ms before 3.33 envoy 1.11 sidecar 1.14 worker 2.08 resume 1.02 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9036 /s (100.0% ok)p50 17.8 ms p95 661.4 msin-flight 6461 pool 6461------------------------------envoy 4.44c sidecar 0.00cwrk hottest 0.79c mean 0.54c------------------------------mean request 170.90 ms before 55.23 envoy 57.91 sidecar 1.81 worker 55.96 resume 1.45 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8994 /s (100.0% ok)p50 5.2 ms p95 38.1 msin-flight 320 pool 6461------------------------------envoy 4.10c sidecar 2.50cwrk hottest 0.87c mean 0.52c------------------------------mean request 11.11 ms before 3.42 envoy 2.72 sidecar 0.93 worker 4.04 resume 0.89 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9633 /sachieved 9631 /s (100.0% ok)p50 6.0 ms p95 62.5 msin-flight 467 pool 6461------------------------------envoy 4.35c sidecar 2.67cwrk hottest 0.87c mean 0.54c------------------------------mean request 14.97 ms before 6.16 envoy 2.91 sidecar 0.97 worker 4.93 resume 0.93 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 6.2 ms p95 43.3 msin-flight 578 pool 6461------------------------------envoy 4.54c sidecar 2.77cwrk hottest 0.84c mean 0.57c------------------------------mean request 12.94 ms before 5.30 envoy 2.48 sidecar 1.01 worker 4.14 resume 0.97 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 5.7 ms p95 41.1 msin-flight 445 pool 6461------------------------------envoy 4.55c sidecar 0.00cwrk hottest 0.83c mean 0.57c------------------------------mean request 10.90 ms before 3.88 envoy 2.37 sidecar 0.88 worker 3.77 resume 0.84 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 8.4 ms p95 62.4 msin-flight 855 pool 6461------------------------------envoy 4.72c sidecar 2.80cwrk hottest 0.83c mean 0.59c------------------------------mean request 17.81 ms before 7.45 envoy 3.53 sidecar 0.96 worker 5.87 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.2 ms p95 71.5 msin-flight 587 pool 6461------------------------------envoy 5.10c sidecar 2.91cwrk hottest 0.80c mean 0.64c------------------------------mean request 22.11 ms before 8.40 envoy 4.75 sidecar 0.99 worker 7.97 resume 0.95 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.8 ms p95 69.3 msin-flight 741 pool 2962------------------------------envoy 5.19c sidecar 3.00cwrk hottest 0.80c mean 0.65c------------------------------mean request 21.68 ms before 9.27 envoy 4.28 sidecar 0.99 worker 7.15 resume 0.94 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 18.6 ms p95 118.3 msin-flight 783 pool 854------------------------------envoy 5.30c sidecar 0.00cwrk hottest 0.78c mean 0.67c------------------------------mean request 33.30 ms before 14.46 envoy 6.61 sidecar 1.07 worker 11.15 resume 1.03 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 23.5 ms p95 118.8 msin-flight 1842 pool 1843------------------------------envoy 5.35c sidecar 2.98cwrk hottest 0.77c mean 0.68c------------------------------mean request 40.85 ms before 18.26 envoy 7.76 sidecar 1.51 worker 13.32 resume 1.47 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.8% ok)p50 22.1 ms p95 168.9 msin-flight 2426 pool 2426------------------------------envoy 5.64c sidecar 3.02cwrk hottest 0.76c mean 0.70c------------------------------mean request 46.31 ms before 16.01 envoy 10.95 sidecar 1.80 worker 17.55 resume 1.75 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 14.1 ms p95 239.5 msin-flight 6597 pool 6698------------------------------envoy 5.85c sidecar 2.95cwrk hottest 0.75c mean 0.73c------------------------------mean request 49.43 ms before 17.17 envoy 12.08 sidecar 1.40 worker 18.78 resume 1.34 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 996.9 ms p95 1514.0 msin-flight 21k pool 21k------------------------------envoy 5.88c sidecar 2.43cwrk hottest 0.74c mean 0.66c------------------------------mean request 983.18 ms before 233.22 envoy 607.22 sidecar 2.96 worker 139.78 resume 2.86 (control plane) +rung -1 0 qps setpoint------------------------------offered 83.9 /sachieved 83.7 /s (100.0% ok)p50 3.6 ms p95 11.2 msin-flight 24.0 pool 24.0------------------------------envoy 0.07c sidecar 0.00c------------------------------mean request 4.86 ms before 1.55 envoy 0.66 sidecar 0.77 worker 1.87 resume 0.73 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 2.95 ms before 1.05 envoy 0.67 sidecar 0.77 worker 0.45 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 305 pool 304------------------------------envoy 0.53c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 6.97 ms before 5.03 envoy 0.04 sidecar 1.42 worker 0.47 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 304------------------------------envoy 0.51c sidecar 0.50cwrk hottest 0.28c mean 0.07c------------------------------mean request 2.92 ms before 0.19 envoy 1.41 sidecar 0.79 worker 0.53 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1969 /sachieved 1969 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 116 pool 304------------------------------envoy 0.91c sidecar 0.73cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.25 ms before 1.25 envoy 0.73 sidecar 0.81 worker 0.46 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 17.0 pool 304------------------------------envoy 0.88c sidecar 0.86cwrk hottest 0.72c mean 0.11c------------------------------mean request 3.08 ms before 1.11 envoy 0.63 sidecar 0.80 worker 0.53 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2200 /sachieved 2200 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 27.0 pool 304------------------------------envoy 0.91c sidecar 0.81cwrk hottest 0.69c mean 0.11c------------------------------mean request 3.06 ms before 1.13 envoy 0.55 sidecar 0.80 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 30.0 pool 304------------------------------envoy 1.33c sidecar 0.96cwrk hottest 0.78c mean 0.17c------------------------------mean request 3.16 ms before 1.17 envoy 0.62 sidecar 0.79 worker 0.57 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 21.0 pool 304------------------------------envoy 1.26c sidecar 1.15cwrk hottest 0.79c mean 0.16c------------------------------mean request 3.24 ms before 1.23 envoy 0.59 sidecar 0.80 worker 0.62 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3054 /sachieved 3054 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 43.0 pool 304------------------------------envoy 1.29c sidecar 1.14cwrk hottest 0.82c mean 0.16c------------------------------mean request 3.29 ms before 1.24 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 47.0 pool 116------------------------------envoy 1.60c sidecar 1.31cwrk hottest 0.87c mean 0.20c------------------------------mean request 3.55 ms before 1.32 envoy 0.60 sidecar 0.79 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 39.0 pool 116------------------------------envoy 1.55c sidecar 1.38cwrk hottest 0.85c mean 0.19c------------------------------mean request 3.46 ms before 1.31 envoy 0.53 sidecar 0.80 worker 0.82 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4190 /sachieved 4189 /s (100.0% ok)p50 3.4 ms p95 6.4 msin-flight 52.0 pool 52.0------------------------------envoy 1.69c sidecar 1.35cwrk hottest 0.91c mean 0.20c------------------------------mean request 3.71 ms before 1.37 envoy 0.59 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.2 ms p95 6.1 msin-flight 52.0 pool 52.0------------------------------envoy 2.18c sidecar 1.52cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.55 ms before 1.32 envoy 0.68 sidecar 0.80 worker 0.74 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.1 ms p95 5.5 msin-flight 50.0 pool 54.0------------------------------envoy 2.22c sidecar 1.72cwrk hottest 0.89c mean 0.28c------------------------------mean request 3.39 ms before 1.21 envoy 0.69 sidecar 0.80 worker 0.69 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5151 /sachieved 5150 /s (100.0% ok)p50 3.3 ms p95 6.9 msin-flight 55.0 pool 61.0------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.91c mean 0.28c------------------------------mean request 3.78 ms before 1.38 envoy 0.70 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 7.0 msin-flight 127 pool 126------------------------------envoy 2.84c sidecar 1.95cwrk hottest 0.87c mean 0.36c------------------------------mean request 3.57 ms before 1.28 envoy 0.75 sidecar 0.83 worker 0.71 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.1 ms p95 8.8 msin-flight 74.0 pool 126------------------------------envoy 2.95c sidecar 0.00cwrk hottest 0.89c mean 0.37c------------------------------mean request 4.00 ms before 1.47 envoy 0.88 sidecar 0.84 worker 0.81 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 6.1 msin-flight 117 pool 126------------------------------envoy 2.82c sidecar 2.10cwrk hottest 0.86c mean 0.35c------------------------------mean request 3.50 ms before 1.21 envoy 0.76 sidecar 0.86 worker 0.68 resume 0.82 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6804 /sachieved 6803 /s (99.1% ok)p50 3.4 ms p95 9.1 msin-flight 1386 pool 1385------------------------------envoy 3.17c sidecar 2.11cwrk hottest 0.87c mean 0.40c------------------------------mean request 7.57 ms before 2.04 envoy 0.99 sidecar 2.96 worker 1.57 resume 2.95 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.2% ok)p50 3.3 ms p95 8.1 msin-flight 1349 pool 1385------------------------------envoy 3.29c sidecar 2.27cwrk hottest 0.81c mean 0.41c------------------------------mean request 7.89 ms before 1.56 envoy 1.38 sidecar 3.02 worker 1.93 resume 3.01 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.5% ok)p50 3.3 ms p95 7.5 msin-flight 1384 pool 1385------------------------------envoy 3.32c sidecar 0.00cwrk hottest 0.82c mean 0.41c------------------------------mean request 6.94 ms before 2.13 envoy 0.77 sidecar 2.52 worker 1.52 resume 2.49 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7494 /sachieved 7494 /s (100.0% ok)p50 3.5 ms p95 9.6 msin-flight 348 pool 1385------------------------------envoy 3.56c sidecar 2.28cwrk hottest 0.87c mean 0.44c------------------------------mean request 4.62 ms before 1.65 envoy 0.88 sidecar 0.93 worker 1.16 resume 0.90 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 4.4 ms p95 14.4 msin-flight 167 pool 1385------------------------------envoy 3.64c sidecar 2.44cwrk hottest 0.88c mean 0.46c------------------------------mean request 5.95 ms before 2.31 envoy 1.13 sidecar 0.90 worker 1.61 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 4.0 ms p95 13.6 msin-flight 153 pool 1385------------------------------envoy 3.69c sidecar 0.00cwrk hottest 0.87c mean 0.46c------------------------------mean request 5.55 ms before 2.10 envoy 1.07 sidecar 0.88 worker 1.50 resume 0.84 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8510 /sachieved 8486 /s (100.0% ok)p50 4.8 ms p95 19.5 msin-flight 6250 pool 6250------------------------------envoy 3.85c sidecar 2.46cwrk hottest 0.89c mean 0.48c------------------------------mean request 7.66 ms before 3.33 envoy 1.11 sidecar 1.14 worker 2.08 resume 1.02 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9036 /s (100.0% ok)p50 17.8 ms p95 661.4 msin-flight 6461 pool 6461------------------------------envoy 4.44c sidecar 0.00cwrk hottest 0.79c mean 0.54c------------------------------mean request 170.90 ms before 55.23 envoy 57.91 sidecar 1.81 worker 55.96 resume 1.45 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8994 /s (100.0% ok)p50 5.2 ms p95 38.1 msin-flight 320 pool 6461------------------------------envoy 4.10c sidecar 2.50cwrk hottest 0.87c mean 0.52c------------------------------mean request 11.11 ms before 3.42 envoy 2.72 sidecar 0.93 worker 4.04 resume 0.89 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9633 /sachieved 9631 /s (100.0% ok)p50 6.0 ms p95 62.5 msin-flight 467 pool 6461------------------------------envoy 4.35c sidecar 2.67cwrk hottest 0.87c mean 0.54c------------------------------mean request 14.97 ms before 6.16 envoy 2.91 sidecar 0.97 worker 4.93 resume 0.93 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 6.2 ms p95 43.3 msin-flight 578 pool 6461------------------------------envoy 4.54c sidecar 2.77cwrk hottest 0.84c mean 0.57c------------------------------mean request 12.94 ms before 5.30 envoy 2.48 sidecar 1.01 worker 4.14 resume 0.97 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 5.7 ms p95 41.1 msin-flight 445 pool 6461------------------------------envoy 4.55c sidecar 0.00cwrk hottest 0.83c mean 0.57c------------------------------mean request 10.90 ms before 3.88 envoy 2.37 sidecar 0.88 worker 3.77 resume 0.84 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 8.4 ms p95 62.4 msin-flight 855 pool 6461------------------------------envoy 4.72c sidecar 2.80cwrk hottest 0.83c mean 0.59c------------------------------mean request 17.81 ms before 7.45 envoy 3.53 sidecar 0.96 worker 5.87 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.2 ms p95 71.5 msin-flight 587 pool 6461------------------------------envoy 5.10c sidecar 2.91cwrk hottest 0.80c mean 0.64c------------------------------mean request 22.11 ms before 8.40 envoy 4.75 sidecar 0.99 worker 7.97 resume 0.95 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.8 ms p95 69.3 msin-flight 741 pool 2962------------------------------envoy 5.19c sidecar 3.00cwrk hottest 0.80c mean 0.65c------------------------------mean request 21.68 ms before 9.27 envoy 4.28 sidecar 0.99 worker 7.15 resume 0.94 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 18.6 ms p95 118.3 msin-flight 783 pool 854------------------------------envoy 5.30c sidecar 0.00cwrk hottest 0.78c mean 0.67c------------------------------mean request 33.30 ms before 14.46 envoy 6.61 sidecar 1.07 worker 11.15 resume 1.03 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 23.5 ms p95 118.8 msin-flight 1842 pool 1843------------------------------envoy 5.35c sidecar 2.98cwrk hottest 0.77c mean 0.68c------------------------------mean request 40.85 ms before 18.26 envoy 7.76 sidecar 1.51 worker 13.32 resume 1.47 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.8% ok)p50 22.1 ms p95 168.9 msin-flight 2426 pool 2426------------------------------envoy 5.64c sidecar 3.02cwrk hottest 0.76c mean 0.70c------------------------------mean request 46.31 ms before 16.01 envoy 10.95 sidecar 1.80 worker 17.55 resume 1.75 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 14.1 ms p95 239.5 msin-flight 6597 pool 6698------------------------------envoy 5.85c sidecar 2.95cwrk hottest 0.75c mean 0.73c------------------------------mean request 49.43 ms before 17.17 envoy 12.08 sidecar 1.40 worker 18.78 resume 1.34 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 996.9 ms p95 1514.0 msin-flight 21k pool 21k------------------------------envoy 5.88c sidecar 2.43cwrk hottest 0.74c mean 0.66c------------------------------mean request 983.18 ms before 233.22 envoy 607.22 sidecar 2.96 worker 139.78 resume 2.86 (control plane) +rung -1 0 qps setpoint------------------------------offered 83.9 /sachieved 83.7 /s (100.0% ok)p50 3.6 ms p95 11.2 msin-flight 24.0 pool 24.0------------------------------envoy 0.07c sidecar 0.00c------------------------------mean request 4.86 ms before 1.55 envoy 0.66 sidecar 0.77 worker 1.87 resume 0.73 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 2.95 ms before 1.05 envoy 0.67 sidecar 0.77 worker 0.45 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 305 pool 304------------------------------envoy 0.53c sidecar 0.49cwrk hottest 0.32c mean 0.06c------------------------------mean request 6.97 ms before 5.03 envoy 0.04 sidecar 1.42 worker 0.47 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 9.0 pool 304------------------------------envoy 0.51c sidecar 0.50cwrk hottest 0.28c mean 0.07c------------------------------mean request 2.92 ms before 0.19 envoy 1.41 sidecar 0.79 worker 0.53 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1969 /sachieved 1969 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 116 pool 304------------------------------envoy 0.91c sidecar 0.73cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.25 ms before 1.25 envoy 0.73 sidecar 0.81 worker 0.46 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 17.0 pool 304------------------------------envoy 0.88c sidecar 0.86cwrk hottest 0.72c mean 0.11c------------------------------mean request 3.08 ms before 1.11 envoy 0.63 sidecar 0.80 worker 0.53 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2200 /sachieved 2200 /s (100.0% ok)p50 3.0 ms p95 4.1 msin-flight 27.0 pool 304------------------------------envoy 0.91c sidecar 0.81cwrk hottest 0.69c mean 0.11c------------------------------mean request 3.06 ms before 1.13 envoy 0.55 sidecar 0.80 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 30.0 pool 304------------------------------envoy 1.33c sidecar 0.96cwrk hottest 0.78c mean 0.17c------------------------------mean request 3.16 ms before 1.17 envoy 0.62 sidecar 0.79 worker 0.57 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 21.0 pool 304------------------------------envoy 1.26c sidecar 1.15cwrk hottest 0.79c mean 0.16c------------------------------mean request 3.24 ms before 1.23 envoy 0.59 sidecar 0.80 worker 0.62 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3054 /sachieved 3054 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 43.0 pool 304------------------------------envoy 1.29c sidecar 1.14cwrk hottest 0.82c mean 0.16c------------------------------mean request 3.29 ms before 1.24 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 47.0 pool 116------------------------------envoy 1.60c sidecar 1.31cwrk hottest 0.87c mean 0.20c------------------------------mean request 3.55 ms before 1.32 envoy 0.60 sidecar 0.79 worker 0.83 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 39.0 pool 116------------------------------envoy 1.55c sidecar 1.38cwrk hottest 0.85c mean 0.19c------------------------------mean request 3.46 ms before 1.31 envoy 0.53 sidecar 0.80 worker 0.82 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4190 /sachieved 4189 /s (100.0% ok)p50 3.4 ms p95 6.4 msin-flight 52.0 pool 52.0------------------------------envoy 1.69c sidecar 1.35cwrk hottest 0.91c mean 0.20c------------------------------mean request 3.71 ms before 1.37 envoy 0.59 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 3.2 ms p95 6.1 msin-flight 52.0 pool 52.0------------------------------envoy 2.18c sidecar 1.52cwrk hottest 0.89c mean 0.27c------------------------------mean request 3.55 ms before 1.32 envoy 0.68 sidecar 0.80 worker 0.74 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.1 ms p95 5.5 msin-flight 50.0 pool 54.0------------------------------envoy 2.22c sidecar 1.72cwrk hottest 0.89c mean 0.28c------------------------------mean request 3.39 ms before 1.21 envoy 0.69 sidecar 0.80 worker 0.69 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5151 /sachieved 5150 /s (100.0% ok)p50 3.3 ms p95 6.9 msin-flight 55.0 pool 61.0------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.91c mean 0.28c------------------------------mean request 3.78 ms before 1.38 envoy 0.70 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 7.0 msin-flight 127 pool 126------------------------------envoy 2.84c sidecar 1.95cwrk hottest 0.87c mean 0.36c------------------------------mean request 3.57 ms before 1.28 envoy 0.75 sidecar 0.83 worker 0.71 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.1 ms p95 8.8 msin-flight 74.0 pool 126------------------------------envoy 2.95c sidecar 0.00cwrk hottest 0.89c mean 0.37c------------------------------mean request 4.00 ms before 1.47 envoy 0.88 sidecar 0.84 worker 0.81 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.0 ms p95 6.1 msin-flight 117 pool 126------------------------------envoy 2.82c sidecar 2.10cwrk hottest 0.86c mean 0.35c------------------------------mean request 3.50 ms before 1.21 envoy 0.76 sidecar 0.86 worker 0.68 resume 0.82 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6804 /sachieved 6803 /s (99.1% ok)p50 3.4 ms p95 9.1 msin-flight 1386 pool 1385------------------------------envoy 3.17c sidecar 2.11cwrk hottest 0.87c mean 0.40c------------------------------mean request 7.57 ms before 2.04 envoy 0.99 sidecar 2.96 worker 1.57 resume 2.95 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.2% ok)p50 3.3 ms p95 8.1 msin-flight 1349 pool 1385------------------------------envoy 3.29c sidecar 2.27cwrk hottest 0.81c mean 0.41c------------------------------mean request 7.89 ms before 1.56 envoy 1.38 sidecar 3.02 worker 1.93 resume 3.01 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.5% ok)p50 3.3 ms p95 7.5 msin-flight 1384 pool 1385------------------------------envoy 3.32c sidecar 0.00cwrk hottest 0.82c mean 0.41c------------------------------mean request 6.94 ms before 2.13 envoy 0.77 sidecar 2.52 worker 1.52 resume 2.49 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7494 /sachieved 7494 /s (100.0% ok)p50 3.5 ms p95 9.6 msin-flight 348 pool 1385------------------------------envoy 3.56c sidecar 2.28cwrk hottest 0.87c mean 0.44c------------------------------mean request 4.62 ms before 1.65 envoy 0.88 sidecar 0.93 worker 1.16 resume 0.90 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 4.4 ms p95 14.4 msin-flight 167 pool 1385------------------------------envoy 3.64c sidecar 2.44cwrk hottest 0.88c mean 0.46c------------------------------mean request 5.95 ms before 2.31 envoy 1.13 sidecar 0.90 worker 1.61 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 4.0 ms p95 13.6 msin-flight 153 pool 1385------------------------------envoy 3.69c sidecar 0.00cwrk hottest 0.87c mean 0.46c------------------------------mean request 5.55 ms before 2.10 envoy 1.07 sidecar 0.88 worker 1.50 resume 0.84 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8510 /sachieved 8486 /s (100.0% ok)p50 4.8 ms p95 19.5 msin-flight 6250 pool 6250------------------------------envoy 3.85c sidecar 2.46cwrk hottest 0.89c mean 0.48c------------------------------mean request 7.66 ms before 3.33 envoy 1.11 sidecar 1.14 worker 2.08 resume 1.02 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9036 /s (100.0% ok)p50 17.8 ms p95 661.4 msin-flight 6461 pool 6461------------------------------envoy 4.44c sidecar 0.00cwrk hottest 0.79c mean 0.54c------------------------------mean request 170.90 ms before 55.23 envoy 57.91 sidecar 1.81 worker 55.96 resume 1.45 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8994 /s (100.0% ok)p50 5.2 ms p95 38.1 msin-flight 320 pool 6461------------------------------envoy 4.10c sidecar 2.50cwrk hottest 0.87c mean 0.52c------------------------------mean request 11.11 ms before 3.42 envoy 2.72 sidecar 0.93 worker 4.04 resume 0.89 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9633 /sachieved 9631 /s (100.0% ok)p50 6.0 ms p95 62.5 msin-flight 467 pool 6461------------------------------envoy 4.35c sidecar 2.67cwrk hottest 0.87c mean 0.54c------------------------------mean request 14.97 ms before 6.16 envoy 2.91 sidecar 0.97 worker 4.93 resume 0.93 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 6.2 ms p95 43.3 msin-flight 578 pool 6461------------------------------envoy 4.54c sidecar 2.77cwrk hottest 0.84c mean 0.57c------------------------------mean request 12.94 ms before 5.30 envoy 2.48 sidecar 1.01 worker 4.14 resume 0.97 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 5.7 ms p95 41.1 msin-flight 445 pool 6461------------------------------envoy 4.55c sidecar 0.00cwrk hottest 0.83c mean 0.57c------------------------------mean request 10.90 ms before 3.88 envoy 2.37 sidecar 0.88 worker 3.77 resume 0.84 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 10k /s (100.0% ok)p50 8.4 ms p95 62.4 msin-flight 855 pool 6461------------------------------envoy 4.72c sidecar 2.80cwrk hottest 0.83c mean 0.59c------------------------------mean request 17.81 ms before 7.45 envoy 3.53 sidecar 0.96 worker 5.87 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.2 ms p95 71.5 msin-flight 587 pool 6461------------------------------envoy 5.10c sidecar 2.91cwrk hottest 0.80c mean 0.64c------------------------------mean request 22.11 ms before 8.40 envoy 4.75 sidecar 0.99 worker 7.97 resume 0.95 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 11.8 ms p95 69.3 msin-flight 741 pool 2962------------------------------envoy 5.19c sidecar 3.00cwrk hottest 0.80c mean 0.65c------------------------------mean request 21.68 ms before 9.27 envoy 4.28 sidecar 0.99 worker 7.15 resume 0.94 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 18.6 ms p95 118.3 msin-flight 783 pool 854------------------------------envoy 5.30c sidecar 0.00cwrk hottest 0.78c mean 0.67c------------------------------mean request 33.30 ms before 14.46 envoy 6.61 sidecar 1.07 worker 11.15 resume 1.03 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 23.5 ms p95 118.8 msin-flight 1842 pool 1843------------------------------envoy 5.35c sidecar 2.98cwrk hottest 0.77c mean 0.68c------------------------------mean request 40.85 ms before 18.26 envoy 7.76 sidecar 1.51 worker 13.32 resume 1.47 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.8% ok)p50 22.1 ms p95 168.9 msin-flight 2426 pool 2426------------------------------envoy 5.64c sidecar 3.02cwrk hottest 0.76c mean 0.70c------------------------------mean request 46.31 ms before 16.01 envoy 10.95 sidecar 1.80 worker 17.55 resume 1.75 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 14.1 ms p95 239.5 msin-flight 6597 pool 6698------------------------------envoy 5.85c sidecar 2.95cwrk hottest 0.75c mean 0.73c------------------------------mean request 49.43 ms before 17.17 envoy 12.08 sidecar 1.40 worker 18.78 resume 1.34 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 996.9 ms p95 1514.0 msin-flight 21k pool 21k------------------------------envoy 5.88c sidecar 2.43cwrk hottest 0.74c mean 0.66c------------------------------mean request 983.18 ms before 233.22 envoy 607.22 sidecar 2.96 worker 139.78 resume 2.86 (control plane) +
+ + + +atenet-router capacity — 8c/16t +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 +R9 +R10 +R11 +R12 +R13 + +0 + +10k + +20k + +30k +requests / s + + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + + + + + + +0 + +2.0 + +4.0 + +6.0 + +8.0 +cores + + + + + + + + +envoy CPU limit, 8 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + + + + + + +0 + +500.0M + +1.0G + +1.5G + +2.0G +bytes + +100 + +200 + +300 + +400 + +500 + +600 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 0 /sachieved 0 /s (0.0% ok)p50 0.0 ms p95 0.0 msin-flight 30.0 pool 30.0------------------------------envoy 0.01c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 989 /sachieved 989 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 9.0 pool 30.0------------------------------envoy 0.56c sidecar 0.19cwrk hottest 0.33c mean 0.03c------------------------------mean request 3.23 ms before 1.01 envoy 0.69 sidecar 0.80 worker 0.74 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 6.0 pool 30.0------------------------------envoy 0.54c sidecar 0.50cwrk hottest 0.25c mean 0.03c------------------------------mean request 2.98 ms before 0.98 envoy 0.73 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1113 /sachieved 1113 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 12.0 pool 30.0------------------------------envoy 0.61c sidecar 0.53cwrk hottest 0.27c mean 0.04c------------------------------mean request 2.98 ms before 0.98 envoy 0.76 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 51.0 pool 51.0------------------------------envoy 1.11c sidecar 0.91cwrk hottest 0.46c mean 0.07c------------------------------mean request 3.00 ms before 0.99 envoy 0.80 sidecar 0.82 worker 0.40 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 731 pool 731------------------------------envoy 1.09c sidecar 0.93cwrk hottest 0.51c mean 0.06c------------------------------mean request 8.23 ms before 3.86 envoy 2.21 sidecar 1.44 worker 0.73 resume 1.26 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2264 /sachieved 2264 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 135 pool 731------------------------------envoy 1.10c sidecar 0.88cwrk hottest 0.76c mean 0.07c------------------------------mean request 3.31 ms before 1.27 envoy 0.80 sidecar 0.81 worker 0.43 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 21.0 pool 731------------------------------envoy 1.37c sidecar 1.15cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.05 ms before 1.10 envoy 0.64 sidecar 0.79 worker 0.52 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 272 pool 731------------------------------envoy 1.42c sidecar 1.17cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.82 ms before 1.31 envoy 0.89 sidecar 0.94 worker 0.67 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3315 /sachieved 3315 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 53.0 pool 731------------------------------envoy 1.50c sidecar 1.19cwrk hottest 0.80c mean 0.10c------------------------------mean request 3.17 ms before 1.16 envoy 0.60 sidecar 0.79 worker 0.62 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 44.0 pool 731------------------------------envoy 1.84c sidecar 1.42cwrk hottest 0.73c mean 0.11c------------------------------mean request 3.23 ms before 1.17 envoy 0.60 sidecar 0.80 worker 0.66 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 31.9 msin-flight 989 pool 988------------------------------envoy 1.88c sidecar 1.41cwrk hottest 0.77c mean 0.12c------------------------------mean request 10.42 ms before 4.90 envoy 1.59 sidecar 1.77 worker 2.17 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4368 /sachieved 4368 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 378 pool 988------------------------------envoy 2.12c sidecar 1.55cwrk hottest 0.81c mean 0.13c------------------------------mean request 3.13 ms before 0.83 envoy 0.68 sidecar 1.02 worker 0.60 resume 0.98 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 42.0 pool 988------------------------------envoy 2.67c sidecar 0.00cwrk hottest 0.69c mean 0.17c------------------------------mean request 3.33 ms before 1.34 envoy 0.73 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 894 pool 988------------------------------envoy 2.57c sidecar 1.87cwrk hottest 0.70c mean 0.16c------------------------------mean request 5.20 ms before 1.71 envoy 0.65 sidecar 2.04 worker 0.80 resume 2.00 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 45.0 pool 988------------------------------envoy 2.46c sidecar 1.80cwrk hottest 0.81c mean 0.16c------------------------------mean request 3.19 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.57 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5906 /sachieved 5905 /s (100.0% ok)p50 3.1 ms p95 5.1 msin-flight 68.0 pool 988------------------------------envoy 2.94c sidecar 2.00cwrk hottest 0.75c mean 0.19c------------------------------mean request 3.38 ms before 1.21 envoy 0.68 sidecar 0.82 worker 0.68 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.3 ms p95 6.1 msin-flight 99.0 pool 988------------------------------envoy 2.85c sidecar 2.04cwrk hottest 0.87c mean 0.18c------------------------------mean request 3.67 ms before 1.34 envoy 0.72 sidecar 0.82 worker 0.80 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.3 ms p95 54.8 msin-flight 1719 pool 1718------------------------------envoy 2.99c sidecar 2.06cwrk hottest 0.85c mean 0.18c------------------------------mean request 11.38 ms before 5.48 envoy 2.25 sidecar 1.05 worker 2.61 resume 0.98 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6800 /sachieved 6800 /s (100.0% ok)p50 3.2 ms p95 9.6 msin-flight 2390 pool 2389------------------------------envoy 3.77c sidecar 2.19cwrk hottest 0.79c mean 0.23c------------------------------mean request 10.22 ms before 4.50 envoy 2.32 sidecar 0.89 worker 2.50 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.3% ok)p50 3.5 ms p95 19.2 msin-flight 1327 pool 2389------------------------------envoy 3.46c sidecar 2.39cwrk hottest 0.90c mean 0.22c------------------------------mean request 8.49 ms before 1.57 envoy 1.82 sidecar 2.94 worker 2.16 resume 2.92 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (98.5% ok)p50 3.2 ms p95 16.1 msin-flight 1249 pool 2389------------------------------envoy 3.59c sidecar 2.23cwrk hottest 0.87c mean 0.22c------------------------------mean request 9.64 ms before 1.65 envoy 1.80 sidecar 3.92 worker 2.26 resume 3.94 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7339 /sachieved 7336 /s (100.0% ok)p50 3.4 ms p95 27.5 msin-flight 237 pool 2389------------------------------envoy 3.77c sidecar 2.35cwrk hottest 0.88c mean 0.24c------------------------------mean request 6.89 ms before 3.15 envoy 1.22 sidecar 0.90 worker 1.62 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (98.7% ok)p50 3.5 ms p95 32.0 msin-flight 1355 pool 2389------------------------------envoy 4.24c sidecar 2.57cwrk hottest 0.85c mean 0.27c------------------------------mean request 11.29 ms before 3.37 envoy 1.80 sidecar 3.14 worker 2.98 resume 3.14 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 3.6 ms p95 21.4 msin-flight 1266 pool 2389------------------------------envoy 4.25c sidecar 2.64cwrk hottest 0.84c mean 0.26c------------------------------mean request 9.04 ms before 2.41 envoy 1.92 sidecar 2.06 worker 2.66 resume 2.01 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8002 /s (100.0% ok)p50 3.6 ms p95 19.7 msin-flight 168 pool 2389------------------------------envoy 4.12c sidecar 2.61cwrk hottest 0.85c mean 0.26c------------------------------mean request 6.57 ms before 2.57 envoy 1.33 sidecar 0.85 worker 1.81 resume 0.81 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8785 /sachieved 8779 /s (100.0% ok)p50 3.9 ms p95 29.8 msin-flight 1169 pool 2389------------------------------envoy 4.76c sidecar 0.00cwrk hottest 0.84c mean 0.29c------------------------------mean request 9.29 ms before 2.93 envoy 2.00 sidecar 1.48 worker 2.88 resume 1.44 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8999 /s (100.0% ok)p50 3.7 ms p95 40.4 msin-flight 177 pool 2389------------------------------envoy 4.89c sidecar 2.84cwrk hottest 0.81c mean 0.30c------------------------------mean request 9.81 ms before 3.75 envoy 2.03 sidecar 0.90 worker 3.13 resume 0.86 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9001 /s (100.0% ok)p50 3.8 ms p95 37.4 msin-flight 180 pool 1354------------------------------envoy 4.93c sidecar 2.86cwrk hottest 0.80c mean 0.31c------------------------------mean request 9.67 ms before 3.84 envoy 1.98 sidecar 0.87 worker 2.98 resume 0.83 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9956 /sachieved 9957 /s (100.0% ok)p50 4.5 ms p95 23.9 msin-flight 422 pool 1354------------------------------envoy 5.42c sidecar 3.00cwrk hottest 0.76c mean 0.34c------------------------------mean request 7.42 ms before 2.68 envoy 1.57 sidecar 0.91 worker 2.26 resume 0.87 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9387 /s (100.0% ok)p50 4.7 ms p95 243.7 msin-flight 11k pool 11k------------------------------envoy 5.39c sidecar 3.06cwrk hottest 0.77c mean 0.35c------------------------------mean request 38.76 ms before 21.54 envoy 5.51 sidecar 1.47 worker 10.25 resume 1.15 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 11k /s (100.0% ok)p50 13.9 ms p95 999.8 msin-flight 9138 pool 11k------------------------------envoy 6.16c sidecar 2.69cwrk hottest 0.75c mean 0.37c------------------------------mean request 198.32 ms before 64.26 envoy 60.01 sidecar 1.38 worker 72.67 resume 1.14 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.9 ms p95 57.2 msin-flight 623 pool 11k------------------------------envoy 6.25c sidecar 3.19cwrk hottest 0.69c mean 0.39c------------------------------mean request 17.71 ms before 6.41 envoy 4.20 sidecar 0.97 worker 6.14 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 8.4 ms p95 86.7 msin-flight 1422 pool 11k------------------------------envoy 6.25c sidecar 3.24cwrk hottest 0.68c mean 0.39c------------------------------mean request 23.07 ms before 10.16 envoy 4.52 sidecar 1.41 worker 6.98 resume 1.36 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 9.2 ms p95 68.5 msin-flight 1078 pool 11k------------------------------envoy 6.17c sidecar 3.21cwrk hottest 0.71c mean 0.39c------------------------------mean request 20.75 ms before 7.83 envoy 4.75 sidecar 1.17 worker 7.00 resume 1.13 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 20.1 ms p95 124.5 msin-flight 1575 pool 11k------------------------------envoy 6.72c sidecar 3.24cwrk hottest 0.70c mean 0.42c------------------------------mean request 35.88 ms before 12.63 envoy 8.60 sidecar 1.11 worker 13.54 resume 1.06 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 38.3 ms p95 233.7 msin-flight 1917 pool 11k------------------------------envoy 7.32c sidecar 3.30cwrk hottest 0.57c mean 0.46c------------------------------mean request 63.46 ms before 29.12 envoy 13.01 sidecar 1.15 worker 20.19 resume 1.10 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 30.1 ms p95 179.4 msin-flight 1590 pool 11k------------------------------envoy 7.23c sidecar 0.00cwrk hottest 0.56c mean 0.45c------------------------------mean request 51.16 ms before 14.34 envoy 14.00 sidecar 1.07 worker 21.75 resume 1.02 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 68.4 ms p95 632.6 msin-flight 7146 pool 11k------------------------------envoy 7.37c sidecar 3.20cwrk hottest 0.60c mean 0.47c------------------------------mean request 189.77 ms before 73.60 envoy 46.04 sidecar 2.09 worker 68.04 resume 2.03 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 110.5 ms p95 457.8 msin-flight 3171 pool 8090------------------------------envoy 7.53c sidecar 0.00cwrk hottest 0.55c mean 0.47c------------------------------mean request 158.04 ms before 59.68 envoy 38.34 sidecar 1.48 worker 58.53 resume 1.43 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (99.8% ok)p50 95.6 ms p95 407.3 msin-flight 3687 pool 6847------------------------------envoy 7.52c sidecar 3.04cwrk hottest 0.55c mean 0.47c------------------------------mean request 137.03 ms before 57.83 envoy 29.63 sidecar 2.50 worker 47.07 resume 2.45 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 82.5 ms p95 391.6 msin-flight 3745 pool 6847------------------------------envoy 7.63c sidecar 3.03cwrk hottest 0.55c mean 0.47c------------------------------mean request 135.56 ms before 49.08 envoy 33.38 sidecar 1.42 worker 51.67 resume 1.37 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 13k /s (100.0% ok)p50 278.3 ms p95 699.1 msin-flight 10k pool 10k------------------------------envoy 7.93c sidecar 3.04cwrk hottest 0.51c mean 0.50c------------------------------mean request 307.88 ms before 149.09 envoy 63.31 sidecar 2.02 worker 93.46 resume 1.95 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 13k /s (99.5% ok)p50 1244.6 ms p95 1743.3 msin-flight 24k pool 24k------------------------------envoy 7.94c sidecar 2.43cwrk hottest 0.52c mean 0.50c------------------------------mean request 1248.97 ms before 362.49 envoy 646.84 sidecar 3.27 worker 236.37 resume 3.08 (control plane) +rung 13 14k qps setpoint------------------------------offered 9567 /sachieved 11k /s (93.1% ok)p50 1698.9 ms p95 1976.8 msin-flight 24k pool 23k------------------------------envoy 6.03c sidecar 0.00cwrk hottest 0.51c mean 0.34c------------------------------mean request 1639.68 ms before 226.18 envoy 1159.92 sidecar 3.23 worker 250.35 resume 3.08 (control plane) +rung -1 0 qps setpoint------------------------------offered 0 /sachieved 0 /s (0.0% ok)p50 0.0 ms p95 0.0 msin-flight 30.0 pool 30.0------------------------------envoy 0.01c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 989 /sachieved 989 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 9.0 pool 30.0------------------------------envoy 0.56c sidecar 0.19cwrk hottest 0.33c mean 0.03c------------------------------mean request 3.23 ms before 1.01 envoy 0.69 sidecar 0.80 worker 0.74 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 6.0 pool 30.0------------------------------envoy 0.54c sidecar 0.50cwrk hottest 0.25c mean 0.03c------------------------------mean request 2.98 ms before 0.98 envoy 0.73 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1113 /sachieved 1113 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 12.0 pool 30.0------------------------------envoy 0.61c sidecar 0.53cwrk hottest 0.27c mean 0.04c------------------------------mean request 2.98 ms before 0.98 envoy 0.76 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 51.0 pool 51.0------------------------------envoy 1.11c sidecar 0.91cwrk hottest 0.46c mean 0.07c------------------------------mean request 3.00 ms before 0.99 envoy 0.80 sidecar 0.82 worker 0.40 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 731 pool 731------------------------------envoy 1.09c sidecar 0.93cwrk hottest 0.51c mean 0.06c------------------------------mean request 8.23 ms before 3.86 envoy 2.21 sidecar 1.44 worker 0.73 resume 1.26 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2264 /sachieved 2264 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 135 pool 731------------------------------envoy 1.10c sidecar 0.88cwrk hottest 0.76c mean 0.07c------------------------------mean request 3.31 ms before 1.27 envoy 0.80 sidecar 0.81 worker 0.43 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 21.0 pool 731------------------------------envoy 1.37c sidecar 1.15cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.05 ms before 1.10 envoy 0.64 sidecar 0.79 worker 0.52 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 272 pool 731------------------------------envoy 1.42c sidecar 1.17cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.82 ms before 1.31 envoy 0.89 sidecar 0.94 worker 0.67 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3315 /sachieved 3315 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 53.0 pool 731------------------------------envoy 1.50c sidecar 1.19cwrk hottest 0.80c mean 0.10c------------------------------mean request 3.17 ms before 1.16 envoy 0.60 sidecar 0.79 worker 0.62 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 44.0 pool 731------------------------------envoy 1.84c sidecar 1.42cwrk hottest 0.73c mean 0.11c------------------------------mean request 3.23 ms before 1.17 envoy 0.60 sidecar 0.80 worker 0.66 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 31.9 msin-flight 989 pool 988------------------------------envoy 1.88c sidecar 1.41cwrk hottest 0.77c mean 0.12c------------------------------mean request 10.42 ms before 4.90 envoy 1.59 sidecar 1.77 worker 2.17 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4368 /sachieved 4368 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 378 pool 988------------------------------envoy 2.12c sidecar 1.55cwrk hottest 0.81c mean 0.13c------------------------------mean request 3.13 ms before 0.83 envoy 0.68 sidecar 1.02 worker 0.60 resume 0.98 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 42.0 pool 988------------------------------envoy 2.67c sidecar 0.00cwrk hottest 0.69c mean 0.17c------------------------------mean request 3.33 ms before 1.34 envoy 0.73 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 894 pool 988------------------------------envoy 2.57c sidecar 1.87cwrk hottest 0.70c mean 0.16c------------------------------mean request 5.20 ms before 1.71 envoy 0.65 sidecar 2.04 worker 0.80 resume 2.00 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 45.0 pool 988------------------------------envoy 2.46c sidecar 1.80cwrk hottest 0.81c mean 0.16c------------------------------mean request 3.19 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.57 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5906 /sachieved 5905 /s (100.0% ok)p50 3.1 ms p95 5.1 msin-flight 68.0 pool 988------------------------------envoy 2.94c sidecar 2.00cwrk hottest 0.75c mean 0.19c------------------------------mean request 3.38 ms before 1.21 envoy 0.68 sidecar 0.82 worker 0.68 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.3 ms p95 6.1 msin-flight 99.0 pool 988------------------------------envoy 2.85c sidecar 2.04cwrk hottest 0.87c mean 0.18c------------------------------mean request 3.67 ms before 1.34 envoy 0.72 sidecar 0.82 worker 0.80 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.3 ms p95 54.8 msin-flight 1719 pool 1718------------------------------envoy 2.99c sidecar 2.06cwrk hottest 0.85c mean 0.18c------------------------------mean request 11.38 ms before 5.48 envoy 2.25 sidecar 1.05 worker 2.61 resume 0.98 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6800 /sachieved 6800 /s (100.0% ok)p50 3.2 ms p95 9.6 msin-flight 2390 pool 2389------------------------------envoy 3.77c sidecar 2.19cwrk hottest 0.79c mean 0.23c------------------------------mean request 10.22 ms before 4.50 envoy 2.32 sidecar 0.89 worker 2.50 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.3% ok)p50 3.5 ms p95 19.2 msin-flight 1327 pool 2389------------------------------envoy 3.46c sidecar 2.39cwrk hottest 0.90c mean 0.22c------------------------------mean request 8.49 ms before 1.57 envoy 1.82 sidecar 2.94 worker 2.16 resume 2.92 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (98.5% ok)p50 3.2 ms p95 16.1 msin-flight 1249 pool 2389------------------------------envoy 3.59c sidecar 2.23cwrk hottest 0.87c mean 0.22c------------------------------mean request 9.64 ms before 1.65 envoy 1.80 sidecar 3.92 worker 2.26 resume 3.94 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7339 /sachieved 7336 /s (100.0% ok)p50 3.4 ms p95 27.5 msin-flight 237 pool 2389------------------------------envoy 3.77c sidecar 2.35cwrk hottest 0.88c mean 0.24c------------------------------mean request 6.89 ms before 3.15 envoy 1.22 sidecar 0.90 worker 1.62 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (98.7% ok)p50 3.5 ms p95 32.0 msin-flight 1355 pool 2389------------------------------envoy 4.24c sidecar 2.57cwrk hottest 0.85c mean 0.27c------------------------------mean request 11.29 ms before 3.37 envoy 1.80 sidecar 3.14 worker 2.98 resume 3.14 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 3.6 ms p95 21.4 msin-flight 1266 pool 2389------------------------------envoy 4.25c sidecar 2.64cwrk hottest 0.84c mean 0.26c------------------------------mean request 9.04 ms before 2.41 envoy 1.92 sidecar 2.06 worker 2.66 resume 2.01 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8002 /s (100.0% ok)p50 3.6 ms p95 19.7 msin-flight 168 pool 2389------------------------------envoy 4.12c sidecar 2.61cwrk hottest 0.85c mean 0.26c------------------------------mean request 6.57 ms before 2.57 envoy 1.33 sidecar 0.85 worker 1.81 resume 0.81 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8785 /sachieved 8779 /s (100.0% ok)p50 3.9 ms p95 29.8 msin-flight 1169 pool 2389------------------------------envoy 4.76c sidecar 0.00cwrk hottest 0.84c mean 0.29c------------------------------mean request 9.29 ms before 2.93 envoy 2.00 sidecar 1.48 worker 2.88 resume 1.44 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8999 /s (100.0% ok)p50 3.7 ms p95 40.4 msin-flight 177 pool 2389------------------------------envoy 4.89c sidecar 2.84cwrk hottest 0.81c mean 0.30c------------------------------mean request 9.81 ms before 3.75 envoy 2.03 sidecar 0.90 worker 3.13 resume 0.86 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9001 /s (100.0% ok)p50 3.8 ms p95 37.4 msin-flight 180 pool 1354------------------------------envoy 4.93c sidecar 2.86cwrk hottest 0.80c mean 0.31c------------------------------mean request 9.67 ms before 3.84 envoy 1.98 sidecar 0.87 worker 2.98 resume 0.83 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9956 /sachieved 9957 /s (100.0% ok)p50 4.5 ms p95 23.9 msin-flight 422 pool 1354------------------------------envoy 5.42c sidecar 3.00cwrk hottest 0.76c mean 0.34c------------------------------mean request 7.42 ms before 2.68 envoy 1.57 sidecar 0.91 worker 2.26 resume 0.87 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9387 /s (100.0% ok)p50 4.7 ms p95 243.7 msin-flight 11k pool 11k------------------------------envoy 5.39c sidecar 3.06cwrk hottest 0.77c mean 0.35c------------------------------mean request 38.76 ms before 21.54 envoy 5.51 sidecar 1.47 worker 10.25 resume 1.15 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 11k /s (100.0% ok)p50 13.9 ms p95 999.8 msin-flight 9138 pool 11k------------------------------envoy 6.16c sidecar 2.69cwrk hottest 0.75c mean 0.37c------------------------------mean request 198.32 ms before 64.26 envoy 60.01 sidecar 1.38 worker 72.67 resume 1.14 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.9 ms p95 57.2 msin-flight 623 pool 11k------------------------------envoy 6.25c sidecar 3.19cwrk hottest 0.69c mean 0.39c------------------------------mean request 17.71 ms before 6.41 envoy 4.20 sidecar 0.97 worker 6.14 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 8.4 ms p95 86.7 msin-flight 1422 pool 11k------------------------------envoy 6.25c sidecar 3.24cwrk hottest 0.68c mean 0.39c------------------------------mean request 23.07 ms before 10.16 envoy 4.52 sidecar 1.41 worker 6.98 resume 1.36 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 9.2 ms p95 68.5 msin-flight 1078 pool 11k------------------------------envoy 6.17c sidecar 3.21cwrk hottest 0.71c mean 0.39c------------------------------mean request 20.75 ms before 7.83 envoy 4.75 sidecar 1.17 worker 7.00 resume 1.13 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 20.1 ms p95 124.5 msin-flight 1575 pool 11k------------------------------envoy 6.72c sidecar 3.24cwrk hottest 0.70c mean 0.42c------------------------------mean request 35.88 ms before 12.63 envoy 8.60 sidecar 1.11 worker 13.54 resume 1.06 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 38.3 ms p95 233.7 msin-flight 1917 pool 11k------------------------------envoy 7.32c sidecar 3.30cwrk hottest 0.57c mean 0.46c------------------------------mean request 63.46 ms before 29.12 envoy 13.01 sidecar 1.15 worker 20.19 resume 1.10 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 30.1 ms p95 179.4 msin-flight 1590 pool 11k------------------------------envoy 7.23c sidecar 0.00cwrk hottest 0.56c mean 0.45c------------------------------mean request 51.16 ms before 14.34 envoy 14.00 sidecar 1.07 worker 21.75 resume 1.02 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 68.4 ms p95 632.6 msin-flight 7146 pool 11k------------------------------envoy 7.37c sidecar 3.20cwrk hottest 0.60c mean 0.47c------------------------------mean request 189.77 ms before 73.60 envoy 46.04 sidecar 2.09 worker 68.04 resume 2.03 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 110.5 ms p95 457.8 msin-flight 3171 pool 8090------------------------------envoy 7.53c sidecar 0.00cwrk hottest 0.55c mean 0.47c------------------------------mean request 158.04 ms before 59.68 envoy 38.34 sidecar 1.48 worker 58.53 resume 1.43 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (99.8% ok)p50 95.6 ms p95 407.3 msin-flight 3687 pool 6847------------------------------envoy 7.52c sidecar 3.04cwrk hottest 0.55c mean 0.47c------------------------------mean request 137.03 ms before 57.83 envoy 29.63 sidecar 2.50 worker 47.07 resume 2.45 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 82.5 ms p95 391.6 msin-flight 3745 pool 6847------------------------------envoy 7.63c sidecar 3.03cwrk hottest 0.55c mean 0.47c------------------------------mean request 135.56 ms before 49.08 envoy 33.38 sidecar 1.42 worker 51.67 resume 1.37 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 13k /s (100.0% ok)p50 278.3 ms p95 699.1 msin-flight 10k pool 10k------------------------------envoy 7.93c sidecar 3.04cwrk hottest 0.51c mean 0.50c------------------------------mean request 307.88 ms before 149.09 envoy 63.31 sidecar 2.02 worker 93.46 resume 1.95 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 13k /s (99.5% ok)p50 1244.6 ms p95 1743.3 msin-flight 24k pool 24k------------------------------envoy 7.94c sidecar 2.43cwrk hottest 0.52c mean 0.50c------------------------------mean request 1248.97 ms before 362.49 envoy 646.84 sidecar 3.27 worker 236.37 resume 3.08 (control plane) +rung 13 14k qps setpoint------------------------------offered 9567 /sachieved 11k /s (93.1% ok)p50 1698.9 ms p95 1976.8 msin-flight 24k pool 23k------------------------------envoy 6.03c sidecar 0.00cwrk hottest 0.51c mean 0.34c------------------------------mean request 1639.68 ms before 226.18 envoy 1159.92 sidecar 3.23 worker 250.35 resume 3.08 (control plane) +rung -1 0 qps setpoint------------------------------offered 0 /sachieved 0 /s (0.0% ok)p50 0.0 ms p95 0.0 msin-flight 30.0 pool 30.0------------------------------envoy 0.01c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 989 /sachieved 989 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 9.0 pool 30.0------------------------------envoy 0.56c sidecar 0.19cwrk hottest 0.33c mean 0.03c------------------------------mean request 3.23 ms before 1.01 envoy 0.69 sidecar 0.80 worker 0.74 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 6.0 pool 30.0------------------------------envoy 0.54c sidecar 0.50cwrk hottest 0.25c mean 0.03c------------------------------mean request 2.98 ms before 0.98 envoy 0.73 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1113 /sachieved 1113 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 12.0 pool 30.0------------------------------envoy 0.61c sidecar 0.53cwrk hottest 0.27c mean 0.04c------------------------------mean request 2.98 ms before 0.98 envoy 0.76 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 51.0 pool 51.0------------------------------envoy 1.11c sidecar 0.91cwrk hottest 0.46c mean 0.07c------------------------------mean request 3.00 ms before 0.99 envoy 0.80 sidecar 0.82 worker 0.40 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 731 pool 731------------------------------envoy 1.09c sidecar 0.93cwrk hottest 0.51c mean 0.06c------------------------------mean request 8.23 ms before 3.86 envoy 2.21 sidecar 1.44 worker 0.73 resume 1.26 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2264 /sachieved 2264 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 135 pool 731------------------------------envoy 1.10c sidecar 0.88cwrk hottest 0.76c mean 0.07c------------------------------mean request 3.31 ms before 1.27 envoy 0.80 sidecar 0.81 worker 0.43 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 21.0 pool 731------------------------------envoy 1.37c sidecar 1.15cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.05 ms before 1.10 envoy 0.64 sidecar 0.79 worker 0.52 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 272 pool 731------------------------------envoy 1.42c sidecar 1.17cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.82 ms before 1.31 envoy 0.89 sidecar 0.94 worker 0.67 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3315 /sachieved 3315 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 53.0 pool 731------------------------------envoy 1.50c sidecar 1.19cwrk hottest 0.80c mean 0.10c------------------------------mean request 3.17 ms before 1.16 envoy 0.60 sidecar 0.79 worker 0.62 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 44.0 pool 731------------------------------envoy 1.84c sidecar 1.42cwrk hottest 0.73c mean 0.11c------------------------------mean request 3.23 ms before 1.17 envoy 0.60 sidecar 0.80 worker 0.66 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 31.9 msin-flight 989 pool 988------------------------------envoy 1.88c sidecar 1.41cwrk hottest 0.77c mean 0.12c------------------------------mean request 10.42 ms before 4.90 envoy 1.59 sidecar 1.77 worker 2.17 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4368 /sachieved 4368 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 378 pool 988------------------------------envoy 2.12c sidecar 1.55cwrk hottest 0.81c mean 0.13c------------------------------mean request 3.13 ms before 0.83 envoy 0.68 sidecar 1.02 worker 0.60 resume 0.98 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 42.0 pool 988------------------------------envoy 2.67c sidecar 0.00cwrk hottest 0.69c mean 0.17c------------------------------mean request 3.33 ms before 1.34 envoy 0.73 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 894 pool 988------------------------------envoy 2.57c sidecar 1.87cwrk hottest 0.70c mean 0.16c------------------------------mean request 5.20 ms before 1.71 envoy 0.65 sidecar 2.04 worker 0.80 resume 2.00 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 45.0 pool 988------------------------------envoy 2.46c sidecar 1.80cwrk hottest 0.81c mean 0.16c------------------------------mean request 3.19 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.57 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5906 /sachieved 5905 /s (100.0% ok)p50 3.1 ms p95 5.1 msin-flight 68.0 pool 988------------------------------envoy 2.94c sidecar 2.00cwrk hottest 0.75c mean 0.19c------------------------------mean request 3.38 ms before 1.21 envoy 0.68 sidecar 0.82 worker 0.68 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.3 ms p95 6.1 msin-flight 99.0 pool 988------------------------------envoy 2.85c sidecar 2.04cwrk hottest 0.87c mean 0.18c------------------------------mean request 3.67 ms before 1.34 envoy 0.72 sidecar 0.82 worker 0.80 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.3 ms p95 54.8 msin-flight 1719 pool 1718------------------------------envoy 2.99c sidecar 2.06cwrk hottest 0.85c mean 0.18c------------------------------mean request 11.38 ms before 5.48 envoy 2.25 sidecar 1.05 worker 2.61 resume 0.98 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6800 /sachieved 6800 /s (100.0% ok)p50 3.2 ms p95 9.6 msin-flight 2390 pool 2389------------------------------envoy 3.77c sidecar 2.19cwrk hottest 0.79c mean 0.23c------------------------------mean request 10.22 ms before 4.50 envoy 2.32 sidecar 0.89 worker 2.50 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.3% ok)p50 3.5 ms p95 19.2 msin-flight 1327 pool 2389------------------------------envoy 3.46c sidecar 2.39cwrk hottest 0.90c mean 0.22c------------------------------mean request 8.49 ms before 1.57 envoy 1.82 sidecar 2.94 worker 2.16 resume 2.92 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (98.5% ok)p50 3.2 ms p95 16.1 msin-flight 1249 pool 2389------------------------------envoy 3.59c sidecar 2.23cwrk hottest 0.87c mean 0.22c------------------------------mean request 9.64 ms before 1.65 envoy 1.80 sidecar 3.92 worker 2.26 resume 3.94 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7339 /sachieved 7336 /s (100.0% ok)p50 3.4 ms p95 27.5 msin-flight 237 pool 2389------------------------------envoy 3.77c sidecar 2.35cwrk hottest 0.88c mean 0.24c------------------------------mean request 6.89 ms before 3.15 envoy 1.22 sidecar 0.90 worker 1.62 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (98.7% ok)p50 3.5 ms p95 32.0 msin-flight 1355 pool 2389------------------------------envoy 4.24c sidecar 2.57cwrk hottest 0.85c mean 0.27c------------------------------mean request 11.29 ms before 3.37 envoy 1.80 sidecar 3.14 worker 2.98 resume 3.14 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 3.6 ms p95 21.4 msin-flight 1266 pool 2389------------------------------envoy 4.25c sidecar 2.64cwrk hottest 0.84c mean 0.26c------------------------------mean request 9.04 ms before 2.41 envoy 1.92 sidecar 2.06 worker 2.66 resume 2.01 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8002 /s (100.0% ok)p50 3.6 ms p95 19.7 msin-flight 168 pool 2389------------------------------envoy 4.12c sidecar 2.61cwrk hottest 0.85c mean 0.26c------------------------------mean request 6.57 ms before 2.57 envoy 1.33 sidecar 0.85 worker 1.81 resume 0.81 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8785 /sachieved 8779 /s (100.0% ok)p50 3.9 ms p95 29.8 msin-flight 1169 pool 2389------------------------------envoy 4.76c sidecar 0.00cwrk hottest 0.84c mean 0.29c------------------------------mean request 9.29 ms before 2.93 envoy 2.00 sidecar 1.48 worker 2.88 resume 1.44 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8999 /s (100.0% ok)p50 3.7 ms p95 40.4 msin-flight 177 pool 2389------------------------------envoy 4.89c sidecar 2.84cwrk hottest 0.81c mean 0.30c------------------------------mean request 9.81 ms before 3.75 envoy 2.03 sidecar 0.90 worker 3.13 resume 0.86 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9001 /s (100.0% ok)p50 3.8 ms p95 37.4 msin-flight 180 pool 1354------------------------------envoy 4.93c sidecar 2.86cwrk hottest 0.80c mean 0.31c------------------------------mean request 9.67 ms before 3.84 envoy 1.98 sidecar 0.87 worker 2.98 resume 0.83 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9956 /sachieved 9957 /s (100.0% ok)p50 4.5 ms p95 23.9 msin-flight 422 pool 1354------------------------------envoy 5.42c sidecar 3.00cwrk hottest 0.76c mean 0.34c------------------------------mean request 7.42 ms before 2.68 envoy 1.57 sidecar 0.91 worker 2.26 resume 0.87 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9387 /s (100.0% ok)p50 4.7 ms p95 243.7 msin-flight 11k pool 11k------------------------------envoy 5.39c sidecar 3.06cwrk hottest 0.77c mean 0.35c------------------------------mean request 38.76 ms before 21.54 envoy 5.51 sidecar 1.47 worker 10.25 resume 1.15 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 11k /s (100.0% ok)p50 13.9 ms p95 999.8 msin-flight 9138 pool 11k------------------------------envoy 6.16c sidecar 2.69cwrk hottest 0.75c mean 0.37c------------------------------mean request 198.32 ms before 64.26 envoy 60.01 sidecar 1.38 worker 72.67 resume 1.14 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.9 ms p95 57.2 msin-flight 623 pool 11k------------------------------envoy 6.25c sidecar 3.19cwrk hottest 0.69c mean 0.39c------------------------------mean request 17.71 ms before 6.41 envoy 4.20 sidecar 0.97 worker 6.14 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 8.4 ms p95 86.7 msin-flight 1422 pool 11k------------------------------envoy 6.25c sidecar 3.24cwrk hottest 0.68c mean 0.39c------------------------------mean request 23.07 ms before 10.16 envoy 4.52 sidecar 1.41 worker 6.98 resume 1.36 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 9.2 ms p95 68.5 msin-flight 1078 pool 11k------------------------------envoy 6.17c sidecar 3.21cwrk hottest 0.71c mean 0.39c------------------------------mean request 20.75 ms before 7.83 envoy 4.75 sidecar 1.17 worker 7.00 resume 1.13 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 20.1 ms p95 124.5 msin-flight 1575 pool 11k------------------------------envoy 6.72c sidecar 3.24cwrk hottest 0.70c mean 0.42c------------------------------mean request 35.88 ms before 12.63 envoy 8.60 sidecar 1.11 worker 13.54 resume 1.06 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 38.3 ms p95 233.7 msin-flight 1917 pool 11k------------------------------envoy 7.32c sidecar 3.30cwrk hottest 0.57c mean 0.46c------------------------------mean request 63.46 ms before 29.12 envoy 13.01 sidecar 1.15 worker 20.19 resume 1.10 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 30.1 ms p95 179.4 msin-flight 1590 pool 11k------------------------------envoy 7.23c sidecar 0.00cwrk hottest 0.56c mean 0.45c------------------------------mean request 51.16 ms before 14.34 envoy 14.00 sidecar 1.07 worker 21.75 resume 1.02 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 68.4 ms p95 632.6 msin-flight 7146 pool 11k------------------------------envoy 7.37c sidecar 3.20cwrk hottest 0.60c mean 0.47c------------------------------mean request 189.77 ms before 73.60 envoy 46.04 sidecar 2.09 worker 68.04 resume 2.03 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 110.5 ms p95 457.8 msin-flight 3171 pool 8090------------------------------envoy 7.53c sidecar 0.00cwrk hottest 0.55c mean 0.47c------------------------------mean request 158.04 ms before 59.68 envoy 38.34 sidecar 1.48 worker 58.53 resume 1.43 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (99.8% ok)p50 95.6 ms p95 407.3 msin-flight 3687 pool 6847------------------------------envoy 7.52c sidecar 3.04cwrk hottest 0.55c mean 0.47c------------------------------mean request 137.03 ms before 57.83 envoy 29.63 sidecar 2.50 worker 47.07 resume 2.45 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 82.5 ms p95 391.6 msin-flight 3745 pool 6847------------------------------envoy 7.63c sidecar 3.03cwrk hottest 0.55c mean 0.47c------------------------------mean request 135.56 ms before 49.08 envoy 33.38 sidecar 1.42 worker 51.67 resume 1.37 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 13k /s (100.0% ok)p50 278.3 ms p95 699.1 msin-flight 10k pool 10k------------------------------envoy 7.93c sidecar 3.04cwrk hottest 0.51c mean 0.50c------------------------------mean request 307.88 ms before 149.09 envoy 63.31 sidecar 2.02 worker 93.46 resume 1.95 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 13k /s (99.5% ok)p50 1244.6 ms p95 1743.3 msin-flight 24k pool 24k------------------------------envoy 7.94c sidecar 2.43cwrk hottest 0.52c mean 0.50c------------------------------mean request 1248.97 ms before 362.49 envoy 646.84 sidecar 3.27 worker 236.37 resume 3.08 (control plane) +rung 13 14k qps setpoint------------------------------offered 9567 /sachieved 11k /s (93.1% ok)p50 1698.9 ms p95 1976.8 msin-flight 24k pool 23k------------------------------envoy 6.03c sidecar 0.00cwrk hottest 0.51c mean 0.34c------------------------------mean request 1639.68 ms before 226.18 envoy 1159.92 sidecar 3.23 worker 250.35 resume 3.08 (control plane) +rung -1 0 qps setpoint------------------------------offered 0 /sachieved 0 /s (0.0% ok)p50 0.0 ms p95 0.0 msin-flight 30.0 pool 30.0------------------------------envoy 0.01c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 989 /sachieved 989 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 9.0 pool 30.0------------------------------envoy 0.56c sidecar 0.19cwrk hottest 0.33c mean 0.03c------------------------------mean request 3.23 ms before 1.01 envoy 0.69 sidecar 0.80 worker 0.74 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 6.0 pool 30.0------------------------------envoy 0.54c sidecar 0.50cwrk hottest 0.25c mean 0.03c------------------------------mean request 2.98 ms before 0.98 envoy 0.73 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1113 /sachieved 1113 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 12.0 pool 30.0------------------------------envoy 0.61c sidecar 0.53cwrk hottest 0.27c mean 0.04c------------------------------mean request 2.98 ms before 0.98 envoy 0.76 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 51.0 pool 51.0------------------------------envoy 1.11c sidecar 0.91cwrk hottest 0.46c mean 0.07c------------------------------mean request 3.00 ms before 0.99 envoy 0.80 sidecar 0.82 worker 0.40 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 731 pool 731------------------------------envoy 1.09c sidecar 0.93cwrk hottest 0.51c mean 0.06c------------------------------mean request 8.23 ms before 3.86 envoy 2.21 sidecar 1.44 worker 0.73 resume 1.26 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2264 /sachieved 2264 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 135 pool 731------------------------------envoy 1.10c sidecar 0.88cwrk hottest 0.76c mean 0.07c------------------------------mean request 3.31 ms before 1.27 envoy 0.80 sidecar 0.81 worker 0.43 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 21.0 pool 731------------------------------envoy 1.37c sidecar 1.15cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.05 ms before 1.10 envoy 0.64 sidecar 0.79 worker 0.52 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 272 pool 731------------------------------envoy 1.42c sidecar 1.17cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.82 ms before 1.31 envoy 0.89 sidecar 0.94 worker 0.67 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3315 /sachieved 3315 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 53.0 pool 731------------------------------envoy 1.50c sidecar 1.19cwrk hottest 0.80c mean 0.10c------------------------------mean request 3.17 ms before 1.16 envoy 0.60 sidecar 0.79 worker 0.62 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 44.0 pool 731------------------------------envoy 1.84c sidecar 1.42cwrk hottest 0.73c mean 0.11c------------------------------mean request 3.23 ms before 1.17 envoy 0.60 sidecar 0.80 worker 0.66 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 31.9 msin-flight 989 pool 988------------------------------envoy 1.88c sidecar 1.41cwrk hottest 0.77c mean 0.12c------------------------------mean request 10.42 ms before 4.90 envoy 1.59 sidecar 1.77 worker 2.17 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4368 /sachieved 4368 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 378 pool 988------------------------------envoy 2.12c sidecar 1.55cwrk hottest 0.81c mean 0.13c------------------------------mean request 3.13 ms before 0.83 envoy 0.68 sidecar 1.02 worker 0.60 resume 0.98 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 42.0 pool 988------------------------------envoy 2.67c sidecar 0.00cwrk hottest 0.69c mean 0.17c------------------------------mean request 3.33 ms before 1.34 envoy 0.73 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 894 pool 988------------------------------envoy 2.57c sidecar 1.87cwrk hottest 0.70c mean 0.16c------------------------------mean request 5.20 ms before 1.71 envoy 0.65 sidecar 2.04 worker 0.80 resume 2.00 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 45.0 pool 988------------------------------envoy 2.46c sidecar 1.80cwrk hottest 0.81c mean 0.16c------------------------------mean request 3.19 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.57 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5906 /sachieved 5905 /s (100.0% ok)p50 3.1 ms p95 5.1 msin-flight 68.0 pool 988------------------------------envoy 2.94c sidecar 2.00cwrk hottest 0.75c mean 0.19c------------------------------mean request 3.38 ms before 1.21 envoy 0.68 sidecar 0.82 worker 0.68 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.3 ms p95 6.1 msin-flight 99.0 pool 988------------------------------envoy 2.85c sidecar 2.04cwrk hottest 0.87c mean 0.18c------------------------------mean request 3.67 ms before 1.34 envoy 0.72 sidecar 0.82 worker 0.80 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.3 ms p95 54.8 msin-flight 1719 pool 1718------------------------------envoy 2.99c sidecar 2.06cwrk hottest 0.85c mean 0.18c------------------------------mean request 11.38 ms before 5.48 envoy 2.25 sidecar 1.05 worker 2.61 resume 0.98 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6800 /sachieved 6800 /s (100.0% ok)p50 3.2 ms p95 9.6 msin-flight 2390 pool 2389------------------------------envoy 3.77c sidecar 2.19cwrk hottest 0.79c mean 0.23c------------------------------mean request 10.22 ms before 4.50 envoy 2.32 sidecar 0.89 worker 2.50 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.3% ok)p50 3.5 ms p95 19.2 msin-flight 1327 pool 2389------------------------------envoy 3.46c sidecar 2.39cwrk hottest 0.90c mean 0.22c------------------------------mean request 8.49 ms before 1.57 envoy 1.82 sidecar 2.94 worker 2.16 resume 2.92 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (98.5% ok)p50 3.2 ms p95 16.1 msin-flight 1249 pool 2389------------------------------envoy 3.59c sidecar 2.23cwrk hottest 0.87c mean 0.22c------------------------------mean request 9.64 ms before 1.65 envoy 1.80 sidecar 3.92 worker 2.26 resume 3.94 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7339 /sachieved 7336 /s (100.0% ok)p50 3.4 ms p95 27.5 msin-flight 237 pool 2389------------------------------envoy 3.77c sidecar 2.35cwrk hottest 0.88c mean 0.24c------------------------------mean request 6.89 ms before 3.15 envoy 1.22 sidecar 0.90 worker 1.62 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (98.7% ok)p50 3.5 ms p95 32.0 msin-flight 1355 pool 2389------------------------------envoy 4.24c sidecar 2.57cwrk hottest 0.85c mean 0.27c------------------------------mean request 11.29 ms before 3.37 envoy 1.80 sidecar 3.14 worker 2.98 resume 3.14 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 3.6 ms p95 21.4 msin-flight 1266 pool 2389------------------------------envoy 4.25c sidecar 2.64cwrk hottest 0.84c mean 0.26c------------------------------mean request 9.04 ms before 2.41 envoy 1.92 sidecar 2.06 worker 2.66 resume 2.01 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8002 /s (100.0% ok)p50 3.6 ms p95 19.7 msin-flight 168 pool 2389------------------------------envoy 4.12c sidecar 2.61cwrk hottest 0.85c mean 0.26c------------------------------mean request 6.57 ms before 2.57 envoy 1.33 sidecar 0.85 worker 1.81 resume 0.81 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8785 /sachieved 8779 /s (100.0% ok)p50 3.9 ms p95 29.8 msin-flight 1169 pool 2389------------------------------envoy 4.76c sidecar 0.00cwrk hottest 0.84c mean 0.29c------------------------------mean request 9.29 ms before 2.93 envoy 2.00 sidecar 1.48 worker 2.88 resume 1.44 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8999 /s (100.0% ok)p50 3.7 ms p95 40.4 msin-flight 177 pool 2389------------------------------envoy 4.89c sidecar 2.84cwrk hottest 0.81c mean 0.30c------------------------------mean request 9.81 ms before 3.75 envoy 2.03 sidecar 0.90 worker 3.13 resume 0.86 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9001 /s (100.0% ok)p50 3.8 ms p95 37.4 msin-flight 180 pool 1354------------------------------envoy 4.93c sidecar 2.86cwrk hottest 0.80c mean 0.31c------------------------------mean request 9.67 ms before 3.84 envoy 1.98 sidecar 0.87 worker 2.98 resume 0.83 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9956 /sachieved 9957 /s (100.0% ok)p50 4.5 ms p95 23.9 msin-flight 422 pool 1354------------------------------envoy 5.42c sidecar 3.00cwrk hottest 0.76c mean 0.34c------------------------------mean request 7.42 ms before 2.68 envoy 1.57 sidecar 0.91 worker 2.26 resume 0.87 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9387 /s (100.0% ok)p50 4.7 ms p95 243.7 msin-flight 11k pool 11k------------------------------envoy 5.39c sidecar 3.06cwrk hottest 0.77c mean 0.35c------------------------------mean request 38.76 ms before 21.54 envoy 5.51 sidecar 1.47 worker 10.25 resume 1.15 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 11k /s (100.0% ok)p50 13.9 ms p95 999.8 msin-flight 9138 pool 11k------------------------------envoy 6.16c sidecar 2.69cwrk hottest 0.75c mean 0.37c------------------------------mean request 198.32 ms before 64.26 envoy 60.01 sidecar 1.38 worker 72.67 resume 1.14 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.9 ms p95 57.2 msin-flight 623 pool 11k------------------------------envoy 6.25c sidecar 3.19cwrk hottest 0.69c mean 0.39c------------------------------mean request 17.71 ms before 6.41 envoy 4.20 sidecar 0.97 worker 6.14 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 8.4 ms p95 86.7 msin-flight 1422 pool 11k------------------------------envoy 6.25c sidecar 3.24cwrk hottest 0.68c mean 0.39c------------------------------mean request 23.07 ms before 10.16 envoy 4.52 sidecar 1.41 worker 6.98 resume 1.36 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 9.2 ms p95 68.5 msin-flight 1078 pool 11k------------------------------envoy 6.17c sidecar 3.21cwrk hottest 0.71c mean 0.39c------------------------------mean request 20.75 ms before 7.83 envoy 4.75 sidecar 1.17 worker 7.00 resume 1.13 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 20.1 ms p95 124.5 msin-flight 1575 pool 11k------------------------------envoy 6.72c sidecar 3.24cwrk hottest 0.70c mean 0.42c------------------------------mean request 35.88 ms before 12.63 envoy 8.60 sidecar 1.11 worker 13.54 resume 1.06 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 38.3 ms p95 233.7 msin-flight 1917 pool 11k------------------------------envoy 7.32c sidecar 3.30cwrk hottest 0.57c mean 0.46c------------------------------mean request 63.46 ms before 29.12 envoy 13.01 sidecar 1.15 worker 20.19 resume 1.10 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 30.1 ms p95 179.4 msin-flight 1590 pool 11k------------------------------envoy 7.23c sidecar 0.00cwrk hottest 0.56c mean 0.45c------------------------------mean request 51.16 ms before 14.34 envoy 14.00 sidecar 1.07 worker 21.75 resume 1.02 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 68.4 ms p95 632.6 msin-flight 7146 pool 11k------------------------------envoy 7.37c sidecar 3.20cwrk hottest 0.60c mean 0.47c------------------------------mean request 189.77 ms before 73.60 envoy 46.04 sidecar 2.09 worker 68.04 resume 2.03 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 110.5 ms p95 457.8 msin-flight 3171 pool 8090------------------------------envoy 7.53c sidecar 0.00cwrk hottest 0.55c mean 0.47c------------------------------mean request 158.04 ms before 59.68 envoy 38.34 sidecar 1.48 worker 58.53 resume 1.43 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (99.8% ok)p50 95.6 ms p95 407.3 msin-flight 3687 pool 6847------------------------------envoy 7.52c sidecar 3.04cwrk hottest 0.55c mean 0.47c------------------------------mean request 137.03 ms before 57.83 envoy 29.63 sidecar 2.50 worker 47.07 resume 2.45 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 82.5 ms p95 391.6 msin-flight 3745 pool 6847------------------------------envoy 7.63c sidecar 3.03cwrk hottest 0.55c mean 0.47c------------------------------mean request 135.56 ms before 49.08 envoy 33.38 sidecar 1.42 worker 51.67 resume 1.37 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 13k /s (100.0% ok)p50 278.3 ms p95 699.1 msin-flight 10k pool 10k------------------------------envoy 7.93c sidecar 3.04cwrk hottest 0.51c mean 0.50c------------------------------mean request 307.88 ms before 149.09 envoy 63.31 sidecar 2.02 worker 93.46 resume 1.95 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 13k /s (99.5% ok)p50 1244.6 ms p95 1743.3 msin-flight 24k pool 24k------------------------------envoy 7.94c sidecar 2.43cwrk hottest 0.52c mean 0.50c------------------------------mean request 1248.97 ms before 362.49 envoy 646.84 sidecar 3.27 worker 236.37 resume 3.08 (control plane) +rung 13 14k qps setpoint------------------------------offered 9567 /sachieved 11k /s (93.1% ok)p50 1698.9 ms p95 1976.8 msin-flight 24k pool 23k------------------------------envoy 6.03c sidecar 0.00cwrk hottest 0.51c mean 0.34c------------------------------mean request 1639.68 ms before 226.18 envoy 1159.92 sidecar 3.23 worker 250.35 resume 3.08 (control plane) +rung -1 0 qps setpoint------------------------------offered 0 /sachieved 0 /s (0.0% ok)p50 0.0 ms p95 0.0 msin-flight 30.0 pool 30.0------------------------------envoy 0.01c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 989 /sachieved 989 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 9.0 pool 30.0------------------------------envoy 0.56c sidecar 0.19cwrk hottest 0.33c mean 0.03c------------------------------mean request 3.23 ms before 1.01 envoy 0.69 sidecar 0.80 worker 0.74 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 6.0 pool 30.0------------------------------envoy 0.54c sidecar 0.50cwrk hottest 0.25c mean 0.03c------------------------------mean request 2.98 ms before 0.98 envoy 0.73 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1113 /sachieved 1113 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 12.0 pool 30.0------------------------------envoy 0.61c sidecar 0.53cwrk hottest 0.27c mean 0.04c------------------------------mean request 2.98 ms before 0.98 envoy 0.76 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 51.0 pool 51.0------------------------------envoy 1.11c sidecar 0.91cwrk hottest 0.46c mean 0.07c------------------------------mean request 3.00 ms before 0.99 envoy 0.80 sidecar 0.82 worker 0.40 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 731 pool 731------------------------------envoy 1.09c sidecar 0.93cwrk hottest 0.51c mean 0.06c------------------------------mean request 8.23 ms before 3.86 envoy 2.21 sidecar 1.44 worker 0.73 resume 1.26 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2264 /sachieved 2264 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 135 pool 731------------------------------envoy 1.10c sidecar 0.88cwrk hottest 0.76c mean 0.07c------------------------------mean request 3.31 ms before 1.27 envoy 0.80 sidecar 0.81 worker 0.43 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 21.0 pool 731------------------------------envoy 1.37c sidecar 1.15cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.05 ms before 1.10 envoy 0.64 sidecar 0.79 worker 0.52 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 272 pool 731------------------------------envoy 1.42c sidecar 1.17cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.82 ms before 1.31 envoy 0.89 sidecar 0.94 worker 0.67 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3315 /sachieved 3315 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 53.0 pool 731------------------------------envoy 1.50c sidecar 1.19cwrk hottest 0.80c mean 0.10c------------------------------mean request 3.17 ms before 1.16 envoy 0.60 sidecar 0.79 worker 0.62 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 44.0 pool 731------------------------------envoy 1.84c sidecar 1.42cwrk hottest 0.73c mean 0.11c------------------------------mean request 3.23 ms before 1.17 envoy 0.60 sidecar 0.80 worker 0.66 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 31.9 msin-flight 989 pool 988------------------------------envoy 1.88c sidecar 1.41cwrk hottest 0.77c mean 0.12c------------------------------mean request 10.42 ms before 4.90 envoy 1.59 sidecar 1.77 worker 2.17 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4368 /sachieved 4368 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 378 pool 988------------------------------envoy 2.12c sidecar 1.55cwrk hottest 0.81c mean 0.13c------------------------------mean request 3.13 ms before 0.83 envoy 0.68 sidecar 1.02 worker 0.60 resume 0.98 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 42.0 pool 988------------------------------envoy 2.67c sidecar 0.00cwrk hottest 0.69c mean 0.17c------------------------------mean request 3.33 ms before 1.34 envoy 0.73 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 894 pool 988------------------------------envoy 2.57c sidecar 1.87cwrk hottest 0.70c mean 0.16c------------------------------mean request 5.20 ms before 1.71 envoy 0.65 sidecar 2.04 worker 0.80 resume 2.00 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 45.0 pool 988------------------------------envoy 2.46c sidecar 1.80cwrk hottest 0.81c mean 0.16c------------------------------mean request 3.19 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.57 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5906 /sachieved 5905 /s (100.0% ok)p50 3.1 ms p95 5.1 msin-flight 68.0 pool 988------------------------------envoy 2.94c sidecar 2.00cwrk hottest 0.75c mean 0.19c------------------------------mean request 3.38 ms before 1.21 envoy 0.68 sidecar 0.82 worker 0.68 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.3 ms p95 6.1 msin-flight 99.0 pool 988------------------------------envoy 2.85c sidecar 2.04cwrk hottest 0.87c mean 0.18c------------------------------mean request 3.67 ms before 1.34 envoy 0.72 sidecar 0.82 worker 0.80 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.3 ms p95 54.8 msin-flight 1719 pool 1718------------------------------envoy 2.99c sidecar 2.06cwrk hottest 0.85c mean 0.18c------------------------------mean request 11.38 ms before 5.48 envoy 2.25 sidecar 1.05 worker 2.61 resume 0.98 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6800 /sachieved 6800 /s (100.0% ok)p50 3.2 ms p95 9.6 msin-flight 2390 pool 2389------------------------------envoy 3.77c sidecar 2.19cwrk hottest 0.79c mean 0.23c------------------------------mean request 10.22 ms before 4.50 envoy 2.32 sidecar 0.89 worker 2.50 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.3% ok)p50 3.5 ms p95 19.2 msin-flight 1327 pool 2389------------------------------envoy 3.46c sidecar 2.39cwrk hottest 0.90c mean 0.22c------------------------------mean request 8.49 ms before 1.57 envoy 1.82 sidecar 2.94 worker 2.16 resume 2.92 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (98.5% ok)p50 3.2 ms p95 16.1 msin-flight 1249 pool 2389------------------------------envoy 3.59c sidecar 2.23cwrk hottest 0.87c mean 0.22c------------------------------mean request 9.64 ms before 1.65 envoy 1.80 sidecar 3.92 worker 2.26 resume 3.94 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7339 /sachieved 7336 /s (100.0% ok)p50 3.4 ms p95 27.5 msin-flight 237 pool 2389------------------------------envoy 3.77c sidecar 2.35cwrk hottest 0.88c mean 0.24c------------------------------mean request 6.89 ms before 3.15 envoy 1.22 sidecar 0.90 worker 1.62 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (98.7% ok)p50 3.5 ms p95 32.0 msin-flight 1355 pool 2389------------------------------envoy 4.24c sidecar 2.57cwrk hottest 0.85c mean 0.27c------------------------------mean request 11.29 ms before 3.37 envoy 1.80 sidecar 3.14 worker 2.98 resume 3.14 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 3.6 ms p95 21.4 msin-flight 1266 pool 2389------------------------------envoy 4.25c sidecar 2.64cwrk hottest 0.84c mean 0.26c------------------------------mean request 9.04 ms before 2.41 envoy 1.92 sidecar 2.06 worker 2.66 resume 2.01 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8002 /s (100.0% ok)p50 3.6 ms p95 19.7 msin-flight 168 pool 2389------------------------------envoy 4.12c sidecar 2.61cwrk hottest 0.85c mean 0.26c------------------------------mean request 6.57 ms before 2.57 envoy 1.33 sidecar 0.85 worker 1.81 resume 0.81 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8785 /sachieved 8779 /s (100.0% ok)p50 3.9 ms p95 29.8 msin-flight 1169 pool 2389------------------------------envoy 4.76c sidecar 0.00cwrk hottest 0.84c mean 0.29c------------------------------mean request 9.29 ms before 2.93 envoy 2.00 sidecar 1.48 worker 2.88 resume 1.44 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8999 /s (100.0% ok)p50 3.7 ms p95 40.4 msin-flight 177 pool 2389------------------------------envoy 4.89c sidecar 2.84cwrk hottest 0.81c mean 0.30c------------------------------mean request 9.81 ms before 3.75 envoy 2.03 sidecar 0.90 worker 3.13 resume 0.86 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9001 /s (100.0% ok)p50 3.8 ms p95 37.4 msin-flight 180 pool 1354------------------------------envoy 4.93c sidecar 2.86cwrk hottest 0.80c mean 0.31c------------------------------mean request 9.67 ms before 3.84 envoy 1.98 sidecar 0.87 worker 2.98 resume 0.83 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9956 /sachieved 9957 /s (100.0% ok)p50 4.5 ms p95 23.9 msin-flight 422 pool 1354------------------------------envoy 5.42c sidecar 3.00cwrk hottest 0.76c mean 0.34c------------------------------mean request 7.42 ms before 2.68 envoy 1.57 sidecar 0.91 worker 2.26 resume 0.87 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9387 /s (100.0% ok)p50 4.7 ms p95 243.7 msin-flight 11k pool 11k------------------------------envoy 5.39c sidecar 3.06cwrk hottest 0.77c mean 0.35c------------------------------mean request 38.76 ms before 21.54 envoy 5.51 sidecar 1.47 worker 10.25 resume 1.15 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 11k /s (100.0% ok)p50 13.9 ms p95 999.8 msin-flight 9138 pool 11k------------------------------envoy 6.16c sidecar 2.69cwrk hottest 0.75c mean 0.37c------------------------------mean request 198.32 ms before 64.26 envoy 60.01 sidecar 1.38 worker 72.67 resume 1.14 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.9 ms p95 57.2 msin-flight 623 pool 11k------------------------------envoy 6.25c sidecar 3.19cwrk hottest 0.69c mean 0.39c------------------------------mean request 17.71 ms before 6.41 envoy 4.20 sidecar 0.97 worker 6.14 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 8.4 ms p95 86.7 msin-flight 1422 pool 11k------------------------------envoy 6.25c sidecar 3.24cwrk hottest 0.68c mean 0.39c------------------------------mean request 23.07 ms before 10.16 envoy 4.52 sidecar 1.41 worker 6.98 resume 1.36 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 9.2 ms p95 68.5 msin-flight 1078 pool 11k------------------------------envoy 6.17c sidecar 3.21cwrk hottest 0.71c mean 0.39c------------------------------mean request 20.75 ms before 7.83 envoy 4.75 sidecar 1.17 worker 7.00 resume 1.13 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 20.1 ms p95 124.5 msin-flight 1575 pool 11k------------------------------envoy 6.72c sidecar 3.24cwrk hottest 0.70c mean 0.42c------------------------------mean request 35.88 ms before 12.63 envoy 8.60 sidecar 1.11 worker 13.54 resume 1.06 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 38.3 ms p95 233.7 msin-flight 1917 pool 11k------------------------------envoy 7.32c sidecar 3.30cwrk hottest 0.57c mean 0.46c------------------------------mean request 63.46 ms before 29.12 envoy 13.01 sidecar 1.15 worker 20.19 resume 1.10 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 30.1 ms p95 179.4 msin-flight 1590 pool 11k------------------------------envoy 7.23c sidecar 0.00cwrk hottest 0.56c mean 0.45c------------------------------mean request 51.16 ms before 14.34 envoy 14.00 sidecar 1.07 worker 21.75 resume 1.02 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 68.4 ms p95 632.6 msin-flight 7146 pool 11k------------------------------envoy 7.37c sidecar 3.20cwrk hottest 0.60c mean 0.47c------------------------------mean request 189.77 ms before 73.60 envoy 46.04 sidecar 2.09 worker 68.04 resume 2.03 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 110.5 ms p95 457.8 msin-flight 3171 pool 8090------------------------------envoy 7.53c sidecar 0.00cwrk hottest 0.55c mean 0.47c------------------------------mean request 158.04 ms before 59.68 envoy 38.34 sidecar 1.48 worker 58.53 resume 1.43 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (99.8% ok)p50 95.6 ms p95 407.3 msin-flight 3687 pool 6847------------------------------envoy 7.52c sidecar 3.04cwrk hottest 0.55c mean 0.47c------------------------------mean request 137.03 ms before 57.83 envoy 29.63 sidecar 2.50 worker 47.07 resume 2.45 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 82.5 ms p95 391.6 msin-flight 3745 pool 6847------------------------------envoy 7.63c sidecar 3.03cwrk hottest 0.55c mean 0.47c------------------------------mean request 135.56 ms before 49.08 envoy 33.38 sidecar 1.42 worker 51.67 resume 1.37 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 13k /s (100.0% ok)p50 278.3 ms p95 699.1 msin-flight 10k pool 10k------------------------------envoy 7.93c sidecar 3.04cwrk hottest 0.51c mean 0.50c------------------------------mean request 307.88 ms before 149.09 envoy 63.31 sidecar 2.02 worker 93.46 resume 1.95 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 13k /s (99.5% ok)p50 1244.6 ms p95 1743.3 msin-flight 24k pool 24k------------------------------envoy 7.94c sidecar 2.43cwrk hottest 0.52c mean 0.50c------------------------------mean request 1248.97 ms before 362.49 envoy 646.84 sidecar 3.27 worker 236.37 resume 3.08 (control plane) +rung 13 14k qps setpoint------------------------------offered 9567 /sachieved 11k /s (93.1% ok)p50 1698.9 ms p95 1976.8 msin-flight 24k pool 23k------------------------------envoy 6.03c sidecar 0.00cwrk hottest 0.51c mean 0.34c------------------------------mean request 1639.68 ms before 226.18 envoy 1159.92 sidecar 3.23 worker 250.35 resume 3.08 (control plane) +rung -1 0 qps setpoint------------------------------offered 0 /sachieved 0 /s (0.0% ok)p50 0.0 ms p95 0.0 msin-flight 30.0 pool 30.0------------------------------envoy 0.01c sidecar 0.00c +rung 0 1000 qps setpoint [warmup]------------------------------offered 989 /sachieved 989 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 9.0 pool 30.0------------------------------envoy 0.56c sidecar 0.19cwrk hottest 0.33c mean 0.03c------------------------------mean request 3.23 ms before 1.01 envoy 0.69 sidecar 0.80 worker 0.74 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 6.0 pool 30.0------------------------------envoy 0.54c sidecar 0.50cwrk hottest 0.25c mean 0.03c------------------------------mean request 2.98 ms before 0.98 envoy 0.73 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1113 /sachieved 1113 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 12.0 pool 30.0------------------------------envoy 0.61c sidecar 0.53cwrk hottest 0.27c mean 0.04c------------------------------mean request 2.98 ms before 0.98 envoy 0.76 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 51.0 pool 51.0------------------------------envoy 1.11c sidecar 0.91cwrk hottest 0.46c mean 0.07c------------------------------mean request 3.00 ms before 0.99 envoy 0.80 sidecar 0.82 worker 0.40 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 731 pool 731------------------------------envoy 1.09c sidecar 0.93cwrk hottest 0.51c mean 0.06c------------------------------mean request 8.23 ms before 3.86 envoy 2.21 sidecar 1.44 worker 0.73 resume 1.26 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2264 /sachieved 2264 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 135 pool 731------------------------------envoy 1.10c sidecar 0.88cwrk hottest 0.76c mean 0.07c------------------------------mean request 3.31 ms before 1.27 envoy 0.80 sidecar 0.81 worker 0.43 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 21.0 pool 731------------------------------envoy 1.37c sidecar 1.15cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.05 ms before 1.10 envoy 0.64 sidecar 0.79 worker 0.52 resume 0.75 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.5 msin-flight 272 pool 731------------------------------envoy 1.42c sidecar 1.17cwrk hottest 0.72c mean 0.09c------------------------------mean request 3.82 ms before 1.31 envoy 0.89 sidecar 0.94 worker 0.67 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3315 /sachieved 3315 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 53.0 pool 731------------------------------envoy 1.50c sidecar 1.19cwrk hottest 0.80c mean 0.10c------------------------------mean request 3.17 ms before 1.16 envoy 0.60 sidecar 0.79 worker 0.62 resume 0.75 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 44.0 pool 731------------------------------envoy 1.84c sidecar 1.42cwrk hottest 0.73c mean 0.11c------------------------------mean request 3.23 ms before 1.17 envoy 0.60 sidecar 0.80 worker 0.66 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 31.9 msin-flight 989 pool 988------------------------------envoy 1.88c sidecar 1.41cwrk hottest 0.77c mean 0.12c------------------------------mean request 10.42 ms before 4.90 envoy 1.59 sidecar 1.77 worker 2.17 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4368 /sachieved 4368 /s (100.0% ok)p50 3.0 ms p95 4.4 msin-flight 378 pool 988------------------------------envoy 2.12c sidecar 1.55cwrk hottest 0.81c mean 0.13c------------------------------mean request 3.13 ms before 0.83 envoy 0.68 sidecar 1.02 worker 0.60 resume 0.98 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 42.0 pool 988------------------------------envoy 2.67c sidecar 0.00cwrk hottest 0.69c mean 0.17c------------------------------mean request 3.33 ms before 1.34 envoy 0.73 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 894 pool 988------------------------------envoy 2.57c sidecar 1.87cwrk hottest 0.70c mean 0.16c------------------------------mean request 5.20 ms before 1.71 envoy 0.65 sidecar 2.04 worker 0.80 resume 2.00 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.0 ms p95 4.7 msin-flight 45.0 pool 988------------------------------envoy 2.46c sidecar 1.80cwrk hottest 0.81c mean 0.16c------------------------------mean request 3.19 ms before 1.11 envoy 0.70 sidecar 0.81 worker 0.57 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5906 /sachieved 5905 /s (100.0% ok)p50 3.1 ms p95 5.1 msin-flight 68.0 pool 988------------------------------envoy 2.94c sidecar 2.00cwrk hottest 0.75c mean 0.19c------------------------------mean request 3.38 ms before 1.21 envoy 0.68 sidecar 0.82 worker 0.68 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5999 /s (100.0% ok)p50 3.3 ms p95 6.1 msin-flight 99.0 pool 988------------------------------envoy 2.85c sidecar 2.04cwrk hottest 0.87c mean 0.18c------------------------------mean request 3.67 ms before 1.34 envoy 0.72 sidecar 0.82 worker 0.80 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.3 ms p95 54.8 msin-flight 1719 pool 1718------------------------------envoy 2.99c sidecar 2.06cwrk hottest 0.85c mean 0.18c------------------------------mean request 11.38 ms before 5.48 envoy 2.25 sidecar 1.05 worker 2.61 resume 0.98 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6800 /sachieved 6800 /s (100.0% ok)p50 3.2 ms p95 9.6 msin-flight 2390 pool 2389------------------------------envoy 3.77c sidecar 2.19cwrk hottest 0.79c mean 0.23c------------------------------mean request 10.22 ms before 4.50 envoy 2.32 sidecar 0.89 worker 2.50 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (99.3% ok)p50 3.5 ms p95 19.2 msin-flight 1327 pool 2389------------------------------envoy 3.46c sidecar 2.39cwrk hottest 0.90c mean 0.22c------------------------------mean request 8.49 ms before 1.57 envoy 1.82 sidecar 2.94 worker 2.16 resume 2.92 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (98.5% ok)p50 3.2 ms p95 16.1 msin-flight 1249 pool 2389------------------------------envoy 3.59c sidecar 2.23cwrk hottest 0.87c mean 0.22c------------------------------mean request 9.64 ms before 1.65 envoy 1.80 sidecar 3.92 worker 2.26 resume 3.94 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7339 /sachieved 7336 /s (100.0% ok)p50 3.4 ms p95 27.5 msin-flight 237 pool 2389------------------------------envoy 3.77c sidecar 2.35cwrk hottest 0.88c mean 0.24c------------------------------mean request 6.89 ms before 3.15 envoy 1.22 sidecar 0.90 worker 1.62 resume 0.86 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8003 /s (98.7% ok)p50 3.5 ms p95 32.0 msin-flight 1355 pool 2389------------------------------envoy 4.24c sidecar 2.57cwrk hottest 0.85c mean 0.27c------------------------------mean request 11.29 ms before 3.37 envoy 1.80 sidecar 3.14 worker 2.98 resume 3.14 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7997 /s (100.0% ok)p50 3.6 ms p95 21.4 msin-flight 1266 pool 2389------------------------------envoy 4.25c sidecar 2.64cwrk hottest 0.84c mean 0.26c------------------------------mean request 9.04 ms before 2.41 envoy 1.92 sidecar 2.06 worker 2.66 resume 2.01 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8002 /s (100.0% ok)p50 3.6 ms p95 19.7 msin-flight 168 pool 2389------------------------------envoy 4.12c sidecar 2.61cwrk hottest 0.85c mean 0.26c------------------------------mean request 6.57 ms before 2.57 envoy 1.33 sidecar 0.85 worker 1.81 resume 0.81 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8785 /sachieved 8779 /s (100.0% ok)p50 3.9 ms p95 29.8 msin-flight 1169 pool 2389------------------------------envoy 4.76c sidecar 0.00cwrk hottest 0.84c mean 0.29c------------------------------mean request 9.29 ms before 2.93 envoy 2.00 sidecar 1.48 worker 2.88 resume 1.44 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8999 /s (100.0% ok)p50 3.7 ms p95 40.4 msin-flight 177 pool 2389------------------------------envoy 4.89c sidecar 2.84cwrk hottest 0.81c mean 0.30c------------------------------mean request 9.81 ms before 3.75 envoy 2.03 sidecar 0.90 worker 3.13 resume 0.86 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9001 /s (100.0% ok)p50 3.8 ms p95 37.4 msin-flight 180 pool 1354------------------------------envoy 4.93c sidecar 2.86cwrk hottest 0.80c mean 0.31c------------------------------mean request 9.67 ms before 3.84 envoy 1.98 sidecar 0.87 worker 2.98 resume 0.83 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9956 /sachieved 9957 /s (100.0% ok)p50 4.5 ms p95 23.9 msin-flight 422 pool 1354------------------------------envoy 5.42c sidecar 3.00cwrk hottest 0.76c mean 0.34c------------------------------mean request 7.42 ms before 2.68 envoy 1.57 sidecar 0.91 worker 2.26 resume 0.87 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9387 /s (100.0% ok)p50 4.7 ms p95 243.7 msin-flight 11k pool 11k------------------------------envoy 5.39c sidecar 3.06cwrk hottest 0.77c mean 0.35c------------------------------mean request 38.76 ms before 21.54 envoy 5.51 sidecar 1.47 worker 10.25 resume 1.15 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 11k /s (100.0% ok)p50 13.9 ms p95 999.8 msin-flight 9138 pool 11k------------------------------envoy 6.16c sidecar 2.69cwrk hottest 0.75c mean 0.37c------------------------------mean request 198.32 ms before 64.26 envoy 60.01 sidecar 1.38 worker 72.67 resume 1.14 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.9 ms p95 57.2 msin-flight 623 pool 11k------------------------------envoy 6.25c sidecar 3.19cwrk hottest 0.69c mean 0.39c------------------------------mean request 17.71 ms before 6.41 envoy 4.20 sidecar 0.97 worker 6.14 resume 0.92 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 8.4 ms p95 86.7 msin-flight 1422 pool 11k------------------------------envoy 6.25c sidecar 3.24cwrk hottest 0.68c mean 0.39c------------------------------mean request 23.07 ms before 10.16 envoy 4.52 sidecar 1.41 worker 6.98 resume 1.36 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 9.2 ms p95 68.5 msin-flight 1078 pool 11k------------------------------envoy 6.17c sidecar 3.21cwrk hottest 0.71c mean 0.39c------------------------------mean request 20.75 ms before 7.83 envoy 4.75 sidecar 1.17 worker 7.00 resume 1.13 (control plane) +rung 11 12k qps setpoint [warmup]------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 20.1 ms p95 124.5 msin-flight 1575 pool 11k------------------------------envoy 6.72c sidecar 3.24cwrk hottest 0.70c mean 0.42c------------------------------mean request 35.88 ms before 12.63 envoy 8.60 sidecar 1.11 worker 13.54 resume 1.06 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 38.3 ms p95 233.7 msin-flight 1917 pool 11k------------------------------envoy 7.32c sidecar 3.30cwrk hottest 0.57c mean 0.46c------------------------------mean request 63.46 ms before 29.12 envoy 13.01 sidecar 1.15 worker 20.19 resume 1.10 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 30.1 ms p95 179.4 msin-flight 1590 pool 11k------------------------------envoy 7.23c sidecar 0.00cwrk hottest 0.56c mean 0.45c------------------------------mean request 51.16 ms before 14.34 envoy 14.00 sidecar 1.07 worker 21.75 resume 1.02 (control plane) +rung 12 13k qps setpoint [warmup]------------------------------offered 13k /sachieved 12k /s (99.9% ok)p50 68.4 ms p95 632.6 msin-flight 7146 pool 11k------------------------------envoy 7.37c sidecar 3.20cwrk hottest 0.60c mean 0.47c------------------------------mean request 189.77 ms before 73.60 envoy 46.04 sidecar 2.09 worker 68.04 resume 2.03 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 110.5 ms p95 457.8 msin-flight 3171 pool 8090------------------------------envoy 7.53c sidecar 0.00cwrk hottest 0.55c mean 0.47c------------------------------mean request 158.04 ms before 59.68 envoy 38.34 sidecar 1.48 worker 58.53 resume 1.43 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (99.8% ok)p50 95.6 ms p95 407.3 msin-flight 3687 pool 6847------------------------------envoy 7.52c sidecar 3.04cwrk hottest 0.55c mean 0.47c------------------------------mean request 137.03 ms before 57.83 envoy 29.63 sidecar 2.50 worker 47.07 resume 2.45 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 82.5 ms p95 391.6 msin-flight 3745 pool 6847------------------------------envoy 7.63c sidecar 3.03cwrk hottest 0.55c mean 0.47c------------------------------mean request 135.56 ms before 49.08 envoy 33.38 sidecar 1.42 worker 51.67 resume 1.37 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 13k /s (100.0% ok)p50 278.3 ms p95 699.1 msin-flight 10k pool 10k------------------------------envoy 7.93c sidecar 3.04cwrk hottest 0.51c mean 0.50c------------------------------mean request 307.88 ms before 149.09 envoy 63.31 sidecar 2.02 worker 93.46 resume 1.95 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 13k /s (99.5% ok)p50 1244.6 ms p95 1743.3 msin-flight 24k pool 24k------------------------------envoy 7.94c sidecar 2.43cwrk hottest 0.52c mean 0.50c------------------------------mean request 1248.97 ms before 362.49 envoy 646.84 sidecar 3.27 worker 236.37 resume 3.08 (control plane) +rung 13 14k qps setpoint------------------------------offered 9567 /sachieved 11k /s (93.1% ok)p50 1698.9 ms p95 1976.8 msin-flight 24k pool 23k------------------------------envoy 6.03c sidecar 0.00cwrk hottest 0.51c mean 0.34c------------------------------mean request 1639.68 ms before 226.18 envoy 1159.92 sidecar 3.23 worker 250.35 resume 3.08 (control plane) +
+ + + +atenet-router capacity — 16c +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 +R9 +R10 +R11 +R12 +R13 +R14 + +0 + +10k + +20k + +30k +requests / s + + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + + + + + + + +0 + +5.0 + +10.0 + +15.0 +cores + + + + + + + + +envoy CPU limit, 16 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + + + + + + + +0 + +500.0M + +1.0G + +1.5G + +2.0G +bytes + +100 + +200 + +300 + +400 + +500 + +600 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung 0 1000 qps setpoint [warmup]------------------------------offered 855 /sachieved 855 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 24.0 pool 24.0------------------------------envoy 0.49c sidecar 0.13cwrk hottest 0.23c mean 0.03c------------------------------mean request 3.12 ms before 1.08 envoy 0.72 sidecar 0.78 worker 0.55 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.48cwrk hottest 0.23c mean 0.03c------------------------------mean request 2.87 ms before 1.09 envoy 0.73 sidecar 0.77 worker 0.28 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 8.0 pool 24.0------------------------------envoy 0.48c sidecar 0.48cwrk hottest 0.23c mean 0.04c------------------------------mean request 2.85 ms before 1.11 envoy 0.73 sidecar 0.78 worker 0.23 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1945 /sachieved 1945 /s (100.0% ok)p50 2.7 ms p95 3.5 msin-flight 9.0 pool 24.0------------------------------envoy 1.06c sidecar 0.74cwrk hottest 0.26c mean 0.07c------------------------------mean request 2.79 ms before 0.95 envoy 0.84 sidecar 0.79 worker 0.22 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.7 msin-flight 178 pool 178------------------------------envoy 1.07c sidecar 0.92cwrk hottest 0.44c mean 0.07c------------------------------mean request 3.01 ms before 1.17 envoy 0.87 sidecar 0.81 worker 0.15 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 22.0 pool 178------------------------------envoy 0.97c sidecar 0.88cwrk hottest 0.50c mean 0.06c------------------------------mean request 3.24 ms before 1.09 envoy 0.89 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2658 /sachieved 2658 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 178------------------------------envoy 1.25c sidecar 1.04cwrk hottest 0.67c mean 0.08c------------------------------mean request 2.97 ms before 1.05 envoy 0.70 sidecar 0.80 worker 0.41 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 178------------------------------envoy 1.45c sidecar 1.21cwrk hottest 0.73c mean 0.09c------------------------------mean request 2.99 ms before 1.03 envoy 0.69 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 17.0 pool 178------------------------------envoy 1.40c sidecar 1.22cwrk hottest 0.64c mean 0.09c------------------------------mean request 2.96 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.43 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3877 /sachieved 3877 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 22.0 pool 178------------------------------envoy 1.90c sidecar 1.38cwrk hottest 0.67c mean 0.11c------------------------------mean request 2.94 ms before 1.02 envoy 0.69 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 23.0 pool 178------------------------------envoy 1.94c sidecar 0.00cwrk hottest 0.77c mean 0.12c------------------------------mean request 3.03 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.50 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 37.0 pool 178------------------------------envoy 1.97c sidecar 1.55cwrk hottest 0.66c mean 0.12c------------------------------mean request 2.90 ms before 0.97 envoy 0.72 sidecar 0.80 worker 0.41 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4347 /sachieved 4347 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 36.0 pool 178------------------------------envoy 2.15c sidecar 1.51cwrk hottest 0.70c mean 0.14c------------------------------mean request 2.93 ms before 1.01 envoy 0.70 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 99.0 pool 100------------------------------envoy 2.57c sidecar 1.84cwrk hottest 0.67c mean 0.16c------------------------------mean request 2.92 ms before 1.00 envoy 0.72 sidecar 0.79 worker 0.41 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 73.0 pool 100------------------------------envoy 2.58c sidecar 1.91cwrk hottest 0.71c mean 0.16c------------------------------mean request 2.93 ms before 0.99 envoy 0.74 sidecar 0.80 worker 0.40 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 2.8 ms p95 4.4 msin-flight 37.0 pool 100------------------------------envoy 2.59c sidecar 1.90cwrk hottest 0.79c mean 0.17c------------------------------mean request 3.01 ms before 1.05 envoy 0.76 sidecar 0.79 worker 0.42 resume 0.75 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5953 /sachieved 5953 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 547 pool 547------------------------------envoy 3.23c sidecar 2.07cwrk hottest 0.66c mean 0.21c------------------------------mean request 3.75 ms before 1.72 envoy 0.33 sidecar 1.21 worker 0.48 resume 1.17 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 57.0 pool 547------------------------------envoy 3.35c sidecar 0.00cwrk hottest 0.65c mean 0.21c------------------------------mean request 3.00 ms before 0.33 envoy 1.29 sidecar 0.81 worker 0.58 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 52.0 pool 547------------------------------envoy 3.32c sidecar 2.27cwrk hottest 0.72c mean 0.20c------------------------------mean request 3.06 ms before 1.02 envoy 0.79 sidecar 0.80 worker 0.44 resume 0.76 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 71.0 pool 547------------------------------envoy 3.27c sidecar 2.25cwrk hottest 0.75c mean 0.21c------------------------------mean request 3.03 ms before 1.00 envoy 0.76 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6766 /sachieved 6766 /s (100.0% ok)p50 3.0 ms p95 6.5 msin-flight 636 pool 637------------------------------envoy 3.74c sidecar 0.00cwrk hottest 0.80c mean 0.23c------------------------------mean request 4.34 ms before 1.11 envoy 1.06 sidecar 1.29 worker 0.87 resume 1.25 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 6.9 msin-flight 109 pool 637------------------------------envoy 3.70c sidecar 2.41cwrk hottest 0.82c mean 0.23c------------------------------mean request 3.65 ms before 1.28 envoy 0.84 sidecar 0.84 worker 0.69 resume 0.80 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 5.6 msin-flight 84.0 pool 637------------------------------envoy 3.81c sidecar 2.50cwrk hottest 0.83c mean 0.24c------------------------------mean request 3.34 ms before 1.13 envoy 0.80 sidecar 0.82 worker 0.59 resume 0.78 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7527 /sachieved 7527 /s (100.0% ok)p50 3.1 ms p95 7.6 msin-flight 1329 pool 1329------------------------------envoy 4.14c sidecar 2.53cwrk hottest 0.83c mean 0.27c------------------------------mean request 3.83 ms before 1.34 envoy -1.82 sidecar 3.56 worker 0.75 resume 3.57 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (98.9% ok)p50 3.5 ms p95 9.9 msin-flight 857 pool 1329------------------------------envoy 4.35c sidecar 2.71cwrk hottest 0.77c mean 0.27c------------------------------mean request 8.30 ms before 2.19 envoy 3.05 sidecar 1.42 worker 1.63 resume 1.38 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7998 /s (100.0% ok)p50 3.6 ms p95 13.7 msin-flight 942 pool 1329------------------------------envoy 4.18c sidecar 2.70cwrk hottest 0.81c mean 0.26c------------------------------mean request 5.80 ms before 1.66 envoy 1.23 sidecar 1.36 worker 1.55 resume 1.32 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8024 /sachieved 8022 /s (99.7% ok)p50 3.4 ms p95 18.0 msin-flight 1305 pool 1329------------------------------envoy 4.36c sidecar 0.00cwrk hottest 0.83c mean 0.28c------------------------------mean request 7.58 ms before 1.62 envoy 1.75 sidecar 2.13 worker 2.09 resume 2.09 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 9000 /sachieved 9002 /s (100.0% ok)p50 3.8 ms p95 23.4 msin-flight 979 pool 1329------------------------------envoy 4.94c sidecar 2.69cwrk hottest 0.80c mean 0.31c------------------------------mean request 8.13 ms before 2.74 envoy 1.55 sidecar 1.64 worker 2.20 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8996 /s (98.9% ok)p50 4.3 ms p95 159.9 msin-flight 3186 pool 3187------------------------------envoy 4.90c sidecar 2.86cwrk hottest 0.82c mean 0.30c------------------------------mean request 23.13 ms before 9.44 envoy 4.08 sidecar 3.02 worker 6.58 resume 3.01 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9405 /sachieved 9402 /s (100.0% ok)p50 4.6 ms p95 45.2 msin-flight 284 pool 3187------------------------------envoy 4.85c sidecar 2.84cwrk hottest 0.82c mean 0.31c------------------------------mean request 10.74 ms before 4.78 envoy 1.96 sidecar 0.94 worker 3.07 resume 0.89 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 4.6 ms p95 66.8 msin-flight 321 pool 3187------------------------------envoy 5.18c sidecar 3.03cwrk hottest 0.78c mean 0.32c------------------------------mean request 14.58 ms before 5.88 envoy 2.90 sidecar 1.00 worker 4.80 resume 0.96 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 5.3 ms p95 68.2 msin-flight 428 pool 3187------------------------------envoy 5.31c sidecar 3.09cwrk hottest 0.77c mean 0.33c------------------------------mean request 15.14 ms before 6.10 envoy 3.06 sidecar 1.01 worker 4.98 resume 0.97 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (98.5% ok)p50 6.1 ms p95 78.3 msin-flight 1575 pool 3187------------------------------envoy 6.04c sidecar 3.18cwrk hottest 0.71c mean 0.38c------------------------------mean request 17.57 ms before 3.99 envoy 4.28 sidecar 2.85 worker 6.44 resume 2.86 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (99.2% ok)p50 6.3 ms p95 32.3 msin-flight 1472 pool 3187------------------------------envoy 6.39c sidecar 3.39cwrk hottest 0.69c mean 0.40c------------------------------mean request 11.93 ms before 5.22 envoy 1.64 sidecar 2.10 worker 2.96 resume 2.08 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.1 ms p95 23.8 msin-flight 777 pool 3187------------------------------envoy 6.26c sidecar 3.36cwrk hottest 0.71c mean 0.39c------------------------------mean request 9.02 ms before 2.38 envoy 2.42 sidecar 1.02 worker 3.19 resume 0.98 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 7.4 ms p95 42.5 msin-flight 1304 pool 3187------------------------------envoy 6.52c sidecar 0.00cwrk hottest 0.71c mean 0.41c------------------------------mean request 14.08 ms before 5.11 envoy 3.11 sidecar 1.34 worker 4.52 resume 1.29 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.2 ms p95 53.2 msin-flight 591 pool 1574------------------------------envoy 7.02c sidecar 3.48cwrk hottest 0.65c mean 0.44c------------------------------mean request 18.41 ms before 7.17 envoy 4.06 sidecar 1.00 worker 6.19 resume 0.95 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 8.8 ms p95 54.8 msin-flight 1644 pool 1645------------------------------envoy 7.26c sidecar 0.00cwrk hottest 0.63c mean 0.46c------------------------------mean request 16.50 ms before 4.93 envoy 4.09 sidecar 1.49 worker 5.98 resume 1.45 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.0 ms p95 107.7 msin-flight 1556 pool 1645------------------------------envoy 7.54c sidecar 3.59cwrk hottest 0.66c mean 0.47c------------------------------mean request 27.77 ms before 14.61 envoy 4.71 sidecar 1.14 worker 7.31 resume 1.09 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 51.3 ms p95 252.3 msin-flight 3711 pool 3697------------------------------envoy 8.49c sidecar 3.60cwrk hottest 0.56c mean 0.53c------------------------------mean request 77.31 ms before 32.88 envoy 16.68 sidecar 1.28 worker 26.46 resume 1.23 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 62.4 ms p95 145.4 msin-flight 1558 pool 3697------------------------------envoy 8.32c sidecar 3.36cwrk hottest 0.54c mean 0.52c------------------------------mean request 67.88 ms before 24.61 envoy 16.09 sidecar 1.23 worker 25.94 resume 1.18 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 31.6 ms p95 103.0 msin-flight 1581 pool 3697------------------------------envoy 8.29c sidecar 3.57cwrk hottest 0.56c mean 0.52c------------------------------mean request 40.44 ms before 17.22 envoy 8.63 sidecar 1.12 worker 13.47 resume 1.07 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (99.9% ok)p50 218.0 ms p95 642.0 msin-flight 9585 pool 9587------------------------------envoy 8.56c sidecar 3.52cwrk hottest 0.56c mean 0.54c------------------------------mean request 294.49 ms before 150.87 envoy 54.81 sidecar 2.43 worker 86.38 resume 2.35 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 231.6 ms p95 667.3 msin-flight 9745 pool 9746------------------------------envoy 8.48c sidecar 2.99cwrk hottest 0.56c mean 0.53c------------------------------mean request 285.65 ms before 79.02 envoy 86.03 sidecar 1.84 worker 118.76 resume 1.77 (control plane) +rung 14 15k qps setpoint [warmup]------------------------------offered 15k /sachieved 14k /s (99.2% ok)p50 800.2 ms p95 1586.2 msin-flight 24k pool 24k------------------------------envoy 8.53c sidecar 2.85cwrk hottest 0.56c mean 0.53c------------------------------mean request 801.73 ms before 228.58 envoy 390.90 sidecar 2.90 worker 179.34 resume 2.78 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 855 /sachieved 855 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 24.0 pool 24.0------------------------------envoy 0.49c sidecar 0.13cwrk hottest 0.23c mean 0.03c------------------------------mean request 3.12 ms before 1.08 envoy 0.72 sidecar 0.78 worker 0.55 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.48cwrk hottest 0.23c mean 0.03c------------------------------mean request 2.87 ms before 1.09 envoy 0.73 sidecar 0.77 worker 0.28 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 8.0 pool 24.0------------------------------envoy 0.48c sidecar 0.48cwrk hottest 0.23c mean 0.04c------------------------------mean request 2.85 ms before 1.11 envoy 0.73 sidecar 0.78 worker 0.23 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1945 /sachieved 1945 /s (100.0% ok)p50 2.7 ms p95 3.5 msin-flight 9.0 pool 24.0------------------------------envoy 1.06c sidecar 0.74cwrk hottest 0.26c mean 0.07c------------------------------mean request 2.79 ms before 0.95 envoy 0.84 sidecar 0.79 worker 0.22 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.7 msin-flight 178 pool 178------------------------------envoy 1.07c sidecar 0.92cwrk hottest 0.44c mean 0.07c------------------------------mean request 3.01 ms before 1.17 envoy 0.87 sidecar 0.81 worker 0.15 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 22.0 pool 178------------------------------envoy 0.97c sidecar 0.88cwrk hottest 0.50c mean 0.06c------------------------------mean request 3.24 ms before 1.09 envoy 0.89 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2658 /sachieved 2658 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 178------------------------------envoy 1.25c sidecar 1.04cwrk hottest 0.67c mean 0.08c------------------------------mean request 2.97 ms before 1.05 envoy 0.70 sidecar 0.80 worker 0.41 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 178------------------------------envoy 1.45c sidecar 1.21cwrk hottest 0.73c mean 0.09c------------------------------mean request 2.99 ms before 1.03 envoy 0.69 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 17.0 pool 178------------------------------envoy 1.40c sidecar 1.22cwrk hottest 0.64c mean 0.09c------------------------------mean request 2.96 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.43 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3877 /sachieved 3877 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 22.0 pool 178------------------------------envoy 1.90c sidecar 1.38cwrk hottest 0.67c mean 0.11c------------------------------mean request 2.94 ms before 1.02 envoy 0.69 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 23.0 pool 178------------------------------envoy 1.94c sidecar 0.00cwrk hottest 0.77c mean 0.12c------------------------------mean request 3.03 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.50 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 37.0 pool 178------------------------------envoy 1.97c sidecar 1.55cwrk hottest 0.66c mean 0.12c------------------------------mean request 2.90 ms before 0.97 envoy 0.72 sidecar 0.80 worker 0.41 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4347 /sachieved 4347 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 36.0 pool 178------------------------------envoy 2.15c sidecar 1.51cwrk hottest 0.70c mean 0.14c------------------------------mean request 2.93 ms before 1.01 envoy 0.70 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 99.0 pool 100------------------------------envoy 2.57c sidecar 1.84cwrk hottest 0.67c mean 0.16c------------------------------mean request 2.92 ms before 1.00 envoy 0.72 sidecar 0.79 worker 0.41 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 73.0 pool 100------------------------------envoy 2.58c sidecar 1.91cwrk hottest 0.71c mean 0.16c------------------------------mean request 2.93 ms before 0.99 envoy 0.74 sidecar 0.80 worker 0.40 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 2.8 ms p95 4.4 msin-flight 37.0 pool 100------------------------------envoy 2.59c sidecar 1.90cwrk hottest 0.79c mean 0.17c------------------------------mean request 3.01 ms before 1.05 envoy 0.76 sidecar 0.79 worker 0.42 resume 0.75 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5953 /sachieved 5953 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 547 pool 547------------------------------envoy 3.23c sidecar 2.07cwrk hottest 0.66c mean 0.21c------------------------------mean request 3.75 ms before 1.72 envoy 0.33 sidecar 1.21 worker 0.48 resume 1.17 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 57.0 pool 547------------------------------envoy 3.35c sidecar 0.00cwrk hottest 0.65c mean 0.21c------------------------------mean request 3.00 ms before 0.33 envoy 1.29 sidecar 0.81 worker 0.58 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 52.0 pool 547------------------------------envoy 3.32c sidecar 2.27cwrk hottest 0.72c mean 0.20c------------------------------mean request 3.06 ms before 1.02 envoy 0.79 sidecar 0.80 worker 0.44 resume 0.76 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 71.0 pool 547------------------------------envoy 3.27c sidecar 2.25cwrk hottest 0.75c mean 0.21c------------------------------mean request 3.03 ms before 1.00 envoy 0.76 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6766 /sachieved 6766 /s (100.0% ok)p50 3.0 ms p95 6.5 msin-flight 636 pool 637------------------------------envoy 3.74c sidecar 0.00cwrk hottest 0.80c mean 0.23c------------------------------mean request 4.34 ms before 1.11 envoy 1.06 sidecar 1.29 worker 0.87 resume 1.25 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 6.9 msin-flight 109 pool 637------------------------------envoy 3.70c sidecar 2.41cwrk hottest 0.82c mean 0.23c------------------------------mean request 3.65 ms before 1.28 envoy 0.84 sidecar 0.84 worker 0.69 resume 0.80 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 5.6 msin-flight 84.0 pool 637------------------------------envoy 3.81c sidecar 2.50cwrk hottest 0.83c mean 0.24c------------------------------mean request 3.34 ms before 1.13 envoy 0.80 sidecar 0.82 worker 0.59 resume 0.78 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7527 /sachieved 7527 /s (100.0% ok)p50 3.1 ms p95 7.6 msin-flight 1329 pool 1329------------------------------envoy 4.14c sidecar 2.53cwrk hottest 0.83c mean 0.27c------------------------------mean request 3.83 ms before 1.34 envoy -1.82 sidecar 3.56 worker 0.75 resume 3.57 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (98.9% ok)p50 3.5 ms p95 9.9 msin-flight 857 pool 1329------------------------------envoy 4.35c sidecar 2.71cwrk hottest 0.77c mean 0.27c------------------------------mean request 8.30 ms before 2.19 envoy 3.05 sidecar 1.42 worker 1.63 resume 1.38 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7998 /s (100.0% ok)p50 3.6 ms p95 13.7 msin-flight 942 pool 1329------------------------------envoy 4.18c sidecar 2.70cwrk hottest 0.81c mean 0.26c------------------------------mean request 5.80 ms before 1.66 envoy 1.23 sidecar 1.36 worker 1.55 resume 1.32 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8024 /sachieved 8022 /s (99.7% ok)p50 3.4 ms p95 18.0 msin-flight 1305 pool 1329------------------------------envoy 4.36c sidecar 0.00cwrk hottest 0.83c mean 0.28c------------------------------mean request 7.58 ms before 1.62 envoy 1.75 sidecar 2.13 worker 2.09 resume 2.09 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 9000 /sachieved 9002 /s (100.0% ok)p50 3.8 ms p95 23.4 msin-flight 979 pool 1329------------------------------envoy 4.94c sidecar 2.69cwrk hottest 0.80c mean 0.31c------------------------------mean request 8.13 ms before 2.74 envoy 1.55 sidecar 1.64 worker 2.20 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8996 /s (98.9% ok)p50 4.3 ms p95 159.9 msin-flight 3186 pool 3187------------------------------envoy 4.90c sidecar 2.86cwrk hottest 0.82c mean 0.30c------------------------------mean request 23.13 ms before 9.44 envoy 4.08 sidecar 3.02 worker 6.58 resume 3.01 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9405 /sachieved 9402 /s (100.0% ok)p50 4.6 ms p95 45.2 msin-flight 284 pool 3187------------------------------envoy 4.85c sidecar 2.84cwrk hottest 0.82c mean 0.31c------------------------------mean request 10.74 ms before 4.78 envoy 1.96 sidecar 0.94 worker 3.07 resume 0.89 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 4.6 ms p95 66.8 msin-flight 321 pool 3187------------------------------envoy 5.18c sidecar 3.03cwrk hottest 0.78c mean 0.32c------------------------------mean request 14.58 ms before 5.88 envoy 2.90 sidecar 1.00 worker 4.80 resume 0.96 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 5.3 ms p95 68.2 msin-flight 428 pool 3187------------------------------envoy 5.31c sidecar 3.09cwrk hottest 0.77c mean 0.33c------------------------------mean request 15.14 ms before 6.10 envoy 3.06 sidecar 1.01 worker 4.98 resume 0.97 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (98.5% ok)p50 6.1 ms p95 78.3 msin-flight 1575 pool 3187------------------------------envoy 6.04c sidecar 3.18cwrk hottest 0.71c mean 0.38c------------------------------mean request 17.57 ms before 3.99 envoy 4.28 sidecar 2.85 worker 6.44 resume 2.86 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (99.2% ok)p50 6.3 ms p95 32.3 msin-flight 1472 pool 3187------------------------------envoy 6.39c sidecar 3.39cwrk hottest 0.69c mean 0.40c------------------------------mean request 11.93 ms before 5.22 envoy 1.64 sidecar 2.10 worker 2.96 resume 2.08 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.1 ms p95 23.8 msin-flight 777 pool 3187------------------------------envoy 6.26c sidecar 3.36cwrk hottest 0.71c mean 0.39c------------------------------mean request 9.02 ms before 2.38 envoy 2.42 sidecar 1.02 worker 3.19 resume 0.98 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 7.4 ms p95 42.5 msin-flight 1304 pool 3187------------------------------envoy 6.52c sidecar 0.00cwrk hottest 0.71c mean 0.41c------------------------------mean request 14.08 ms before 5.11 envoy 3.11 sidecar 1.34 worker 4.52 resume 1.29 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.2 ms p95 53.2 msin-flight 591 pool 1574------------------------------envoy 7.02c sidecar 3.48cwrk hottest 0.65c mean 0.44c------------------------------mean request 18.41 ms before 7.17 envoy 4.06 sidecar 1.00 worker 6.19 resume 0.95 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 8.8 ms p95 54.8 msin-flight 1644 pool 1645------------------------------envoy 7.26c sidecar 0.00cwrk hottest 0.63c mean 0.46c------------------------------mean request 16.50 ms before 4.93 envoy 4.09 sidecar 1.49 worker 5.98 resume 1.45 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.0 ms p95 107.7 msin-flight 1556 pool 1645------------------------------envoy 7.54c sidecar 3.59cwrk hottest 0.66c mean 0.47c------------------------------mean request 27.77 ms before 14.61 envoy 4.71 sidecar 1.14 worker 7.31 resume 1.09 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 51.3 ms p95 252.3 msin-flight 3711 pool 3697------------------------------envoy 8.49c sidecar 3.60cwrk hottest 0.56c mean 0.53c------------------------------mean request 77.31 ms before 32.88 envoy 16.68 sidecar 1.28 worker 26.46 resume 1.23 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 62.4 ms p95 145.4 msin-flight 1558 pool 3697------------------------------envoy 8.32c sidecar 3.36cwrk hottest 0.54c mean 0.52c------------------------------mean request 67.88 ms before 24.61 envoy 16.09 sidecar 1.23 worker 25.94 resume 1.18 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 31.6 ms p95 103.0 msin-flight 1581 pool 3697------------------------------envoy 8.29c sidecar 3.57cwrk hottest 0.56c mean 0.52c------------------------------mean request 40.44 ms before 17.22 envoy 8.63 sidecar 1.12 worker 13.47 resume 1.07 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (99.9% ok)p50 218.0 ms p95 642.0 msin-flight 9585 pool 9587------------------------------envoy 8.56c sidecar 3.52cwrk hottest 0.56c mean 0.54c------------------------------mean request 294.49 ms before 150.87 envoy 54.81 sidecar 2.43 worker 86.38 resume 2.35 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 231.6 ms p95 667.3 msin-flight 9745 pool 9746------------------------------envoy 8.48c sidecar 2.99cwrk hottest 0.56c mean 0.53c------------------------------mean request 285.65 ms before 79.02 envoy 86.03 sidecar 1.84 worker 118.76 resume 1.77 (control plane) +rung 14 15k qps setpoint [warmup]------------------------------offered 15k /sachieved 14k /s (99.2% ok)p50 800.2 ms p95 1586.2 msin-flight 24k pool 24k------------------------------envoy 8.53c sidecar 2.85cwrk hottest 0.56c mean 0.53c------------------------------mean request 801.73 ms before 228.58 envoy 390.90 sidecar 2.90 worker 179.34 resume 2.78 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 855 /sachieved 855 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 24.0 pool 24.0------------------------------envoy 0.49c sidecar 0.13cwrk hottest 0.23c mean 0.03c------------------------------mean request 3.12 ms before 1.08 envoy 0.72 sidecar 0.78 worker 0.55 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.48cwrk hottest 0.23c mean 0.03c------------------------------mean request 2.87 ms before 1.09 envoy 0.73 sidecar 0.77 worker 0.28 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 8.0 pool 24.0------------------------------envoy 0.48c sidecar 0.48cwrk hottest 0.23c mean 0.04c------------------------------mean request 2.85 ms before 1.11 envoy 0.73 sidecar 0.78 worker 0.23 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1945 /sachieved 1945 /s (100.0% ok)p50 2.7 ms p95 3.5 msin-flight 9.0 pool 24.0------------------------------envoy 1.06c sidecar 0.74cwrk hottest 0.26c mean 0.07c------------------------------mean request 2.79 ms before 0.95 envoy 0.84 sidecar 0.79 worker 0.22 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.7 msin-flight 178 pool 178------------------------------envoy 1.07c sidecar 0.92cwrk hottest 0.44c mean 0.07c------------------------------mean request 3.01 ms before 1.17 envoy 0.87 sidecar 0.81 worker 0.15 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 22.0 pool 178------------------------------envoy 0.97c sidecar 0.88cwrk hottest 0.50c mean 0.06c------------------------------mean request 3.24 ms before 1.09 envoy 0.89 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2658 /sachieved 2658 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 178------------------------------envoy 1.25c sidecar 1.04cwrk hottest 0.67c mean 0.08c------------------------------mean request 2.97 ms before 1.05 envoy 0.70 sidecar 0.80 worker 0.41 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 178------------------------------envoy 1.45c sidecar 1.21cwrk hottest 0.73c mean 0.09c------------------------------mean request 2.99 ms before 1.03 envoy 0.69 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 17.0 pool 178------------------------------envoy 1.40c sidecar 1.22cwrk hottest 0.64c mean 0.09c------------------------------mean request 2.96 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.43 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3877 /sachieved 3877 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 22.0 pool 178------------------------------envoy 1.90c sidecar 1.38cwrk hottest 0.67c mean 0.11c------------------------------mean request 2.94 ms before 1.02 envoy 0.69 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 23.0 pool 178------------------------------envoy 1.94c sidecar 0.00cwrk hottest 0.77c mean 0.12c------------------------------mean request 3.03 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.50 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 37.0 pool 178------------------------------envoy 1.97c sidecar 1.55cwrk hottest 0.66c mean 0.12c------------------------------mean request 2.90 ms before 0.97 envoy 0.72 sidecar 0.80 worker 0.41 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4347 /sachieved 4347 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 36.0 pool 178------------------------------envoy 2.15c sidecar 1.51cwrk hottest 0.70c mean 0.14c------------------------------mean request 2.93 ms before 1.01 envoy 0.70 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 99.0 pool 100------------------------------envoy 2.57c sidecar 1.84cwrk hottest 0.67c mean 0.16c------------------------------mean request 2.92 ms before 1.00 envoy 0.72 sidecar 0.79 worker 0.41 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 73.0 pool 100------------------------------envoy 2.58c sidecar 1.91cwrk hottest 0.71c mean 0.16c------------------------------mean request 2.93 ms before 0.99 envoy 0.74 sidecar 0.80 worker 0.40 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 2.8 ms p95 4.4 msin-flight 37.0 pool 100------------------------------envoy 2.59c sidecar 1.90cwrk hottest 0.79c mean 0.17c------------------------------mean request 3.01 ms before 1.05 envoy 0.76 sidecar 0.79 worker 0.42 resume 0.75 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5953 /sachieved 5953 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 547 pool 547------------------------------envoy 3.23c sidecar 2.07cwrk hottest 0.66c mean 0.21c------------------------------mean request 3.75 ms before 1.72 envoy 0.33 sidecar 1.21 worker 0.48 resume 1.17 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 57.0 pool 547------------------------------envoy 3.35c sidecar 0.00cwrk hottest 0.65c mean 0.21c------------------------------mean request 3.00 ms before 0.33 envoy 1.29 sidecar 0.81 worker 0.58 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 52.0 pool 547------------------------------envoy 3.32c sidecar 2.27cwrk hottest 0.72c mean 0.20c------------------------------mean request 3.06 ms before 1.02 envoy 0.79 sidecar 0.80 worker 0.44 resume 0.76 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 71.0 pool 547------------------------------envoy 3.27c sidecar 2.25cwrk hottest 0.75c mean 0.21c------------------------------mean request 3.03 ms before 1.00 envoy 0.76 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6766 /sachieved 6766 /s (100.0% ok)p50 3.0 ms p95 6.5 msin-flight 636 pool 637------------------------------envoy 3.74c sidecar 0.00cwrk hottest 0.80c mean 0.23c------------------------------mean request 4.34 ms before 1.11 envoy 1.06 sidecar 1.29 worker 0.87 resume 1.25 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 6.9 msin-flight 109 pool 637------------------------------envoy 3.70c sidecar 2.41cwrk hottest 0.82c mean 0.23c------------------------------mean request 3.65 ms before 1.28 envoy 0.84 sidecar 0.84 worker 0.69 resume 0.80 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 5.6 msin-flight 84.0 pool 637------------------------------envoy 3.81c sidecar 2.50cwrk hottest 0.83c mean 0.24c------------------------------mean request 3.34 ms before 1.13 envoy 0.80 sidecar 0.82 worker 0.59 resume 0.78 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7527 /sachieved 7527 /s (100.0% ok)p50 3.1 ms p95 7.6 msin-flight 1329 pool 1329------------------------------envoy 4.14c sidecar 2.53cwrk hottest 0.83c mean 0.27c------------------------------mean request 3.83 ms before 1.34 envoy -1.82 sidecar 3.56 worker 0.75 resume 3.57 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (98.9% ok)p50 3.5 ms p95 9.9 msin-flight 857 pool 1329------------------------------envoy 4.35c sidecar 2.71cwrk hottest 0.77c mean 0.27c------------------------------mean request 8.30 ms before 2.19 envoy 3.05 sidecar 1.42 worker 1.63 resume 1.38 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7998 /s (100.0% ok)p50 3.6 ms p95 13.7 msin-flight 942 pool 1329------------------------------envoy 4.18c sidecar 2.70cwrk hottest 0.81c mean 0.26c------------------------------mean request 5.80 ms before 1.66 envoy 1.23 sidecar 1.36 worker 1.55 resume 1.32 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8024 /sachieved 8022 /s (99.7% ok)p50 3.4 ms p95 18.0 msin-flight 1305 pool 1329------------------------------envoy 4.36c sidecar 0.00cwrk hottest 0.83c mean 0.28c------------------------------mean request 7.58 ms before 1.62 envoy 1.75 sidecar 2.13 worker 2.09 resume 2.09 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 9000 /sachieved 9002 /s (100.0% ok)p50 3.8 ms p95 23.4 msin-flight 979 pool 1329------------------------------envoy 4.94c sidecar 2.69cwrk hottest 0.80c mean 0.31c------------------------------mean request 8.13 ms before 2.74 envoy 1.55 sidecar 1.64 worker 2.20 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8996 /s (98.9% ok)p50 4.3 ms p95 159.9 msin-flight 3186 pool 3187------------------------------envoy 4.90c sidecar 2.86cwrk hottest 0.82c mean 0.30c------------------------------mean request 23.13 ms before 9.44 envoy 4.08 sidecar 3.02 worker 6.58 resume 3.01 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9405 /sachieved 9402 /s (100.0% ok)p50 4.6 ms p95 45.2 msin-flight 284 pool 3187------------------------------envoy 4.85c sidecar 2.84cwrk hottest 0.82c mean 0.31c------------------------------mean request 10.74 ms before 4.78 envoy 1.96 sidecar 0.94 worker 3.07 resume 0.89 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 4.6 ms p95 66.8 msin-flight 321 pool 3187------------------------------envoy 5.18c sidecar 3.03cwrk hottest 0.78c mean 0.32c------------------------------mean request 14.58 ms before 5.88 envoy 2.90 sidecar 1.00 worker 4.80 resume 0.96 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 5.3 ms p95 68.2 msin-flight 428 pool 3187------------------------------envoy 5.31c sidecar 3.09cwrk hottest 0.77c mean 0.33c------------------------------mean request 15.14 ms before 6.10 envoy 3.06 sidecar 1.01 worker 4.98 resume 0.97 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (98.5% ok)p50 6.1 ms p95 78.3 msin-flight 1575 pool 3187------------------------------envoy 6.04c sidecar 3.18cwrk hottest 0.71c mean 0.38c------------------------------mean request 17.57 ms before 3.99 envoy 4.28 sidecar 2.85 worker 6.44 resume 2.86 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (99.2% ok)p50 6.3 ms p95 32.3 msin-flight 1472 pool 3187------------------------------envoy 6.39c sidecar 3.39cwrk hottest 0.69c mean 0.40c------------------------------mean request 11.93 ms before 5.22 envoy 1.64 sidecar 2.10 worker 2.96 resume 2.08 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.1 ms p95 23.8 msin-flight 777 pool 3187------------------------------envoy 6.26c sidecar 3.36cwrk hottest 0.71c mean 0.39c------------------------------mean request 9.02 ms before 2.38 envoy 2.42 sidecar 1.02 worker 3.19 resume 0.98 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 7.4 ms p95 42.5 msin-flight 1304 pool 3187------------------------------envoy 6.52c sidecar 0.00cwrk hottest 0.71c mean 0.41c------------------------------mean request 14.08 ms before 5.11 envoy 3.11 sidecar 1.34 worker 4.52 resume 1.29 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.2 ms p95 53.2 msin-flight 591 pool 1574------------------------------envoy 7.02c sidecar 3.48cwrk hottest 0.65c mean 0.44c------------------------------mean request 18.41 ms before 7.17 envoy 4.06 sidecar 1.00 worker 6.19 resume 0.95 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 8.8 ms p95 54.8 msin-flight 1644 pool 1645------------------------------envoy 7.26c sidecar 0.00cwrk hottest 0.63c mean 0.46c------------------------------mean request 16.50 ms before 4.93 envoy 4.09 sidecar 1.49 worker 5.98 resume 1.45 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.0 ms p95 107.7 msin-flight 1556 pool 1645------------------------------envoy 7.54c sidecar 3.59cwrk hottest 0.66c mean 0.47c------------------------------mean request 27.77 ms before 14.61 envoy 4.71 sidecar 1.14 worker 7.31 resume 1.09 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 51.3 ms p95 252.3 msin-flight 3711 pool 3697------------------------------envoy 8.49c sidecar 3.60cwrk hottest 0.56c mean 0.53c------------------------------mean request 77.31 ms before 32.88 envoy 16.68 sidecar 1.28 worker 26.46 resume 1.23 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 62.4 ms p95 145.4 msin-flight 1558 pool 3697------------------------------envoy 8.32c sidecar 3.36cwrk hottest 0.54c mean 0.52c------------------------------mean request 67.88 ms before 24.61 envoy 16.09 sidecar 1.23 worker 25.94 resume 1.18 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 31.6 ms p95 103.0 msin-flight 1581 pool 3697------------------------------envoy 8.29c sidecar 3.57cwrk hottest 0.56c mean 0.52c------------------------------mean request 40.44 ms before 17.22 envoy 8.63 sidecar 1.12 worker 13.47 resume 1.07 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (99.9% ok)p50 218.0 ms p95 642.0 msin-flight 9585 pool 9587------------------------------envoy 8.56c sidecar 3.52cwrk hottest 0.56c mean 0.54c------------------------------mean request 294.49 ms before 150.87 envoy 54.81 sidecar 2.43 worker 86.38 resume 2.35 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 231.6 ms p95 667.3 msin-flight 9745 pool 9746------------------------------envoy 8.48c sidecar 2.99cwrk hottest 0.56c mean 0.53c------------------------------mean request 285.65 ms before 79.02 envoy 86.03 sidecar 1.84 worker 118.76 resume 1.77 (control plane) +rung 14 15k qps setpoint [warmup]------------------------------offered 15k /sachieved 14k /s (99.2% ok)p50 800.2 ms p95 1586.2 msin-flight 24k pool 24k------------------------------envoy 8.53c sidecar 2.85cwrk hottest 0.56c mean 0.53c------------------------------mean request 801.73 ms before 228.58 envoy 390.90 sidecar 2.90 worker 179.34 resume 2.78 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 855 /sachieved 855 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 24.0 pool 24.0------------------------------envoy 0.49c sidecar 0.13cwrk hottest 0.23c mean 0.03c------------------------------mean request 3.12 ms before 1.08 envoy 0.72 sidecar 0.78 worker 0.55 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.48cwrk hottest 0.23c mean 0.03c------------------------------mean request 2.87 ms before 1.09 envoy 0.73 sidecar 0.77 worker 0.28 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 8.0 pool 24.0------------------------------envoy 0.48c sidecar 0.48cwrk hottest 0.23c mean 0.04c------------------------------mean request 2.85 ms before 1.11 envoy 0.73 sidecar 0.78 worker 0.23 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1945 /sachieved 1945 /s (100.0% ok)p50 2.7 ms p95 3.5 msin-flight 9.0 pool 24.0------------------------------envoy 1.06c sidecar 0.74cwrk hottest 0.26c mean 0.07c------------------------------mean request 2.79 ms before 0.95 envoy 0.84 sidecar 0.79 worker 0.22 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.7 msin-flight 178 pool 178------------------------------envoy 1.07c sidecar 0.92cwrk hottest 0.44c mean 0.07c------------------------------mean request 3.01 ms before 1.17 envoy 0.87 sidecar 0.81 worker 0.15 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 22.0 pool 178------------------------------envoy 0.97c sidecar 0.88cwrk hottest 0.50c mean 0.06c------------------------------mean request 3.24 ms before 1.09 envoy 0.89 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2658 /sachieved 2658 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 178------------------------------envoy 1.25c sidecar 1.04cwrk hottest 0.67c mean 0.08c------------------------------mean request 2.97 ms before 1.05 envoy 0.70 sidecar 0.80 worker 0.41 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 178------------------------------envoy 1.45c sidecar 1.21cwrk hottest 0.73c mean 0.09c------------------------------mean request 2.99 ms before 1.03 envoy 0.69 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 17.0 pool 178------------------------------envoy 1.40c sidecar 1.22cwrk hottest 0.64c mean 0.09c------------------------------mean request 2.96 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.43 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3877 /sachieved 3877 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 22.0 pool 178------------------------------envoy 1.90c sidecar 1.38cwrk hottest 0.67c mean 0.11c------------------------------mean request 2.94 ms before 1.02 envoy 0.69 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 23.0 pool 178------------------------------envoy 1.94c sidecar 0.00cwrk hottest 0.77c mean 0.12c------------------------------mean request 3.03 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.50 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 37.0 pool 178------------------------------envoy 1.97c sidecar 1.55cwrk hottest 0.66c mean 0.12c------------------------------mean request 2.90 ms before 0.97 envoy 0.72 sidecar 0.80 worker 0.41 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4347 /sachieved 4347 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 36.0 pool 178------------------------------envoy 2.15c sidecar 1.51cwrk hottest 0.70c mean 0.14c------------------------------mean request 2.93 ms before 1.01 envoy 0.70 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 99.0 pool 100------------------------------envoy 2.57c sidecar 1.84cwrk hottest 0.67c mean 0.16c------------------------------mean request 2.92 ms before 1.00 envoy 0.72 sidecar 0.79 worker 0.41 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 73.0 pool 100------------------------------envoy 2.58c sidecar 1.91cwrk hottest 0.71c mean 0.16c------------------------------mean request 2.93 ms before 0.99 envoy 0.74 sidecar 0.80 worker 0.40 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 2.8 ms p95 4.4 msin-flight 37.0 pool 100------------------------------envoy 2.59c sidecar 1.90cwrk hottest 0.79c mean 0.17c------------------------------mean request 3.01 ms before 1.05 envoy 0.76 sidecar 0.79 worker 0.42 resume 0.75 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5953 /sachieved 5953 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 547 pool 547------------------------------envoy 3.23c sidecar 2.07cwrk hottest 0.66c mean 0.21c------------------------------mean request 3.75 ms before 1.72 envoy 0.33 sidecar 1.21 worker 0.48 resume 1.17 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 57.0 pool 547------------------------------envoy 3.35c sidecar 0.00cwrk hottest 0.65c mean 0.21c------------------------------mean request 3.00 ms before 0.33 envoy 1.29 sidecar 0.81 worker 0.58 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 52.0 pool 547------------------------------envoy 3.32c sidecar 2.27cwrk hottest 0.72c mean 0.20c------------------------------mean request 3.06 ms before 1.02 envoy 0.79 sidecar 0.80 worker 0.44 resume 0.76 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 71.0 pool 547------------------------------envoy 3.27c sidecar 2.25cwrk hottest 0.75c mean 0.21c------------------------------mean request 3.03 ms before 1.00 envoy 0.76 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6766 /sachieved 6766 /s (100.0% ok)p50 3.0 ms p95 6.5 msin-flight 636 pool 637------------------------------envoy 3.74c sidecar 0.00cwrk hottest 0.80c mean 0.23c------------------------------mean request 4.34 ms before 1.11 envoy 1.06 sidecar 1.29 worker 0.87 resume 1.25 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 6.9 msin-flight 109 pool 637------------------------------envoy 3.70c sidecar 2.41cwrk hottest 0.82c mean 0.23c------------------------------mean request 3.65 ms before 1.28 envoy 0.84 sidecar 0.84 worker 0.69 resume 0.80 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 5.6 msin-flight 84.0 pool 637------------------------------envoy 3.81c sidecar 2.50cwrk hottest 0.83c mean 0.24c------------------------------mean request 3.34 ms before 1.13 envoy 0.80 sidecar 0.82 worker 0.59 resume 0.78 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7527 /sachieved 7527 /s (100.0% ok)p50 3.1 ms p95 7.6 msin-flight 1329 pool 1329------------------------------envoy 4.14c sidecar 2.53cwrk hottest 0.83c mean 0.27c------------------------------mean request 3.83 ms before 1.34 envoy -1.82 sidecar 3.56 worker 0.75 resume 3.57 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (98.9% ok)p50 3.5 ms p95 9.9 msin-flight 857 pool 1329------------------------------envoy 4.35c sidecar 2.71cwrk hottest 0.77c mean 0.27c------------------------------mean request 8.30 ms before 2.19 envoy 3.05 sidecar 1.42 worker 1.63 resume 1.38 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7998 /s (100.0% ok)p50 3.6 ms p95 13.7 msin-flight 942 pool 1329------------------------------envoy 4.18c sidecar 2.70cwrk hottest 0.81c mean 0.26c------------------------------mean request 5.80 ms before 1.66 envoy 1.23 sidecar 1.36 worker 1.55 resume 1.32 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8024 /sachieved 8022 /s (99.7% ok)p50 3.4 ms p95 18.0 msin-flight 1305 pool 1329------------------------------envoy 4.36c sidecar 0.00cwrk hottest 0.83c mean 0.28c------------------------------mean request 7.58 ms before 1.62 envoy 1.75 sidecar 2.13 worker 2.09 resume 2.09 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 9000 /sachieved 9002 /s (100.0% ok)p50 3.8 ms p95 23.4 msin-flight 979 pool 1329------------------------------envoy 4.94c sidecar 2.69cwrk hottest 0.80c mean 0.31c------------------------------mean request 8.13 ms before 2.74 envoy 1.55 sidecar 1.64 worker 2.20 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8996 /s (98.9% ok)p50 4.3 ms p95 159.9 msin-flight 3186 pool 3187------------------------------envoy 4.90c sidecar 2.86cwrk hottest 0.82c mean 0.30c------------------------------mean request 23.13 ms before 9.44 envoy 4.08 sidecar 3.02 worker 6.58 resume 3.01 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9405 /sachieved 9402 /s (100.0% ok)p50 4.6 ms p95 45.2 msin-flight 284 pool 3187------------------------------envoy 4.85c sidecar 2.84cwrk hottest 0.82c mean 0.31c------------------------------mean request 10.74 ms before 4.78 envoy 1.96 sidecar 0.94 worker 3.07 resume 0.89 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 4.6 ms p95 66.8 msin-flight 321 pool 3187------------------------------envoy 5.18c sidecar 3.03cwrk hottest 0.78c mean 0.32c------------------------------mean request 14.58 ms before 5.88 envoy 2.90 sidecar 1.00 worker 4.80 resume 0.96 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 5.3 ms p95 68.2 msin-flight 428 pool 3187------------------------------envoy 5.31c sidecar 3.09cwrk hottest 0.77c mean 0.33c------------------------------mean request 15.14 ms before 6.10 envoy 3.06 sidecar 1.01 worker 4.98 resume 0.97 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (98.5% ok)p50 6.1 ms p95 78.3 msin-flight 1575 pool 3187------------------------------envoy 6.04c sidecar 3.18cwrk hottest 0.71c mean 0.38c------------------------------mean request 17.57 ms before 3.99 envoy 4.28 sidecar 2.85 worker 6.44 resume 2.86 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (99.2% ok)p50 6.3 ms p95 32.3 msin-flight 1472 pool 3187------------------------------envoy 6.39c sidecar 3.39cwrk hottest 0.69c mean 0.40c------------------------------mean request 11.93 ms before 5.22 envoy 1.64 sidecar 2.10 worker 2.96 resume 2.08 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.1 ms p95 23.8 msin-flight 777 pool 3187------------------------------envoy 6.26c sidecar 3.36cwrk hottest 0.71c mean 0.39c------------------------------mean request 9.02 ms before 2.38 envoy 2.42 sidecar 1.02 worker 3.19 resume 0.98 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 7.4 ms p95 42.5 msin-flight 1304 pool 3187------------------------------envoy 6.52c sidecar 0.00cwrk hottest 0.71c mean 0.41c------------------------------mean request 14.08 ms before 5.11 envoy 3.11 sidecar 1.34 worker 4.52 resume 1.29 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.2 ms p95 53.2 msin-flight 591 pool 1574------------------------------envoy 7.02c sidecar 3.48cwrk hottest 0.65c mean 0.44c------------------------------mean request 18.41 ms before 7.17 envoy 4.06 sidecar 1.00 worker 6.19 resume 0.95 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 8.8 ms p95 54.8 msin-flight 1644 pool 1645------------------------------envoy 7.26c sidecar 0.00cwrk hottest 0.63c mean 0.46c------------------------------mean request 16.50 ms before 4.93 envoy 4.09 sidecar 1.49 worker 5.98 resume 1.45 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.0 ms p95 107.7 msin-flight 1556 pool 1645------------------------------envoy 7.54c sidecar 3.59cwrk hottest 0.66c mean 0.47c------------------------------mean request 27.77 ms before 14.61 envoy 4.71 sidecar 1.14 worker 7.31 resume 1.09 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 51.3 ms p95 252.3 msin-flight 3711 pool 3697------------------------------envoy 8.49c sidecar 3.60cwrk hottest 0.56c mean 0.53c------------------------------mean request 77.31 ms before 32.88 envoy 16.68 sidecar 1.28 worker 26.46 resume 1.23 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 62.4 ms p95 145.4 msin-flight 1558 pool 3697------------------------------envoy 8.32c sidecar 3.36cwrk hottest 0.54c mean 0.52c------------------------------mean request 67.88 ms before 24.61 envoy 16.09 sidecar 1.23 worker 25.94 resume 1.18 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 31.6 ms p95 103.0 msin-flight 1581 pool 3697------------------------------envoy 8.29c sidecar 3.57cwrk hottest 0.56c mean 0.52c------------------------------mean request 40.44 ms before 17.22 envoy 8.63 sidecar 1.12 worker 13.47 resume 1.07 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (99.9% ok)p50 218.0 ms p95 642.0 msin-flight 9585 pool 9587------------------------------envoy 8.56c sidecar 3.52cwrk hottest 0.56c mean 0.54c------------------------------mean request 294.49 ms before 150.87 envoy 54.81 sidecar 2.43 worker 86.38 resume 2.35 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 231.6 ms p95 667.3 msin-flight 9745 pool 9746------------------------------envoy 8.48c sidecar 2.99cwrk hottest 0.56c mean 0.53c------------------------------mean request 285.65 ms before 79.02 envoy 86.03 sidecar 1.84 worker 118.76 resume 1.77 (control plane) +rung 14 15k qps setpoint [warmup]------------------------------offered 15k /sachieved 14k /s (99.2% ok)p50 800.2 ms p95 1586.2 msin-flight 24k pool 24k------------------------------envoy 8.53c sidecar 2.85cwrk hottest 0.56c mean 0.53c------------------------------mean request 801.73 ms before 228.58 envoy 390.90 sidecar 2.90 worker 179.34 resume 2.78 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 855 /sachieved 855 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 24.0 pool 24.0------------------------------envoy 0.49c sidecar 0.13cwrk hottest 0.23c mean 0.03c------------------------------mean request 3.12 ms before 1.08 envoy 0.72 sidecar 0.78 worker 0.55 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.48cwrk hottest 0.23c mean 0.03c------------------------------mean request 2.87 ms before 1.09 envoy 0.73 sidecar 0.77 worker 0.28 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 8.0 pool 24.0------------------------------envoy 0.48c sidecar 0.48cwrk hottest 0.23c mean 0.04c------------------------------mean request 2.85 ms before 1.11 envoy 0.73 sidecar 0.78 worker 0.23 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1945 /sachieved 1945 /s (100.0% ok)p50 2.7 ms p95 3.5 msin-flight 9.0 pool 24.0------------------------------envoy 1.06c sidecar 0.74cwrk hottest 0.26c mean 0.07c------------------------------mean request 2.79 ms before 0.95 envoy 0.84 sidecar 0.79 worker 0.22 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.7 msin-flight 178 pool 178------------------------------envoy 1.07c sidecar 0.92cwrk hottest 0.44c mean 0.07c------------------------------mean request 3.01 ms before 1.17 envoy 0.87 sidecar 0.81 worker 0.15 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 22.0 pool 178------------------------------envoy 0.97c sidecar 0.88cwrk hottest 0.50c mean 0.06c------------------------------mean request 3.24 ms before 1.09 envoy 0.89 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2658 /sachieved 2658 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 178------------------------------envoy 1.25c sidecar 1.04cwrk hottest 0.67c mean 0.08c------------------------------mean request 2.97 ms before 1.05 envoy 0.70 sidecar 0.80 worker 0.41 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 178------------------------------envoy 1.45c sidecar 1.21cwrk hottest 0.73c mean 0.09c------------------------------mean request 2.99 ms before 1.03 envoy 0.69 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 17.0 pool 178------------------------------envoy 1.40c sidecar 1.22cwrk hottest 0.64c mean 0.09c------------------------------mean request 2.96 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.43 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3877 /sachieved 3877 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 22.0 pool 178------------------------------envoy 1.90c sidecar 1.38cwrk hottest 0.67c mean 0.11c------------------------------mean request 2.94 ms before 1.02 envoy 0.69 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 23.0 pool 178------------------------------envoy 1.94c sidecar 0.00cwrk hottest 0.77c mean 0.12c------------------------------mean request 3.03 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.50 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 37.0 pool 178------------------------------envoy 1.97c sidecar 1.55cwrk hottest 0.66c mean 0.12c------------------------------mean request 2.90 ms before 0.97 envoy 0.72 sidecar 0.80 worker 0.41 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4347 /sachieved 4347 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 36.0 pool 178------------------------------envoy 2.15c sidecar 1.51cwrk hottest 0.70c mean 0.14c------------------------------mean request 2.93 ms before 1.01 envoy 0.70 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 99.0 pool 100------------------------------envoy 2.57c sidecar 1.84cwrk hottest 0.67c mean 0.16c------------------------------mean request 2.92 ms before 1.00 envoy 0.72 sidecar 0.79 worker 0.41 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 73.0 pool 100------------------------------envoy 2.58c sidecar 1.91cwrk hottest 0.71c mean 0.16c------------------------------mean request 2.93 ms before 0.99 envoy 0.74 sidecar 0.80 worker 0.40 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 2.8 ms p95 4.4 msin-flight 37.0 pool 100------------------------------envoy 2.59c sidecar 1.90cwrk hottest 0.79c mean 0.17c------------------------------mean request 3.01 ms before 1.05 envoy 0.76 sidecar 0.79 worker 0.42 resume 0.75 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5953 /sachieved 5953 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 547 pool 547------------------------------envoy 3.23c sidecar 2.07cwrk hottest 0.66c mean 0.21c------------------------------mean request 3.75 ms before 1.72 envoy 0.33 sidecar 1.21 worker 0.48 resume 1.17 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 57.0 pool 547------------------------------envoy 3.35c sidecar 0.00cwrk hottest 0.65c mean 0.21c------------------------------mean request 3.00 ms before 0.33 envoy 1.29 sidecar 0.81 worker 0.58 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 52.0 pool 547------------------------------envoy 3.32c sidecar 2.27cwrk hottest 0.72c mean 0.20c------------------------------mean request 3.06 ms before 1.02 envoy 0.79 sidecar 0.80 worker 0.44 resume 0.76 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 71.0 pool 547------------------------------envoy 3.27c sidecar 2.25cwrk hottest 0.75c mean 0.21c------------------------------mean request 3.03 ms before 1.00 envoy 0.76 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6766 /sachieved 6766 /s (100.0% ok)p50 3.0 ms p95 6.5 msin-flight 636 pool 637------------------------------envoy 3.74c sidecar 0.00cwrk hottest 0.80c mean 0.23c------------------------------mean request 4.34 ms before 1.11 envoy 1.06 sidecar 1.29 worker 0.87 resume 1.25 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 6.9 msin-flight 109 pool 637------------------------------envoy 3.70c sidecar 2.41cwrk hottest 0.82c mean 0.23c------------------------------mean request 3.65 ms before 1.28 envoy 0.84 sidecar 0.84 worker 0.69 resume 0.80 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 5.6 msin-flight 84.0 pool 637------------------------------envoy 3.81c sidecar 2.50cwrk hottest 0.83c mean 0.24c------------------------------mean request 3.34 ms before 1.13 envoy 0.80 sidecar 0.82 worker 0.59 resume 0.78 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7527 /sachieved 7527 /s (100.0% ok)p50 3.1 ms p95 7.6 msin-flight 1329 pool 1329------------------------------envoy 4.14c sidecar 2.53cwrk hottest 0.83c mean 0.27c------------------------------mean request 3.83 ms before 1.34 envoy -1.82 sidecar 3.56 worker 0.75 resume 3.57 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (98.9% ok)p50 3.5 ms p95 9.9 msin-flight 857 pool 1329------------------------------envoy 4.35c sidecar 2.71cwrk hottest 0.77c mean 0.27c------------------------------mean request 8.30 ms before 2.19 envoy 3.05 sidecar 1.42 worker 1.63 resume 1.38 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7998 /s (100.0% ok)p50 3.6 ms p95 13.7 msin-flight 942 pool 1329------------------------------envoy 4.18c sidecar 2.70cwrk hottest 0.81c mean 0.26c------------------------------mean request 5.80 ms before 1.66 envoy 1.23 sidecar 1.36 worker 1.55 resume 1.32 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8024 /sachieved 8022 /s (99.7% ok)p50 3.4 ms p95 18.0 msin-flight 1305 pool 1329------------------------------envoy 4.36c sidecar 0.00cwrk hottest 0.83c mean 0.28c------------------------------mean request 7.58 ms before 1.62 envoy 1.75 sidecar 2.13 worker 2.09 resume 2.09 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 9000 /sachieved 9002 /s (100.0% ok)p50 3.8 ms p95 23.4 msin-flight 979 pool 1329------------------------------envoy 4.94c sidecar 2.69cwrk hottest 0.80c mean 0.31c------------------------------mean request 8.13 ms before 2.74 envoy 1.55 sidecar 1.64 worker 2.20 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8996 /s (98.9% ok)p50 4.3 ms p95 159.9 msin-flight 3186 pool 3187------------------------------envoy 4.90c sidecar 2.86cwrk hottest 0.82c mean 0.30c------------------------------mean request 23.13 ms before 9.44 envoy 4.08 sidecar 3.02 worker 6.58 resume 3.01 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9405 /sachieved 9402 /s (100.0% ok)p50 4.6 ms p95 45.2 msin-flight 284 pool 3187------------------------------envoy 4.85c sidecar 2.84cwrk hottest 0.82c mean 0.31c------------------------------mean request 10.74 ms before 4.78 envoy 1.96 sidecar 0.94 worker 3.07 resume 0.89 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 4.6 ms p95 66.8 msin-flight 321 pool 3187------------------------------envoy 5.18c sidecar 3.03cwrk hottest 0.78c mean 0.32c------------------------------mean request 14.58 ms before 5.88 envoy 2.90 sidecar 1.00 worker 4.80 resume 0.96 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 5.3 ms p95 68.2 msin-flight 428 pool 3187------------------------------envoy 5.31c sidecar 3.09cwrk hottest 0.77c mean 0.33c------------------------------mean request 15.14 ms before 6.10 envoy 3.06 sidecar 1.01 worker 4.98 resume 0.97 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (98.5% ok)p50 6.1 ms p95 78.3 msin-flight 1575 pool 3187------------------------------envoy 6.04c sidecar 3.18cwrk hottest 0.71c mean 0.38c------------------------------mean request 17.57 ms before 3.99 envoy 4.28 sidecar 2.85 worker 6.44 resume 2.86 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (99.2% ok)p50 6.3 ms p95 32.3 msin-flight 1472 pool 3187------------------------------envoy 6.39c sidecar 3.39cwrk hottest 0.69c mean 0.40c------------------------------mean request 11.93 ms before 5.22 envoy 1.64 sidecar 2.10 worker 2.96 resume 2.08 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.1 ms p95 23.8 msin-flight 777 pool 3187------------------------------envoy 6.26c sidecar 3.36cwrk hottest 0.71c mean 0.39c------------------------------mean request 9.02 ms before 2.38 envoy 2.42 sidecar 1.02 worker 3.19 resume 0.98 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 7.4 ms p95 42.5 msin-flight 1304 pool 3187------------------------------envoy 6.52c sidecar 0.00cwrk hottest 0.71c mean 0.41c------------------------------mean request 14.08 ms before 5.11 envoy 3.11 sidecar 1.34 worker 4.52 resume 1.29 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.2 ms p95 53.2 msin-flight 591 pool 1574------------------------------envoy 7.02c sidecar 3.48cwrk hottest 0.65c mean 0.44c------------------------------mean request 18.41 ms before 7.17 envoy 4.06 sidecar 1.00 worker 6.19 resume 0.95 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 8.8 ms p95 54.8 msin-flight 1644 pool 1645------------------------------envoy 7.26c sidecar 0.00cwrk hottest 0.63c mean 0.46c------------------------------mean request 16.50 ms before 4.93 envoy 4.09 sidecar 1.49 worker 5.98 resume 1.45 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.0 ms p95 107.7 msin-flight 1556 pool 1645------------------------------envoy 7.54c sidecar 3.59cwrk hottest 0.66c mean 0.47c------------------------------mean request 27.77 ms before 14.61 envoy 4.71 sidecar 1.14 worker 7.31 resume 1.09 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 51.3 ms p95 252.3 msin-flight 3711 pool 3697------------------------------envoy 8.49c sidecar 3.60cwrk hottest 0.56c mean 0.53c------------------------------mean request 77.31 ms before 32.88 envoy 16.68 sidecar 1.28 worker 26.46 resume 1.23 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 62.4 ms p95 145.4 msin-flight 1558 pool 3697------------------------------envoy 8.32c sidecar 3.36cwrk hottest 0.54c mean 0.52c------------------------------mean request 67.88 ms before 24.61 envoy 16.09 sidecar 1.23 worker 25.94 resume 1.18 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 31.6 ms p95 103.0 msin-flight 1581 pool 3697------------------------------envoy 8.29c sidecar 3.57cwrk hottest 0.56c mean 0.52c------------------------------mean request 40.44 ms before 17.22 envoy 8.63 sidecar 1.12 worker 13.47 resume 1.07 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (99.9% ok)p50 218.0 ms p95 642.0 msin-flight 9585 pool 9587------------------------------envoy 8.56c sidecar 3.52cwrk hottest 0.56c mean 0.54c------------------------------mean request 294.49 ms before 150.87 envoy 54.81 sidecar 2.43 worker 86.38 resume 2.35 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 231.6 ms p95 667.3 msin-flight 9745 pool 9746------------------------------envoy 8.48c sidecar 2.99cwrk hottest 0.56c mean 0.53c------------------------------mean request 285.65 ms before 79.02 envoy 86.03 sidecar 1.84 worker 118.76 resume 1.77 (control plane) +rung 14 15k qps setpoint [warmup]------------------------------offered 15k /sachieved 14k /s (99.2% ok)p50 800.2 ms p95 1586.2 msin-flight 24k pool 24k------------------------------envoy 8.53c sidecar 2.85cwrk hottest 0.56c mean 0.53c------------------------------mean request 801.73 ms before 228.58 envoy 390.90 sidecar 2.90 worker 179.34 resume 2.78 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 855 /sachieved 855 /s (100.0% ok)p50 2.9 ms p95 4.5 msin-flight 24.0 pool 24.0------------------------------envoy 0.49c sidecar 0.13cwrk hottest 0.23c mean 0.03c------------------------------mean request 3.12 ms before 1.08 envoy 0.72 sidecar 0.78 worker 0.55 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 5.0 pool 24.0------------------------------envoy 0.49c sidecar 0.48cwrk hottest 0.23c mean 0.03c------------------------------mean request 2.87 ms before 1.09 envoy 0.73 sidecar 0.77 worker 0.28 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.8 ms p95 3.5 msin-flight 8.0 pool 24.0------------------------------envoy 0.48c sidecar 0.48cwrk hottest 0.23c mean 0.04c------------------------------mean request 2.85 ms before 1.11 envoy 0.73 sidecar 0.78 worker 0.23 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1945 /sachieved 1945 /s (100.0% ok)p50 2.7 ms p95 3.5 msin-flight 9.0 pool 24.0------------------------------envoy 1.06c sidecar 0.74cwrk hottest 0.26c mean 0.07c------------------------------mean request 2.79 ms before 0.95 envoy 0.84 sidecar 0.79 worker 0.22 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.7 ms p95 3.7 msin-flight 178 pool 178------------------------------envoy 1.07c sidecar 0.92cwrk hottest 0.44c mean 0.07c------------------------------mean request 3.01 ms before 1.17 envoy 0.87 sidecar 0.81 worker 0.15 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.8 msin-flight 22.0 pool 178------------------------------envoy 0.97c sidecar 0.88cwrk hottest 0.50c mean 0.06c------------------------------mean request 3.24 ms before 1.09 envoy 0.89 sidecar 0.81 worker 0.45 resume 0.77 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2658 /sachieved 2658 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 178------------------------------envoy 1.25c sidecar 1.04cwrk hottest 0.67c mean 0.08c------------------------------mean request 2.97 ms before 1.05 envoy 0.70 sidecar 0.80 worker 0.41 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 28.0 pool 178------------------------------envoy 1.45c sidecar 1.21cwrk hottest 0.73c mean 0.09c------------------------------mean request 2.99 ms before 1.03 envoy 0.69 sidecar 0.82 worker 0.45 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 17.0 pool 178------------------------------envoy 1.40c sidecar 1.22cwrk hottest 0.64c mean 0.09c------------------------------mean request 2.96 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.43 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3877 /sachieved 3877 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 22.0 pool 178------------------------------envoy 1.90c sidecar 1.38cwrk hottest 0.67c mean 0.11c------------------------------mean request 2.94 ms before 1.02 envoy 0.69 sidecar 0.81 worker 0.42 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.9 ms p95 4.2 msin-flight 23.0 pool 178------------------------------envoy 1.94c sidecar 0.00cwrk hottest 0.77c mean 0.12c------------------------------mean request 3.03 ms before 1.05 envoy 0.68 sidecar 0.80 worker 0.50 resume 0.76 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 37.0 pool 178------------------------------envoy 1.97c sidecar 1.55cwrk hottest 0.66c mean 0.12c------------------------------mean request 2.90 ms before 0.97 envoy 0.72 sidecar 0.80 worker 0.41 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4347 /sachieved 4347 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 36.0 pool 178------------------------------envoy 2.15c sidecar 1.51cwrk hottest 0.70c mean 0.14c------------------------------mean request 2.93 ms before 1.01 envoy 0.70 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 2.8 ms p95 3.9 msin-flight 99.0 pool 100------------------------------envoy 2.57c sidecar 1.84cwrk hottest 0.67c mean 0.16c------------------------------mean request 2.92 ms before 1.00 envoy 0.72 sidecar 0.79 worker 0.41 resume 0.75 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 2.8 ms p95 4.0 msin-flight 73.0 pool 100------------------------------envoy 2.58c sidecar 1.91cwrk hottest 0.71c mean 0.16c------------------------------mean request 2.93 ms before 0.99 envoy 0.74 sidecar 0.80 worker 0.40 resume 0.76 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 2.8 ms p95 4.4 msin-flight 37.0 pool 100------------------------------envoy 2.59c sidecar 1.90cwrk hottest 0.79c mean 0.17c------------------------------mean request 3.01 ms before 1.05 envoy 0.76 sidecar 0.79 worker 0.42 resume 0.75 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5953 /sachieved 5953 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 547 pool 547------------------------------envoy 3.23c sidecar 2.07cwrk hottest 0.66c mean 0.21c------------------------------mean request 3.75 ms before 1.72 envoy 0.33 sidecar 1.21 worker 0.48 resume 1.17 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.1 msin-flight 57.0 pool 547------------------------------envoy 3.35c sidecar 0.00cwrk hottest 0.65c mean 0.21c------------------------------mean request 3.00 ms before 0.33 envoy 1.29 sidecar 0.81 worker 0.58 resume 0.77 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.4 msin-flight 52.0 pool 547------------------------------envoy 3.32c sidecar 2.27cwrk hottest 0.72c mean 0.20c------------------------------mean request 3.06 ms before 1.02 envoy 0.79 sidecar 0.80 worker 0.44 resume 0.76 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 2.9 ms p95 4.3 msin-flight 71.0 pool 547------------------------------envoy 3.27c sidecar 2.25cwrk hottest 0.75c mean 0.21c------------------------------mean request 3.03 ms before 1.00 envoy 0.76 sidecar 0.81 worker 0.47 resume 0.77 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6766 /sachieved 6766 /s (100.0% ok)p50 3.0 ms p95 6.5 msin-flight 636 pool 637------------------------------envoy 3.74c sidecar 0.00cwrk hottest 0.80c mean 0.23c------------------------------mean request 4.34 ms before 1.11 envoy 1.06 sidecar 1.29 worker 0.87 resume 1.25 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 6.9 msin-flight 109 pool 637------------------------------envoy 3.70c sidecar 2.41cwrk hottest 0.82c mean 0.23c------------------------------mean request 3.65 ms before 1.28 envoy 0.84 sidecar 0.84 worker 0.69 resume 0.80 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 3.0 ms p95 5.6 msin-flight 84.0 pool 637------------------------------envoy 3.81c sidecar 2.50cwrk hottest 0.83c mean 0.24c------------------------------mean request 3.34 ms before 1.13 envoy 0.80 sidecar 0.82 worker 0.59 resume 0.78 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7527 /sachieved 7527 /s (100.0% ok)p50 3.1 ms p95 7.6 msin-flight 1329 pool 1329------------------------------envoy 4.14c sidecar 2.53cwrk hottest 0.83c mean 0.27c------------------------------mean request 3.83 ms before 1.34 envoy -1.82 sidecar 3.56 worker 0.75 resume 3.57 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8000 /s (98.9% ok)p50 3.5 ms p95 9.9 msin-flight 857 pool 1329------------------------------envoy 4.35c sidecar 2.71cwrk hottest 0.77c mean 0.27c------------------------------mean request 8.30 ms before 2.19 envoy 3.05 sidecar 1.42 worker 1.63 resume 1.38 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7998 /s (100.0% ok)p50 3.6 ms p95 13.7 msin-flight 942 pool 1329------------------------------envoy 4.18c sidecar 2.70cwrk hottest 0.81c mean 0.26c------------------------------mean request 5.80 ms before 1.66 envoy 1.23 sidecar 1.36 worker 1.55 resume 1.32 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8024 /sachieved 8022 /s (99.7% ok)p50 3.4 ms p95 18.0 msin-flight 1305 pool 1329------------------------------envoy 4.36c sidecar 0.00cwrk hottest 0.83c mean 0.28c------------------------------mean request 7.58 ms before 1.62 envoy 1.75 sidecar 2.13 worker 2.09 resume 2.09 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 9000 /sachieved 9002 /s (100.0% ok)p50 3.8 ms p95 23.4 msin-flight 979 pool 1329------------------------------envoy 4.94c sidecar 2.69cwrk hottest 0.80c mean 0.31c------------------------------mean request 8.13 ms before 2.74 envoy 1.55 sidecar 1.64 worker 2.20 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 8996 /s (98.9% ok)p50 4.3 ms p95 159.9 msin-flight 3186 pool 3187------------------------------envoy 4.90c sidecar 2.86cwrk hottest 0.82c mean 0.30c------------------------------mean request 23.13 ms before 9.44 envoy 4.08 sidecar 3.02 worker 6.58 resume 3.01 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9405 /sachieved 9402 /s (100.0% ok)p50 4.6 ms p95 45.2 msin-flight 284 pool 3187------------------------------envoy 4.85c sidecar 2.84cwrk hottest 0.82c mean 0.31c------------------------------mean request 10.74 ms before 4.78 envoy 1.96 sidecar 0.94 worker 3.07 resume 0.89 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 4.6 ms p95 66.8 msin-flight 321 pool 3187------------------------------envoy 5.18c sidecar 3.03cwrk hottest 0.78c mean 0.32c------------------------------mean request 14.58 ms before 5.88 envoy 2.90 sidecar 1.00 worker 4.80 resume 0.96 (control plane) +rung 9 10k qps setpoint------------------------------offered 10k /sachieved 9999 /s (100.0% ok)p50 5.3 ms p95 68.2 msin-flight 428 pool 3187------------------------------envoy 5.31c sidecar 3.09cwrk hottest 0.77c mean 0.33c------------------------------mean request 15.14 ms before 6.10 envoy 3.06 sidecar 1.01 worker 4.98 resume 0.97 (control plane) +rung 10 11k qps setpoint [warmup]------------------------------offered 11k /sachieved 11k /s (98.5% ok)p50 6.1 ms p95 78.3 msin-flight 1575 pool 3187------------------------------envoy 6.04c sidecar 3.18cwrk hottest 0.71c mean 0.38c------------------------------mean request 17.57 ms before 3.99 envoy 4.28 sidecar 2.85 worker 6.44 resume 2.86 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (99.2% ok)p50 6.3 ms p95 32.3 msin-flight 1472 pool 3187------------------------------envoy 6.39c sidecar 3.39cwrk hottest 0.69c mean 0.40c------------------------------mean request 11.93 ms before 5.22 envoy 1.64 sidecar 2.10 worker 2.96 resume 2.08 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 6.1 ms p95 23.8 msin-flight 777 pool 3187------------------------------envoy 6.26c sidecar 3.36cwrk hottest 0.71c mean 0.39c------------------------------mean request 9.02 ms before 2.38 envoy 2.42 sidecar 1.02 worker 3.19 resume 0.98 (control plane) +rung 10 11k qps setpoint------------------------------offered 11k /sachieved 11k /s (100.0% ok)p50 7.4 ms p95 42.5 msin-flight 1304 pool 3187------------------------------envoy 6.52c sidecar 0.00cwrk hottest 0.71c mean 0.41c------------------------------mean request 14.08 ms before 5.11 envoy 3.11 sidecar 1.34 worker 4.52 resume 1.29 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.2 ms p95 53.2 msin-flight 591 pool 1574------------------------------envoy 7.02c sidecar 3.48cwrk hottest 0.65c mean 0.44c------------------------------mean request 18.41 ms before 7.17 envoy 4.06 sidecar 1.00 worker 6.19 resume 0.95 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (99.9% ok)p50 8.8 ms p95 54.8 msin-flight 1644 pool 1645------------------------------envoy 7.26c sidecar 0.00cwrk hottest 0.63c mean 0.46c------------------------------mean request 16.50 ms before 4.93 envoy 4.09 sidecar 1.49 worker 5.98 resume 1.45 (control plane) +rung 11 12k qps setpoint------------------------------offered 12k /sachieved 12k /s (100.0% ok)p50 12.0 ms p95 107.7 msin-flight 1556 pool 1645------------------------------envoy 7.54c sidecar 3.59cwrk hottest 0.66c mean 0.47c------------------------------mean request 27.77 ms before 14.61 envoy 4.71 sidecar 1.14 worker 7.31 resume 1.09 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 51.3 ms p95 252.3 msin-flight 3711 pool 3697------------------------------envoy 8.49c sidecar 3.60cwrk hottest 0.56c mean 0.53c------------------------------mean request 77.31 ms before 32.88 envoy 16.68 sidecar 1.28 worker 26.46 resume 1.23 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 62.4 ms p95 145.4 msin-flight 1558 pool 3697------------------------------envoy 8.32c sidecar 3.36cwrk hottest 0.54c mean 0.52c------------------------------mean request 67.88 ms before 24.61 envoy 16.09 sidecar 1.23 worker 25.94 resume 1.18 (control plane) +rung 12 13k qps setpoint------------------------------offered 13k /sachieved 13k /s (100.0% ok)p50 31.6 ms p95 103.0 msin-flight 1581 pool 3697------------------------------envoy 8.29c sidecar 3.57cwrk hottest 0.56c mean 0.52c------------------------------mean request 40.44 ms before 17.22 envoy 8.63 sidecar 1.12 worker 13.47 resume 1.07 (control plane) +rung 13 14k qps setpoint [warmup]------------------------------offered 14k /sachieved 14k /s (99.9% ok)p50 218.0 ms p95 642.0 msin-flight 9585 pool 9587------------------------------envoy 8.56c sidecar 3.52cwrk hottest 0.56c mean 0.54c------------------------------mean request 294.49 ms before 150.87 envoy 54.81 sidecar 2.43 worker 86.38 resume 2.35 (control plane) +rung 13 14k qps setpoint------------------------------offered 14k /sachieved 14k /s (100.0% ok)p50 231.6 ms p95 667.3 msin-flight 9745 pool 9746------------------------------envoy 8.48c sidecar 2.99cwrk hottest 0.56c mean 0.53c------------------------------mean request 285.65 ms before 79.02 envoy 86.03 sidecar 1.84 worker 118.76 resume 1.77 (control plane) +rung 14 15k qps setpoint [warmup]------------------------------offered 15k /sachieved 14k /s (99.2% ok)p50 800.2 ms p95 1586.2 msin-flight 24k pool 24k------------------------------envoy 8.53c sidecar 2.85cwrk hottest 0.56c mean 0.53c------------------------------mean request 801.73 ms before 228.58 envoy 390.90 sidecar 2.90 worker 179.34 resume 2.78 (control plane) +
+ +

Every measured rung

+
Warmup windows excluded. Latency, in-flight and memory are the worst window in the rung; +throughput and CPU are the mean across its windows. The highlighted row is each arm's last sustained rung.
+ + +
armrungofferedachievedsuccesssuccess %p50 msp95 msin-flightenvoy c meansidecar c meanenvoy mem
2c0118511851185100.00%3.04.45170.550.5362.3M
2c1200020002000100.00%3.04.173.00.880.8761.7M
2c2300730073007100.00%3.25.03861.251.1463.0M
2c3400040004000100.00%3.45.133.01.571.4063.7M
2c4503750335033100.00%6.7251.120681.841.50168.1M
2c560005178516899.80%1538.72542.719k1.921.191.2G
4c0105310531053100.00%2.93.645.00.530.5146.8M
4c1204920482048100.00%2.94.176.00.930.7948.7M
4c2304730473047100.00%2.83.920.01.321.1648.5M
4c3407540754075100.00%3.04.332.01.741.4448.8M
4c4518051805180100.00%3.05.03632.201.7266.4M
4c560006000599299.87%4.9339.425842.481.78224.7M
4c6700070007000100.00%4.717.09732.882.09225.3M
4c7800080008000100.00%6.621.73363.222.27206.9M
4c8900090109010100.00%14.247.15553.462.32191.5M
2×4c-1267267267100.00%2.95.820.00.080.0927.6M
2×4c0113711371137100.00%2.93.514.00.210.2930.7M
2×4c1209020902090100.00%2.73.412.00.510.5131.9M
2×4c2308530853085100.00%2.73.424.00.790.7329.5M
2×4c3416441644164100.00%2.94.35321.000.9163.5M
2×4c4512151215121100.00%3.25.01541.161.1063.7M
2×4c5600060006000100.00%3.05.118671.391.2896.9M
2×4c6705870587058100.00%3.414.416111.721.4398.5M
2×4c7808480848084100.00%4.318.222201.951.62115.8M
2×4c8911391119111100.00%3.618.87272.221.78115.9M
2×4c910k10k999099.62%4.078.429332.401.90153.2M
2×4c1011k11k11k100.00%4.2159.124032.772.04195.3M
2×4c1112k12k12k100.00%4.314.03503.032.23259.4M
2×4c1213k13k13k100.00%4.917.121183.252.39181.3M
2×4c1314k14k14k100.00%7.451.424233.412.45244.3M
2×4c1415k15k15k99.88%16.196.435653.452.51243.7M
2×4c1516k16k16k100.00%21.677.411313.522.43229.0M
2×4c1617k17k17k99.85%137.81634.316k3.512.201.1G
8c/2t-1146146146100.00%3.212.133.00.090.0030.7M
8c/2t0100010001000100.00%3.03.86.00.490.5232.2M
8c/2t1200020002000100.00%3.34.61370.860.8938.4M
8c/2t2306730673067100.00%3.25.12201.291.1643.8M
8c/2t3400040004000100.00%3.45.134.01.601.4344.9M
8c/2t4500050455045100.00%6.0247.314001.861.40286.6M
8c/2t561845737498486.87%3131.15534.421k1.931.031.4G
8c/4t-1191191191100.00%3.19.035.00.130.0138.9M
8c/4t0110011001100100.00%3.14.85710.540.4979.5M
8c/4t1200020002000100.00%3.14.324.00.870.8180.6M
8c/4t2300030013001100.00%3.77.892.01.201.0581.6M
8c/4t3413741364136100.00%4.616.91881.681.3180.5M
8c/4t4505150505050100.00%4.628.91592.101.6079.6M
8c/4t5611361146114100.00%3.8252.524272.771.81213.7M
8c/4t6700069996999100.00%4.813.410733.062.07213.4M
8c/4t7800079997999100.00%11.333.58693.252.16206.6M
8c/4t890008919891199.92%37.6276.930583.442.05261.6M
8c/8t-183.983.783.7100.00%3.611.224.00.070.0038.8M
8c/8t0100010001000100.00%2.94.53050.520.5073.0M
8c/8t1210021002100100.00%3.04.127.00.900.8377.4M
8c/8t2301830183018100.00%3.14.943.01.291.0878.8M
8c/8t3409540954095100.00%3.46.452.01.621.3779.5M
8c/8t4505050505050100.00%3.36.955.02.211.6281.0M
8c/8t5600060006000100.00%3.18.81272.872.0381.6M
8c/8t671657165713599.59%3.59.613843.392.28165.5M
8c/8t7800080018001100.00%4.414.41673.662.44165.7M
8c/8t8900090159015100.00%17.8661.464614.272.50499.9M
8c/8t910k10k10k100.00%8.462.48554.602.79498.2M
8c/8t1011k11k11k100.00%11.871.57415.152.95392.2M
8c/8t1112k12k12k99.90%23.5239.565975.612.98403.5M
8c/16t-10000.00%0.00.030.00.010.0019.9M
8c/16t0105610561056100.00%2.93.712.00.570.5260.6M
8c/16t1208820882088100.00%2.94.47311.100.91148.5M
8c/16t2310531053105100.00%3.04.52721.431.17149.6M
8c/16t3412341234123100.00%3.131.99891.951.46185.2M
8c/16t4500050005000100.00%3.04.78942.571.84185.1M
8c/16t5600060006000100.00%3.354.817192.922.05226.2M
8c/16t671137112706099.28%3.527.513273.612.33272.9M
8c/16t780008001796599.55%3.632.013554.212.61268.8M
8c/16t8900090009000100.00%3.840.41804.912.85263.0M
8c/16t910k10k10k100.00%13.9999.811k5.782.88820.2M
8c/16t1011k11k11k100.00%9.286.714226.213.22820.3M
8c/16t1112k12k12k100.00%38.3233.719177.283.30646.8M
8c/16t1213k13k13k99.93%110.5457.837457.563.03735.9M
8c/16t1312k12k12k96.59%1698.91976.824k6.992.431.8G
16c0100010001000100.00%2.83.58.00.480.4864.5M
16c1200020002000100.00%2.83.81781.020.90108.4M
16c2300030003000100.00%2.94.128.01.431.21114.7M
16c3411641164116100.00%2.94.237.02.021.53119.8M
16c4500050005000100.00%2.84.499.02.581.88126.5M
16c5600060006000100.00%2.94.471.03.312.26151.5M
16c6700070007000100.00%3.06.91093.762.46157.2M
16c780088007796899.52%3.618.013054.302.70211.1M
16c892029199914799.44%4.6159.931864.872.85324.6M
16c910k99999999100.00%5.368.24285.253.06325.8M
16c1011k11k11k99.73%7.442.514726.393.37307.8M
16c1112k12k12k99.97%12.2107.716447.283.53304.2M
16c1213k13k13k100.00%62.4252.337118.373.51409.5M
16c1314k14k14k100.00%231.6667.397458.482.99827.8M
+ +
diff --git a/benchmarking/routercap/runs/v139-w200/report.html b/benchmarking/routercap/runs/v139-w200/report.html new file mode 100644 index 000000000..4f732df05 --- /dev/null +++ b/benchmarking/routercap/runs/v139-w200/report.html @@ -0,0 +1,5079 @@ + + + +atenet-router capacity + +
+
+

atenet-router capacity

+
Generated by running python3 benchmarking/routercap/charts.py benchmarking/routercap/runs/v139-w200
+
+ +

Headline

+
Sustainable is the highest rung whose measured windows, summed, kept both completed and +successful requests within 1% of what was offered, with no window's median above 100 ms.
+
2c
4,000 qps sustainable
held through rung 3 of 4 · 2,000 qps per core
4c
6,000 qps sustainable
held through rung 5 of 6 · 1,500 qps per core
8c/2t
4,000 qps sustainable
held through rung 3 of 4 · 500 qps per core
8c/4t
6,000 qps sustainable
held through rung 5 of 6 · 750 qps per core
8c/8t
7,000 qps sustainable
held through rung 6 of 7 · 875 qps per core
16c
8,000 qps sustainable
held through rung 7 of 9 · 500 qps per core
16c pinned
7,000 qps sustainable
held through rung 6 of 8 · 438 qps per core
+ +

Legend

+
Throughput panel
  • offered — requests the pacer scheduled — the independent variable
  • achieved — requests that came back at all
  • success — the achieved subset that returned HTTP 200
  • in-flight — sent and not yet answered — a count, not a rate
  • client pool — TCP connections the generator holds open; steps up when a stall makes blocked requests dial, and holds for the 120 s idle timeout
Time-share panel (stacked bands, bottom up)
  • before Envoy — queued in the generator, plus the dial
  • Envoy itself — Envoy's own work plus the loopback to the sidecar
  • sidecar — the ext_proc handler, actor resume included
  • worker leg — request on the wire to the worker and back
  • hatched — Envoy rounds request times to whole milliseconds; where hatched, that rounding exceeds 5% of the mean request and the split is mostly instrument
CPU and memory panels
  • envoy — the proxy — its CPU limit is the variable under test
  • atenet-router — the Go ext_proc sidecar that picks the worker and resumes the actor; same limit in every arm
Per-thread CPU panel
  • hottest worker — the busiest single Envoy thread (from /proc): a connection stays on its thread for life, so one thread can saturate while the container average reads idle
  • mean worker — the same threads' average — the lines hugging is balanced load, a gap opening is skew: one thread drowning; at 1.0 the hottest thread is saturated outright
Chart furniture
  • grey stripes — alternate rungs, labelled R0, R1, … along the top
  • dashed lines — ceilings, each labelled at the line itself. "envoy sheds above 20,000 in flight" is the actor cluster's circuit breaker: past it Envoy answers 503 itself and counts the rejection, set below the port budget on purpose so overload shows up as an Envoy counter instead of the kernel's opaque connect failure
  • gaps — a hole in a CPU or memory line is a window where cAdvisor returned no fresh sample — nobody looked is not the same as it went idle
+ +

Per arm, over time

+
One chart per arm. The panels share one x-axis and one set of window boundaries, so a +vertical line through them is a single interval. Hover any column for that window's full numbers.
+
+ + + +atenet-router capacity — 2c +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + +R0 +R1 +R2 +R3 +R4 + +0 + +10k + +20k + +30k +requests / s + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + +0 + +0.5 + +1.0 + +1.5 + +2.0 +cores + + + + + + +envoy CPU limit, 2 cores — the variable under test + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + +0 + +200.0M + +400.0M + +600.0M +bytes + +50.0 + +100 + +150 + +200 +seconds since the arm started + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + +atenet-router +rung 0 1000 qps setpoint [warmup]------------------------------offered 917 /sachieved 916 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 31.0 pool 31.0------------------------------envoy 0.51c sidecar 0.17cwrk hottest 0.36c mean 0.27c------------------------------mean request 3.28 ms before 1.07 envoy 0.53 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 3.9 msin-flight 10.0 pool 31.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.36c mean 0.28c------------------------------mean request 3.12 ms before 1.05 envoy 0.47 sidecar 0.80 worker 0.80 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 31.0------------------------------envoy 0.55c sidecar 0.49cwrk hottest 0.42c mean 0.27c------------------------------mean request 3.08 ms before 1.02 envoy 0.60 sidecar 0.81 worker 0.66 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1782 /sachieved 1782 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 119 pool 120------------------------------envoy 0.93c sidecar 0.72cwrk hottest 0.68c mean 0.49c------------------------------mean request 3.45 ms before 1.23 envoy 0.58 sidecar 0.86 worker 0.78 resume 0.82 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 349 pool 349------------------------------envoy 1.07c sidecar 0.87cwrk hottest 0.71c mean 0.54c------------------------------mean request 4.93 ms before 1.91 envoy 0.81 sidecar 0.89 worker 1.33 resume 0.84 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 22.0 pool 349------------------------------envoy 1.01c sidecar 0.88cwrk hottest 0.67c mean 0.51c------------------------------mean request 3.18 ms before 1.11 envoy 0.54 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2991 /sachieved 2991 /s (100.0% ok)p50 3.7 ms p95 178.7 msin-flight 1057 pool 1057------------------------------envoy 1.48c sidecar 0.93cwrk hottest 0.81c mean 0.72c------------------------------mean request 22.65 ms before 10.53 envoy 4.25 sidecar 1.05 worker 6.82 resume 1.00 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.4 ms p95 4.7 msin-flight 42.0 pool 1057------------------------------envoy 1.47c sidecar 1.14cwrk hottest 0.74c mean 0.73c------------------------------mean request 3.54 ms before 1.31 envoy 0.47 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 4.9 msin-flight 42.0 pool 1057------------------------------envoy 1.49c sidecar 1.17cwrk hottest 0.80c mean 0.74c------------------------------mean request 3.60 ms before 1.33 envoy 0.50 sidecar 0.81 worker 0.97 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3920 /sachieved 3896 /s (100.0% ok)p50 7.1 ms p95 394.5 msin-flight 1948 pool 1948------------------------------envoy 1.74c sidecar 1.24cwrk hottest 0.93c mean 0.87c------------------------------mean request 91.11 ms before 68.19 envoy 8.85 sidecar 1.50 worker 12.56 resume 1.44 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4037 /s (100.0% ok)p50 6.3 ms p95 24.4 msin-flight 136 pool 1948------------------------------envoy 1.84c sidecar 1.15cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.59 ms before -72.20 envoy 48.74 sidecar 0.85 worker 32.20 resume 0.81 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 6.1 ms p95 23.7 msin-flight 256 pool 1948------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.93c mean 0.91c------------------------------mean request 8.65 ms before 3.39 envoy 1.63 sidecar 0.86 worker 2.77 resume 0.82 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4855 /sachieved 3966 /s (100.0% ok)p50 364.9 ms p95 1593.9 msin-flight 12k pool 12k------------------------------envoy 1.89c sidecar 1.29cwrk hottest 0.95c mean 0.94c------------------------------mean request 544.40 ms before 171.29 envoy 234.63 sidecar 3.27 worker 135.21 resume 3.07 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 917 /sachieved 916 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 31.0 pool 31.0------------------------------envoy 0.51c sidecar 0.17cwrk hottest 0.36c mean 0.27c------------------------------mean request 3.28 ms before 1.07 envoy 0.53 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 3.9 msin-flight 10.0 pool 31.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.36c mean 0.28c------------------------------mean request 3.12 ms before 1.05 envoy 0.47 sidecar 0.80 worker 0.80 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 31.0------------------------------envoy 0.55c sidecar 0.49cwrk hottest 0.42c mean 0.27c------------------------------mean request 3.08 ms before 1.02 envoy 0.60 sidecar 0.81 worker 0.66 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1782 /sachieved 1782 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 119 pool 120------------------------------envoy 0.93c sidecar 0.72cwrk hottest 0.68c mean 0.49c------------------------------mean request 3.45 ms before 1.23 envoy 0.58 sidecar 0.86 worker 0.78 resume 0.82 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 349 pool 349------------------------------envoy 1.07c sidecar 0.87cwrk hottest 0.71c mean 0.54c------------------------------mean request 4.93 ms before 1.91 envoy 0.81 sidecar 0.89 worker 1.33 resume 0.84 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 22.0 pool 349------------------------------envoy 1.01c sidecar 0.88cwrk hottest 0.67c mean 0.51c------------------------------mean request 3.18 ms before 1.11 envoy 0.54 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2991 /sachieved 2991 /s (100.0% ok)p50 3.7 ms p95 178.7 msin-flight 1057 pool 1057------------------------------envoy 1.48c sidecar 0.93cwrk hottest 0.81c mean 0.72c------------------------------mean request 22.65 ms before 10.53 envoy 4.25 sidecar 1.05 worker 6.82 resume 1.00 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.4 ms p95 4.7 msin-flight 42.0 pool 1057------------------------------envoy 1.47c sidecar 1.14cwrk hottest 0.74c mean 0.73c------------------------------mean request 3.54 ms before 1.31 envoy 0.47 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 4.9 msin-flight 42.0 pool 1057------------------------------envoy 1.49c sidecar 1.17cwrk hottest 0.80c mean 0.74c------------------------------mean request 3.60 ms before 1.33 envoy 0.50 sidecar 0.81 worker 0.97 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3920 /sachieved 3896 /s (100.0% ok)p50 7.1 ms p95 394.5 msin-flight 1948 pool 1948------------------------------envoy 1.74c sidecar 1.24cwrk hottest 0.93c mean 0.87c------------------------------mean request 91.11 ms before 68.19 envoy 8.85 sidecar 1.50 worker 12.56 resume 1.44 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4037 /s (100.0% ok)p50 6.3 ms p95 24.4 msin-flight 136 pool 1948------------------------------envoy 1.84c sidecar 1.15cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.59 ms before -72.20 envoy 48.74 sidecar 0.85 worker 32.20 resume 0.81 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 6.1 ms p95 23.7 msin-flight 256 pool 1948------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.93c mean 0.91c------------------------------mean request 8.65 ms before 3.39 envoy 1.63 sidecar 0.86 worker 2.77 resume 0.82 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4855 /sachieved 3966 /s (100.0% ok)p50 364.9 ms p95 1593.9 msin-flight 12k pool 12k------------------------------envoy 1.89c sidecar 1.29cwrk hottest 0.95c mean 0.94c------------------------------mean request 544.40 ms before 171.29 envoy 234.63 sidecar 3.27 worker 135.21 resume 3.07 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 917 /sachieved 916 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 31.0 pool 31.0------------------------------envoy 0.51c sidecar 0.17cwrk hottest 0.36c mean 0.27c------------------------------mean request 3.28 ms before 1.07 envoy 0.53 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 3.9 msin-flight 10.0 pool 31.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.36c mean 0.28c------------------------------mean request 3.12 ms before 1.05 envoy 0.47 sidecar 0.80 worker 0.80 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 31.0------------------------------envoy 0.55c sidecar 0.49cwrk hottest 0.42c mean 0.27c------------------------------mean request 3.08 ms before 1.02 envoy 0.60 sidecar 0.81 worker 0.66 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1782 /sachieved 1782 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 119 pool 120------------------------------envoy 0.93c sidecar 0.72cwrk hottest 0.68c mean 0.49c------------------------------mean request 3.45 ms before 1.23 envoy 0.58 sidecar 0.86 worker 0.78 resume 0.82 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 349 pool 349------------------------------envoy 1.07c sidecar 0.87cwrk hottest 0.71c mean 0.54c------------------------------mean request 4.93 ms before 1.91 envoy 0.81 sidecar 0.89 worker 1.33 resume 0.84 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 22.0 pool 349------------------------------envoy 1.01c sidecar 0.88cwrk hottest 0.67c mean 0.51c------------------------------mean request 3.18 ms before 1.11 envoy 0.54 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2991 /sachieved 2991 /s (100.0% ok)p50 3.7 ms p95 178.7 msin-flight 1057 pool 1057------------------------------envoy 1.48c sidecar 0.93cwrk hottest 0.81c mean 0.72c------------------------------mean request 22.65 ms before 10.53 envoy 4.25 sidecar 1.05 worker 6.82 resume 1.00 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.4 ms p95 4.7 msin-flight 42.0 pool 1057------------------------------envoy 1.47c sidecar 1.14cwrk hottest 0.74c mean 0.73c------------------------------mean request 3.54 ms before 1.31 envoy 0.47 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 4.9 msin-flight 42.0 pool 1057------------------------------envoy 1.49c sidecar 1.17cwrk hottest 0.80c mean 0.74c------------------------------mean request 3.60 ms before 1.33 envoy 0.50 sidecar 0.81 worker 0.97 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3920 /sachieved 3896 /s (100.0% ok)p50 7.1 ms p95 394.5 msin-flight 1948 pool 1948------------------------------envoy 1.74c sidecar 1.24cwrk hottest 0.93c mean 0.87c------------------------------mean request 91.11 ms before 68.19 envoy 8.85 sidecar 1.50 worker 12.56 resume 1.44 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4037 /s (100.0% ok)p50 6.3 ms p95 24.4 msin-flight 136 pool 1948------------------------------envoy 1.84c sidecar 1.15cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.59 ms before -72.20 envoy 48.74 sidecar 0.85 worker 32.20 resume 0.81 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 6.1 ms p95 23.7 msin-flight 256 pool 1948------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.93c mean 0.91c------------------------------mean request 8.65 ms before 3.39 envoy 1.63 sidecar 0.86 worker 2.77 resume 0.82 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4855 /sachieved 3966 /s (100.0% ok)p50 364.9 ms p95 1593.9 msin-flight 12k pool 12k------------------------------envoy 1.89c sidecar 1.29cwrk hottest 0.95c mean 0.94c------------------------------mean request 544.40 ms before 171.29 envoy 234.63 sidecar 3.27 worker 135.21 resume 3.07 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 917 /sachieved 916 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 31.0 pool 31.0------------------------------envoy 0.51c sidecar 0.17cwrk hottest 0.36c mean 0.27c------------------------------mean request 3.28 ms before 1.07 envoy 0.53 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 3.9 msin-flight 10.0 pool 31.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.36c mean 0.28c------------------------------mean request 3.12 ms before 1.05 envoy 0.47 sidecar 0.80 worker 0.80 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 31.0------------------------------envoy 0.55c sidecar 0.49cwrk hottest 0.42c mean 0.27c------------------------------mean request 3.08 ms before 1.02 envoy 0.60 sidecar 0.81 worker 0.66 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1782 /sachieved 1782 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 119 pool 120------------------------------envoy 0.93c sidecar 0.72cwrk hottest 0.68c mean 0.49c------------------------------mean request 3.45 ms before 1.23 envoy 0.58 sidecar 0.86 worker 0.78 resume 0.82 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 349 pool 349------------------------------envoy 1.07c sidecar 0.87cwrk hottest 0.71c mean 0.54c------------------------------mean request 4.93 ms before 1.91 envoy 0.81 sidecar 0.89 worker 1.33 resume 0.84 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 22.0 pool 349------------------------------envoy 1.01c sidecar 0.88cwrk hottest 0.67c mean 0.51c------------------------------mean request 3.18 ms before 1.11 envoy 0.54 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2991 /sachieved 2991 /s (100.0% ok)p50 3.7 ms p95 178.7 msin-flight 1057 pool 1057------------------------------envoy 1.48c sidecar 0.93cwrk hottest 0.81c mean 0.72c------------------------------mean request 22.65 ms before 10.53 envoy 4.25 sidecar 1.05 worker 6.82 resume 1.00 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.4 ms p95 4.7 msin-flight 42.0 pool 1057------------------------------envoy 1.47c sidecar 1.14cwrk hottest 0.74c mean 0.73c------------------------------mean request 3.54 ms before 1.31 envoy 0.47 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 4.9 msin-flight 42.0 pool 1057------------------------------envoy 1.49c sidecar 1.17cwrk hottest 0.80c mean 0.74c------------------------------mean request 3.60 ms before 1.33 envoy 0.50 sidecar 0.81 worker 0.97 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3920 /sachieved 3896 /s (100.0% ok)p50 7.1 ms p95 394.5 msin-flight 1948 pool 1948------------------------------envoy 1.74c sidecar 1.24cwrk hottest 0.93c mean 0.87c------------------------------mean request 91.11 ms before 68.19 envoy 8.85 sidecar 1.50 worker 12.56 resume 1.44 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4037 /s (100.0% ok)p50 6.3 ms p95 24.4 msin-flight 136 pool 1948------------------------------envoy 1.84c sidecar 1.15cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.59 ms before -72.20 envoy 48.74 sidecar 0.85 worker 32.20 resume 0.81 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 6.1 ms p95 23.7 msin-flight 256 pool 1948------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.93c mean 0.91c------------------------------mean request 8.65 ms before 3.39 envoy 1.63 sidecar 0.86 worker 2.77 resume 0.82 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4855 /sachieved 3966 /s (100.0% ok)p50 364.9 ms p95 1593.9 msin-flight 12k pool 12k------------------------------envoy 1.89c sidecar 1.29cwrk hottest 0.95c mean 0.94c------------------------------mean request 544.40 ms before 171.29 envoy 234.63 sidecar 3.27 worker 135.21 resume 3.07 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 917 /sachieved 916 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 31.0 pool 31.0------------------------------envoy 0.51c sidecar 0.17cwrk hottest 0.36c mean 0.27c------------------------------mean request 3.28 ms before 1.07 envoy 0.53 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 3.9 msin-flight 10.0 pool 31.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.36c mean 0.28c------------------------------mean request 3.12 ms before 1.05 envoy 0.47 sidecar 0.80 worker 0.80 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 31.0------------------------------envoy 0.55c sidecar 0.49cwrk hottest 0.42c mean 0.27c------------------------------mean request 3.08 ms before 1.02 envoy 0.60 sidecar 0.81 worker 0.66 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1782 /sachieved 1782 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 119 pool 120------------------------------envoy 0.93c sidecar 0.72cwrk hottest 0.68c mean 0.49c------------------------------mean request 3.45 ms before 1.23 envoy 0.58 sidecar 0.86 worker 0.78 resume 0.82 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 349 pool 349------------------------------envoy 1.07c sidecar 0.87cwrk hottest 0.71c mean 0.54c------------------------------mean request 4.93 ms before 1.91 envoy 0.81 sidecar 0.89 worker 1.33 resume 0.84 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 22.0 pool 349------------------------------envoy 1.01c sidecar 0.88cwrk hottest 0.67c mean 0.51c------------------------------mean request 3.18 ms before 1.11 envoy 0.54 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2991 /sachieved 2991 /s (100.0% ok)p50 3.7 ms p95 178.7 msin-flight 1057 pool 1057------------------------------envoy 1.48c sidecar 0.93cwrk hottest 0.81c mean 0.72c------------------------------mean request 22.65 ms before 10.53 envoy 4.25 sidecar 1.05 worker 6.82 resume 1.00 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.4 ms p95 4.7 msin-flight 42.0 pool 1057------------------------------envoy 1.47c sidecar 1.14cwrk hottest 0.74c mean 0.73c------------------------------mean request 3.54 ms before 1.31 envoy 0.47 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 4.9 msin-flight 42.0 pool 1057------------------------------envoy 1.49c sidecar 1.17cwrk hottest 0.80c mean 0.74c------------------------------mean request 3.60 ms before 1.33 envoy 0.50 sidecar 0.81 worker 0.97 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3920 /sachieved 3896 /s (100.0% ok)p50 7.1 ms p95 394.5 msin-flight 1948 pool 1948------------------------------envoy 1.74c sidecar 1.24cwrk hottest 0.93c mean 0.87c------------------------------mean request 91.11 ms before 68.19 envoy 8.85 sidecar 1.50 worker 12.56 resume 1.44 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4037 /s (100.0% ok)p50 6.3 ms p95 24.4 msin-flight 136 pool 1948------------------------------envoy 1.84c sidecar 1.15cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.59 ms before -72.20 envoy 48.74 sidecar 0.85 worker 32.20 resume 0.81 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 6.1 ms p95 23.7 msin-flight 256 pool 1948------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.93c mean 0.91c------------------------------mean request 8.65 ms before 3.39 envoy 1.63 sidecar 0.86 worker 2.77 resume 0.82 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4855 /sachieved 3966 /s (100.0% ok)p50 364.9 ms p95 1593.9 msin-flight 12k pool 12k------------------------------envoy 1.89c sidecar 1.29cwrk hottest 0.95c mean 0.94c------------------------------mean request 544.40 ms before 171.29 envoy 234.63 sidecar 3.27 worker 135.21 resume 3.07 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 917 /sachieved 916 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 31.0 pool 31.0------------------------------envoy 0.51c sidecar 0.17cwrk hottest 0.36c mean 0.27c------------------------------mean request 3.28 ms before 1.07 envoy 0.53 sidecar 0.81 worker 0.88 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 3.9 msin-flight 10.0 pool 31.0------------------------------envoy 0.56c sidecar 0.51cwrk hottest 0.36c mean 0.28c------------------------------mean request 3.12 ms before 1.05 envoy 0.47 sidecar 0.80 worker 0.80 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 31.0------------------------------envoy 0.55c sidecar 0.49cwrk hottest 0.42c mean 0.27c------------------------------mean request 3.08 ms before 1.02 envoy 0.60 sidecar 0.81 worker 0.66 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1782 /sachieved 1782 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 119 pool 120------------------------------envoy 0.93c sidecar 0.72cwrk hottest 0.68c mean 0.49c------------------------------mean request 3.45 ms before 1.23 envoy 0.58 sidecar 0.86 worker 0.78 resume 0.82 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 349 pool 349------------------------------envoy 1.07c sidecar 0.87cwrk hottest 0.71c mean 0.54c------------------------------mean request 4.93 ms before 1.91 envoy 0.81 sidecar 0.89 worker 1.33 resume 0.84 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 22.0 pool 349------------------------------envoy 1.01c sidecar 0.88cwrk hottest 0.67c mean 0.51c------------------------------mean request 3.18 ms before 1.11 envoy 0.54 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2991 /sachieved 2991 /s (100.0% ok)p50 3.7 ms p95 178.7 msin-flight 1057 pool 1057------------------------------envoy 1.48c sidecar 0.93cwrk hottest 0.81c mean 0.72c------------------------------mean request 22.65 ms before 10.53 envoy 4.25 sidecar 1.05 worker 6.82 resume 1.00 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.4 ms p95 4.7 msin-flight 42.0 pool 1057------------------------------envoy 1.47c sidecar 1.14cwrk hottest 0.74c mean 0.73c------------------------------mean request 3.54 ms before 1.31 envoy 0.47 sidecar 0.81 worker 0.95 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 4.9 msin-flight 42.0 pool 1057------------------------------envoy 1.49c sidecar 1.17cwrk hottest 0.80c mean 0.74c------------------------------mean request 3.60 ms before 1.33 envoy 0.50 sidecar 0.81 worker 0.97 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3920 /sachieved 3896 /s (100.0% ok)p50 7.1 ms p95 394.5 msin-flight 1948 pool 1948------------------------------envoy 1.74c sidecar 1.24cwrk hottest 0.93c mean 0.87c------------------------------mean request 91.11 ms before 68.19 envoy 8.85 sidecar 1.50 worker 12.56 resume 1.44 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4037 /s (100.0% ok)p50 6.3 ms p95 24.4 msin-flight 136 pool 1948------------------------------envoy 1.84c sidecar 1.15cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.59 ms before -72.20 envoy 48.74 sidecar 0.85 worker 32.20 resume 0.81 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 6.1 ms p95 23.7 msin-flight 256 pool 1948------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.93c mean 0.91c------------------------------mean request 8.65 ms before 3.39 envoy 1.63 sidecar 0.86 worker 2.77 resume 0.82 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4855 /sachieved 3966 /s (100.0% ok)p50 364.9 ms p95 1593.9 msin-flight 12k pool 12k------------------------------envoy 1.89c sidecar 1.29cwrk hottest 0.95c mean 0.94c------------------------------mean request 544.40 ms before 171.29 envoy 234.63 sidecar 3.27 worker 135.21 resume 3.07 (control plane) +
+ + + +atenet-router capacity — 4c +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 + +0 + +10k + +20k + +30k + +40k +requests / s + + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + +0 + +1.0 + +2.0 + +3.0 + +4.0 +cores + + + + + + + + +envoy CPU limit, 4 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + +0 + +1.0G + +2.0G +bytes + +50.0 + +100 + +150 + +200 + +250 + +300 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 100 /sachieved 100.0 /s (100.0% ok)p50 3.3 ms p95 13.0 msin-flight 28.0 pool 28.0------------------------------envoy 0.08c sidecar 0.00cwrk hottest 0.35c mean 0.12c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 7.0 pool 28.0------------------------------envoy 0.57c sidecar 0.26cwrk hottest 0.41c mean 0.14c------------------------------mean request 3.14 ms before 0.94 envoy 0.52 sidecar 0.80 worker 0.88 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 15.0 pool 28.0------------------------------envoy 0.54c sidecar 0.51cwrk hottest 0.41c mean 0.13c------------------------------mean request 3.07 ms before 1.03 envoy 0.61 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 28.0------------------------------envoy 0.58c sidecar 0.50cwrk hottest 0.37c mean 0.14c------------------------------mean request 3.10 ms before 1.03 envoy 0.63 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1895 /sachieved 1895 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 13.0 pool 28.0------------------------------envoy 1.02c sidecar 0.00cwrk hottest 0.58c mean 0.26c------------------------------mean request 3.10 ms before 1.04 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 11.0 pool 28.0------------------------------envoy 1.09c sidecar 0.88cwrk hottest 0.56c mean 0.27c------------------------------mean request 3.06 ms before 1.02 envoy 0.64 sidecar 0.79 worker 0.61 resume 0.74 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2038 /sachieved 2037 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 15.0 pool 28.0------------------------------envoy 1.10c sidecar 0.89cwrk hottest 0.61c mean 0.27c------------------------------mean request 3.10 ms before 1.05 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 17.0 pool 28.0------------------------------envoy 1.60c sidecar 0.96cwrk hottest 0.73c mean 0.39c------------------------------mean request 3.22 ms before 1.11 envoy 0.60 sidecar 0.80 worker 0.71 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 25.0 pool 28.0------------------------------envoy 1.64c sidecar 1.24cwrk hottest 0.76c mean 0.41c------------------------------mean request 3.29 ms before 1.12 envoy 0.61 sidecar 0.81 worker 0.75 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 20.0 pool 24.0------------------------------envoy 1.66c sidecar 1.24cwrk hottest 0.63c mean 0.41c------------------------------mean request 3.23 ms before 1.09 envoy 0.59 sidecar 0.81 worker 0.73 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3559 /sachieved 3558 /s (100.0% ok)p50 3.2 ms p95 4.7 msin-flight 335 pool 335------------------------------envoy 1.96c sidecar 1.34cwrk hottest 0.74c mean 0.50c------------------------------mean request 3.75 ms before 1.25 envoy 0.68 sidecar 0.92 worker 0.89 resume 0.88 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 32.0 pool 335------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.77c mean 0.56c------------------------------mean request 3.53 ms before 1.25 envoy 0.60 sidecar 0.82 worker 0.86 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 29.0 pool 335------------------------------envoy 2.14c sidecar 1.55cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.63 ms before 1.27 envoy 0.65 sidecar 0.81 worker 0.90 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4156 /sachieved 4156 /s (100.0% ok)p50 3.6 ms p95 5.8 msin-flight 41.0 pool 335------------------------------envoy 2.25c sidecar 0.00cwrk hottest 0.82c mean 0.58c------------------------------mean request 3.81 ms before 1.39 envoy 0.63 sidecar 0.83 worker 0.96 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.7 ms p95 5.6 msin-flight 47.0 pool 335------------------------------envoy 2.72c sidecar 1.70cwrk hottest 0.81c mean 0.68c------------------------------mean request 3.89 ms before 1.35 envoy 0.67 sidecar 0.83 worker 1.04 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 7.6 ms p95 789.0 msin-flight 4625 pool 4625------------------------------envoy 2.61c sidecar 1.54cwrk hottest 0.83c mean 0.65c------------------------------mean request 156.09 ms before 39.24 envoy 71.66 sidecar 2.12 worker 43.07 resume 1.47 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5109 /sachieved 5092 /s (100.0% ok)p50 6.5 ms p95 35.5 msin-flight 319 pool 4625------------------------------envoy 2.53c sidecar 0.00cwrk hottest 0.89c mean 0.63c------------------------------mean request 10.81 ms before 3.77 envoy 2.26 sidecar 0.91 worker 3.86 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5992 /s (100.0% ok)p50 27.8 ms p95 100.7 msin-flight 423 pool 4625------------------------------envoy 2.82c sidecar 1.71cwrk hottest 0.88c mean 0.70c------------------------------mean request 37.32 ms before 16.43 envoy 7.15 sidecar 1.11 worker 12.63 resume 1.06 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6012 /s (100.0% ok)p50 28.2 ms p95 108.8 msin-flight 519 pool 4625------------------------------envoy 2.81c sidecar 1.64cwrk hottest 0.87c mean 0.70c------------------------------mean request 38.75 ms before 14.06 envoy 8.58 sidecar 1.08 worker 15.03 resume 1.04 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5977 /s (100.0% ok)p50 33.1 ms p95 153.2 msin-flight 575 pool 4625------------------------------envoy 2.88c sidecar 1.65cwrk hottest 0.87c mean 0.72c------------------------------mean request 49.23 ms before 22.51 envoy 9.30 sidecar 1.16 worker 16.26 resume 1.12 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6759 /sachieved 6217 /s (100.0% ok)p50 333.5 ms p95 1462.8 msin-flight 12k pool 12k------------------------------envoy 3.17c sidecar 1.54cwrk hottest 0.88c mean 0.79c------------------------------mean request 443.46 ms before 119.32 envoy 199.82 sidecar 2.54 worker 121.78 resume 2.44 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5467 /s (100.0% ok)p50 2864.3 ms p95 5747.1 msin-flight 39k pool 39k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3052.89 ms before 980.40 envoy 1244.18 sidecar 4.44 worker 823.86 resume 4.19 (control plane) +rung -1 0 qps setpoint------------------------------offered 100 /sachieved 100.0 /s (100.0% ok)p50 3.3 ms p95 13.0 msin-flight 28.0 pool 28.0------------------------------envoy 0.08c sidecar 0.00cwrk hottest 0.35c mean 0.12c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 7.0 pool 28.0------------------------------envoy 0.57c sidecar 0.26cwrk hottest 0.41c mean 0.14c------------------------------mean request 3.14 ms before 0.94 envoy 0.52 sidecar 0.80 worker 0.88 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 15.0 pool 28.0------------------------------envoy 0.54c sidecar 0.51cwrk hottest 0.41c mean 0.13c------------------------------mean request 3.07 ms before 1.03 envoy 0.61 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 28.0------------------------------envoy 0.58c sidecar 0.50cwrk hottest 0.37c mean 0.14c------------------------------mean request 3.10 ms before 1.03 envoy 0.63 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1895 /sachieved 1895 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 13.0 pool 28.0------------------------------envoy 1.02c sidecar 0.00cwrk hottest 0.58c mean 0.26c------------------------------mean request 3.10 ms before 1.04 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 11.0 pool 28.0------------------------------envoy 1.09c sidecar 0.88cwrk hottest 0.56c mean 0.27c------------------------------mean request 3.06 ms before 1.02 envoy 0.64 sidecar 0.79 worker 0.61 resume 0.74 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2038 /sachieved 2037 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 15.0 pool 28.0------------------------------envoy 1.10c sidecar 0.89cwrk hottest 0.61c mean 0.27c------------------------------mean request 3.10 ms before 1.05 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 17.0 pool 28.0------------------------------envoy 1.60c sidecar 0.96cwrk hottest 0.73c mean 0.39c------------------------------mean request 3.22 ms before 1.11 envoy 0.60 sidecar 0.80 worker 0.71 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 25.0 pool 28.0------------------------------envoy 1.64c sidecar 1.24cwrk hottest 0.76c mean 0.41c------------------------------mean request 3.29 ms before 1.12 envoy 0.61 sidecar 0.81 worker 0.75 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 20.0 pool 24.0------------------------------envoy 1.66c sidecar 1.24cwrk hottest 0.63c mean 0.41c------------------------------mean request 3.23 ms before 1.09 envoy 0.59 sidecar 0.81 worker 0.73 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3559 /sachieved 3558 /s (100.0% ok)p50 3.2 ms p95 4.7 msin-flight 335 pool 335------------------------------envoy 1.96c sidecar 1.34cwrk hottest 0.74c mean 0.50c------------------------------mean request 3.75 ms before 1.25 envoy 0.68 sidecar 0.92 worker 0.89 resume 0.88 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 32.0 pool 335------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.77c mean 0.56c------------------------------mean request 3.53 ms before 1.25 envoy 0.60 sidecar 0.82 worker 0.86 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 29.0 pool 335------------------------------envoy 2.14c sidecar 1.55cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.63 ms before 1.27 envoy 0.65 sidecar 0.81 worker 0.90 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4156 /sachieved 4156 /s (100.0% ok)p50 3.6 ms p95 5.8 msin-flight 41.0 pool 335------------------------------envoy 2.25c sidecar 0.00cwrk hottest 0.82c mean 0.58c------------------------------mean request 3.81 ms before 1.39 envoy 0.63 sidecar 0.83 worker 0.96 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.7 ms p95 5.6 msin-flight 47.0 pool 335------------------------------envoy 2.72c sidecar 1.70cwrk hottest 0.81c mean 0.68c------------------------------mean request 3.89 ms before 1.35 envoy 0.67 sidecar 0.83 worker 1.04 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 7.6 ms p95 789.0 msin-flight 4625 pool 4625------------------------------envoy 2.61c sidecar 1.54cwrk hottest 0.83c mean 0.65c------------------------------mean request 156.09 ms before 39.24 envoy 71.66 sidecar 2.12 worker 43.07 resume 1.47 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5109 /sachieved 5092 /s (100.0% ok)p50 6.5 ms p95 35.5 msin-flight 319 pool 4625------------------------------envoy 2.53c sidecar 0.00cwrk hottest 0.89c mean 0.63c------------------------------mean request 10.81 ms before 3.77 envoy 2.26 sidecar 0.91 worker 3.86 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5992 /s (100.0% ok)p50 27.8 ms p95 100.7 msin-flight 423 pool 4625------------------------------envoy 2.82c sidecar 1.71cwrk hottest 0.88c mean 0.70c------------------------------mean request 37.32 ms before 16.43 envoy 7.15 sidecar 1.11 worker 12.63 resume 1.06 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6012 /s (100.0% ok)p50 28.2 ms p95 108.8 msin-flight 519 pool 4625------------------------------envoy 2.81c sidecar 1.64cwrk hottest 0.87c mean 0.70c------------------------------mean request 38.75 ms before 14.06 envoy 8.58 sidecar 1.08 worker 15.03 resume 1.04 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5977 /s (100.0% ok)p50 33.1 ms p95 153.2 msin-flight 575 pool 4625------------------------------envoy 2.88c sidecar 1.65cwrk hottest 0.87c mean 0.72c------------------------------mean request 49.23 ms before 22.51 envoy 9.30 sidecar 1.16 worker 16.26 resume 1.12 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6759 /sachieved 6217 /s (100.0% ok)p50 333.5 ms p95 1462.8 msin-flight 12k pool 12k------------------------------envoy 3.17c sidecar 1.54cwrk hottest 0.88c mean 0.79c------------------------------mean request 443.46 ms before 119.32 envoy 199.82 sidecar 2.54 worker 121.78 resume 2.44 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5467 /s (100.0% ok)p50 2864.3 ms p95 5747.1 msin-flight 39k pool 39k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3052.89 ms before 980.40 envoy 1244.18 sidecar 4.44 worker 823.86 resume 4.19 (control plane) +rung -1 0 qps setpoint------------------------------offered 100 /sachieved 100.0 /s (100.0% ok)p50 3.3 ms p95 13.0 msin-flight 28.0 pool 28.0------------------------------envoy 0.08c sidecar 0.00cwrk hottest 0.35c mean 0.12c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 7.0 pool 28.0------------------------------envoy 0.57c sidecar 0.26cwrk hottest 0.41c mean 0.14c------------------------------mean request 3.14 ms before 0.94 envoy 0.52 sidecar 0.80 worker 0.88 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 15.0 pool 28.0------------------------------envoy 0.54c sidecar 0.51cwrk hottest 0.41c mean 0.13c------------------------------mean request 3.07 ms before 1.03 envoy 0.61 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 28.0------------------------------envoy 0.58c sidecar 0.50cwrk hottest 0.37c mean 0.14c------------------------------mean request 3.10 ms before 1.03 envoy 0.63 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1895 /sachieved 1895 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 13.0 pool 28.0------------------------------envoy 1.02c sidecar 0.00cwrk hottest 0.58c mean 0.26c------------------------------mean request 3.10 ms before 1.04 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 11.0 pool 28.0------------------------------envoy 1.09c sidecar 0.88cwrk hottest 0.56c mean 0.27c------------------------------mean request 3.06 ms before 1.02 envoy 0.64 sidecar 0.79 worker 0.61 resume 0.74 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2038 /sachieved 2037 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 15.0 pool 28.0------------------------------envoy 1.10c sidecar 0.89cwrk hottest 0.61c mean 0.27c------------------------------mean request 3.10 ms before 1.05 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 17.0 pool 28.0------------------------------envoy 1.60c sidecar 0.96cwrk hottest 0.73c mean 0.39c------------------------------mean request 3.22 ms before 1.11 envoy 0.60 sidecar 0.80 worker 0.71 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 25.0 pool 28.0------------------------------envoy 1.64c sidecar 1.24cwrk hottest 0.76c mean 0.41c------------------------------mean request 3.29 ms before 1.12 envoy 0.61 sidecar 0.81 worker 0.75 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 20.0 pool 24.0------------------------------envoy 1.66c sidecar 1.24cwrk hottest 0.63c mean 0.41c------------------------------mean request 3.23 ms before 1.09 envoy 0.59 sidecar 0.81 worker 0.73 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3559 /sachieved 3558 /s (100.0% ok)p50 3.2 ms p95 4.7 msin-flight 335 pool 335------------------------------envoy 1.96c sidecar 1.34cwrk hottest 0.74c mean 0.50c------------------------------mean request 3.75 ms before 1.25 envoy 0.68 sidecar 0.92 worker 0.89 resume 0.88 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 32.0 pool 335------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.77c mean 0.56c------------------------------mean request 3.53 ms before 1.25 envoy 0.60 sidecar 0.82 worker 0.86 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 29.0 pool 335------------------------------envoy 2.14c sidecar 1.55cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.63 ms before 1.27 envoy 0.65 sidecar 0.81 worker 0.90 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4156 /sachieved 4156 /s (100.0% ok)p50 3.6 ms p95 5.8 msin-flight 41.0 pool 335------------------------------envoy 2.25c sidecar 0.00cwrk hottest 0.82c mean 0.58c------------------------------mean request 3.81 ms before 1.39 envoy 0.63 sidecar 0.83 worker 0.96 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.7 ms p95 5.6 msin-flight 47.0 pool 335------------------------------envoy 2.72c sidecar 1.70cwrk hottest 0.81c mean 0.68c------------------------------mean request 3.89 ms before 1.35 envoy 0.67 sidecar 0.83 worker 1.04 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 7.6 ms p95 789.0 msin-flight 4625 pool 4625------------------------------envoy 2.61c sidecar 1.54cwrk hottest 0.83c mean 0.65c------------------------------mean request 156.09 ms before 39.24 envoy 71.66 sidecar 2.12 worker 43.07 resume 1.47 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5109 /sachieved 5092 /s (100.0% ok)p50 6.5 ms p95 35.5 msin-flight 319 pool 4625------------------------------envoy 2.53c sidecar 0.00cwrk hottest 0.89c mean 0.63c------------------------------mean request 10.81 ms before 3.77 envoy 2.26 sidecar 0.91 worker 3.86 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5992 /s (100.0% ok)p50 27.8 ms p95 100.7 msin-flight 423 pool 4625------------------------------envoy 2.82c sidecar 1.71cwrk hottest 0.88c mean 0.70c------------------------------mean request 37.32 ms before 16.43 envoy 7.15 sidecar 1.11 worker 12.63 resume 1.06 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6012 /s (100.0% ok)p50 28.2 ms p95 108.8 msin-flight 519 pool 4625------------------------------envoy 2.81c sidecar 1.64cwrk hottest 0.87c mean 0.70c------------------------------mean request 38.75 ms before 14.06 envoy 8.58 sidecar 1.08 worker 15.03 resume 1.04 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5977 /s (100.0% ok)p50 33.1 ms p95 153.2 msin-flight 575 pool 4625------------------------------envoy 2.88c sidecar 1.65cwrk hottest 0.87c mean 0.72c------------------------------mean request 49.23 ms before 22.51 envoy 9.30 sidecar 1.16 worker 16.26 resume 1.12 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6759 /sachieved 6217 /s (100.0% ok)p50 333.5 ms p95 1462.8 msin-flight 12k pool 12k------------------------------envoy 3.17c sidecar 1.54cwrk hottest 0.88c mean 0.79c------------------------------mean request 443.46 ms before 119.32 envoy 199.82 sidecar 2.54 worker 121.78 resume 2.44 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5467 /s (100.0% ok)p50 2864.3 ms p95 5747.1 msin-flight 39k pool 39k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3052.89 ms before 980.40 envoy 1244.18 sidecar 4.44 worker 823.86 resume 4.19 (control plane) +rung -1 0 qps setpoint------------------------------offered 100 /sachieved 100.0 /s (100.0% ok)p50 3.3 ms p95 13.0 msin-flight 28.0 pool 28.0------------------------------envoy 0.08c sidecar 0.00cwrk hottest 0.35c mean 0.12c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 7.0 pool 28.0------------------------------envoy 0.57c sidecar 0.26cwrk hottest 0.41c mean 0.14c------------------------------mean request 3.14 ms before 0.94 envoy 0.52 sidecar 0.80 worker 0.88 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 15.0 pool 28.0------------------------------envoy 0.54c sidecar 0.51cwrk hottest 0.41c mean 0.13c------------------------------mean request 3.07 ms before 1.03 envoy 0.61 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 28.0------------------------------envoy 0.58c sidecar 0.50cwrk hottest 0.37c mean 0.14c------------------------------mean request 3.10 ms before 1.03 envoy 0.63 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1895 /sachieved 1895 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 13.0 pool 28.0------------------------------envoy 1.02c sidecar 0.00cwrk hottest 0.58c mean 0.26c------------------------------mean request 3.10 ms before 1.04 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 11.0 pool 28.0------------------------------envoy 1.09c sidecar 0.88cwrk hottest 0.56c mean 0.27c------------------------------mean request 3.06 ms before 1.02 envoy 0.64 sidecar 0.79 worker 0.61 resume 0.74 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2038 /sachieved 2037 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 15.0 pool 28.0------------------------------envoy 1.10c sidecar 0.89cwrk hottest 0.61c mean 0.27c------------------------------mean request 3.10 ms before 1.05 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 17.0 pool 28.0------------------------------envoy 1.60c sidecar 0.96cwrk hottest 0.73c mean 0.39c------------------------------mean request 3.22 ms before 1.11 envoy 0.60 sidecar 0.80 worker 0.71 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 25.0 pool 28.0------------------------------envoy 1.64c sidecar 1.24cwrk hottest 0.76c mean 0.41c------------------------------mean request 3.29 ms before 1.12 envoy 0.61 sidecar 0.81 worker 0.75 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 20.0 pool 24.0------------------------------envoy 1.66c sidecar 1.24cwrk hottest 0.63c mean 0.41c------------------------------mean request 3.23 ms before 1.09 envoy 0.59 sidecar 0.81 worker 0.73 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3559 /sachieved 3558 /s (100.0% ok)p50 3.2 ms p95 4.7 msin-flight 335 pool 335------------------------------envoy 1.96c sidecar 1.34cwrk hottest 0.74c mean 0.50c------------------------------mean request 3.75 ms before 1.25 envoy 0.68 sidecar 0.92 worker 0.89 resume 0.88 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 32.0 pool 335------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.77c mean 0.56c------------------------------mean request 3.53 ms before 1.25 envoy 0.60 sidecar 0.82 worker 0.86 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 29.0 pool 335------------------------------envoy 2.14c sidecar 1.55cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.63 ms before 1.27 envoy 0.65 sidecar 0.81 worker 0.90 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4156 /sachieved 4156 /s (100.0% ok)p50 3.6 ms p95 5.8 msin-flight 41.0 pool 335------------------------------envoy 2.25c sidecar 0.00cwrk hottest 0.82c mean 0.58c------------------------------mean request 3.81 ms before 1.39 envoy 0.63 sidecar 0.83 worker 0.96 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.7 ms p95 5.6 msin-flight 47.0 pool 335------------------------------envoy 2.72c sidecar 1.70cwrk hottest 0.81c mean 0.68c------------------------------mean request 3.89 ms before 1.35 envoy 0.67 sidecar 0.83 worker 1.04 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 7.6 ms p95 789.0 msin-flight 4625 pool 4625------------------------------envoy 2.61c sidecar 1.54cwrk hottest 0.83c mean 0.65c------------------------------mean request 156.09 ms before 39.24 envoy 71.66 sidecar 2.12 worker 43.07 resume 1.47 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5109 /sachieved 5092 /s (100.0% ok)p50 6.5 ms p95 35.5 msin-flight 319 pool 4625------------------------------envoy 2.53c sidecar 0.00cwrk hottest 0.89c mean 0.63c------------------------------mean request 10.81 ms before 3.77 envoy 2.26 sidecar 0.91 worker 3.86 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5992 /s (100.0% ok)p50 27.8 ms p95 100.7 msin-flight 423 pool 4625------------------------------envoy 2.82c sidecar 1.71cwrk hottest 0.88c mean 0.70c------------------------------mean request 37.32 ms before 16.43 envoy 7.15 sidecar 1.11 worker 12.63 resume 1.06 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6012 /s (100.0% ok)p50 28.2 ms p95 108.8 msin-flight 519 pool 4625------------------------------envoy 2.81c sidecar 1.64cwrk hottest 0.87c mean 0.70c------------------------------mean request 38.75 ms before 14.06 envoy 8.58 sidecar 1.08 worker 15.03 resume 1.04 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5977 /s (100.0% ok)p50 33.1 ms p95 153.2 msin-flight 575 pool 4625------------------------------envoy 2.88c sidecar 1.65cwrk hottest 0.87c mean 0.72c------------------------------mean request 49.23 ms before 22.51 envoy 9.30 sidecar 1.16 worker 16.26 resume 1.12 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6759 /sachieved 6217 /s (100.0% ok)p50 333.5 ms p95 1462.8 msin-flight 12k pool 12k------------------------------envoy 3.17c sidecar 1.54cwrk hottest 0.88c mean 0.79c------------------------------mean request 443.46 ms before 119.32 envoy 199.82 sidecar 2.54 worker 121.78 resume 2.44 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5467 /s (100.0% ok)p50 2864.3 ms p95 5747.1 msin-flight 39k pool 39k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3052.89 ms before 980.40 envoy 1244.18 sidecar 4.44 worker 823.86 resume 4.19 (control plane) +rung -1 0 qps setpoint------------------------------offered 100 /sachieved 100.0 /s (100.0% ok)p50 3.3 ms p95 13.0 msin-flight 28.0 pool 28.0------------------------------envoy 0.08c sidecar 0.00cwrk hottest 0.35c mean 0.12c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 7.0 pool 28.0------------------------------envoy 0.57c sidecar 0.26cwrk hottest 0.41c mean 0.14c------------------------------mean request 3.14 ms before 0.94 envoy 0.52 sidecar 0.80 worker 0.88 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 15.0 pool 28.0------------------------------envoy 0.54c sidecar 0.51cwrk hottest 0.41c mean 0.13c------------------------------mean request 3.07 ms before 1.03 envoy 0.61 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 28.0------------------------------envoy 0.58c sidecar 0.50cwrk hottest 0.37c mean 0.14c------------------------------mean request 3.10 ms before 1.03 envoy 0.63 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1895 /sachieved 1895 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 13.0 pool 28.0------------------------------envoy 1.02c sidecar 0.00cwrk hottest 0.58c mean 0.26c------------------------------mean request 3.10 ms before 1.04 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 11.0 pool 28.0------------------------------envoy 1.09c sidecar 0.88cwrk hottest 0.56c mean 0.27c------------------------------mean request 3.06 ms before 1.02 envoy 0.64 sidecar 0.79 worker 0.61 resume 0.74 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2038 /sachieved 2037 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 15.0 pool 28.0------------------------------envoy 1.10c sidecar 0.89cwrk hottest 0.61c mean 0.27c------------------------------mean request 3.10 ms before 1.05 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 17.0 pool 28.0------------------------------envoy 1.60c sidecar 0.96cwrk hottest 0.73c mean 0.39c------------------------------mean request 3.22 ms before 1.11 envoy 0.60 sidecar 0.80 worker 0.71 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 25.0 pool 28.0------------------------------envoy 1.64c sidecar 1.24cwrk hottest 0.76c mean 0.41c------------------------------mean request 3.29 ms before 1.12 envoy 0.61 sidecar 0.81 worker 0.75 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 20.0 pool 24.0------------------------------envoy 1.66c sidecar 1.24cwrk hottest 0.63c mean 0.41c------------------------------mean request 3.23 ms before 1.09 envoy 0.59 sidecar 0.81 worker 0.73 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3559 /sachieved 3558 /s (100.0% ok)p50 3.2 ms p95 4.7 msin-flight 335 pool 335------------------------------envoy 1.96c sidecar 1.34cwrk hottest 0.74c mean 0.50c------------------------------mean request 3.75 ms before 1.25 envoy 0.68 sidecar 0.92 worker 0.89 resume 0.88 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 32.0 pool 335------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.77c mean 0.56c------------------------------mean request 3.53 ms before 1.25 envoy 0.60 sidecar 0.82 worker 0.86 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 29.0 pool 335------------------------------envoy 2.14c sidecar 1.55cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.63 ms before 1.27 envoy 0.65 sidecar 0.81 worker 0.90 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4156 /sachieved 4156 /s (100.0% ok)p50 3.6 ms p95 5.8 msin-flight 41.0 pool 335------------------------------envoy 2.25c sidecar 0.00cwrk hottest 0.82c mean 0.58c------------------------------mean request 3.81 ms before 1.39 envoy 0.63 sidecar 0.83 worker 0.96 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.7 ms p95 5.6 msin-flight 47.0 pool 335------------------------------envoy 2.72c sidecar 1.70cwrk hottest 0.81c mean 0.68c------------------------------mean request 3.89 ms before 1.35 envoy 0.67 sidecar 0.83 worker 1.04 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 7.6 ms p95 789.0 msin-flight 4625 pool 4625------------------------------envoy 2.61c sidecar 1.54cwrk hottest 0.83c mean 0.65c------------------------------mean request 156.09 ms before 39.24 envoy 71.66 sidecar 2.12 worker 43.07 resume 1.47 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5109 /sachieved 5092 /s (100.0% ok)p50 6.5 ms p95 35.5 msin-flight 319 pool 4625------------------------------envoy 2.53c sidecar 0.00cwrk hottest 0.89c mean 0.63c------------------------------mean request 10.81 ms before 3.77 envoy 2.26 sidecar 0.91 worker 3.86 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5992 /s (100.0% ok)p50 27.8 ms p95 100.7 msin-flight 423 pool 4625------------------------------envoy 2.82c sidecar 1.71cwrk hottest 0.88c mean 0.70c------------------------------mean request 37.32 ms before 16.43 envoy 7.15 sidecar 1.11 worker 12.63 resume 1.06 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6012 /s (100.0% ok)p50 28.2 ms p95 108.8 msin-flight 519 pool 4625------------------------------envoy 2.81c sidecar 1.64cwrk hottest 0.87c mean 0.70c------------------------------mean request 38.75 ms before 14.06 envoy 8.58 sidecar 1.08 worker 15.03 resume 1.04 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5977 /s (100.0% ok)p50 33.1 ms p95 153.2 msin-flight 575 pool 4625------------------------------envoy 2.88c sidecar 1.65cwrk hottest 0.87c mean 0.72c------------------------------mean request 49.23 ms before 22.51 envoy 9.30 sidecar 1.16 worker 16.26 resume 1.12 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6759 /sachieved 6217 /s (100.0% ok)p50 333.5 ms p95 1462.8 msin-flight 12k pool 12k------------------------------envoy 3.17c sidecar 1.54cwrk hottest 0.88c mean 0.79c------------------------------mean request 443.46 ms before 119.32 envoy 199.82 sidecar 2.54 worker 121.78 resume 2.44 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5467 /s (100.0% ok)p50 2864.3 ms p95 5747.1 msin-flight 39k pool 39k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3052.89 ms before 980.40 envoy 1244.18 sidecar 4.44 worker 823.86 resume 4.19 (control plane) +rung -1 0 qps setpoint------------------------------offered 100 /sachieved 100.0 /s (100.0% ok)p50 3.3 ms p95 13.0 msin-flight 28.0 pool 28.0------------------------------envoy 0.08c sidecar 0.00cwrk hottest 0.35c mean 0.12c +rung 0 1000 qps setpoint [warmup]------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 7.0 pool 28.0------------------------------envoy 0.57c sidecar 0.26cwrk hottest 0.41c mean 0.14c------------------------------mean request 3.14 ms before 0.94 envoy 0.52 sidecar 0.80 worker 0.88 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 15.0 pool 28.0------------------------------envoy 0.54c sidecar 0.51cwrk hottest 0.41c mean 0.13c------------------------------mean request 3.07 ms before 1.03 envoy 0.61 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 28.0------------------------------envoy 0.58c sidecar 0.50cwrk hottest 0.37c mean 0.14c------------------------------mean request 3.10 ms before 1.03 envoy 0.63 sidecar 0.80 worker 0.64 resume 0.76 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1895 /sachieved 1895 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 13.0 pool 28.0------------------------------envoy 1.02c sidecar 0.00cwrk hottest 0.58c mean 0.26c------------------------------mean request 3.10 ms before 1.04 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 11.0 pool 28.0------------------------------envoy 1.09c sidecar 0.88cwrk hottest 0.56c mean 0.27c------------------------------mean request 3.06 ms before 1.02 envoy 0.64 sidecar 0.79 worker 0.61 resume 0.74 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2038 /sachieved 2037 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 15.0 pool 28.0------------------------------envoy 1.10c sidecar 0.89cwrk hottest 0.61c mean 0.27c------------------------------mean request 3.10 ms before 1.05 envoy 0.60 sidecar 0.79 worker 0.66 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 17.0 pool 28.0------------------------------envoy 1.60c sidecar 0.96cwrk hottest 0.73c mean 0.39c------------------------------mean request 3.22 ms before 1.11 envoy 0.60 sidecar 0.80 worker 0.71 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 25.0 pool 28.0------------------------------envoy 1.64c sidecar 1.24cwrk hottest 0.76c mean 0.41c------------------------------mean request 3.29 ms before 1.12 envoy 0.61 sidecar 0.81 worker 0.75 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.4 msin-flight 20.0 pool 24.0------------------------------envoy 1.66c sidecar 1.24cwrk hottest 0.63c mean 0.41c------------------------------mean request 3.23 ms before 1.09 envoy 0.59 sidecar 0.81 worker 0.73 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3559 /sachieved 3558 /s (100.0% ok)p50 3.2 ms p95 4.7 msin-flight 335 pool 335------------------------------envoy 1.96c sidecar 1.34cwrk hottest 0.74c mean 0.50c------------------------------mean request 3.75 ms before 1.25 envoy 0.68 sidecar 0.92 worker 0.89 resume 0.88 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.0 msin-flight 32.0 pool 335------------------------------envoy 2.24c sidecar 0.00cwrk hottest 0.77c mean 0.56c------------------------------mean request 3.53 ms before 1.25 envoy 0.60 sidecar 0.82 worker 0.86 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 29.0 pool 335------------------------------envoy 2.14c sidecar 1.55cwrk hottest 0.84c mean 0.54c------------------------------mean request 3.63 ms before 1.27 envoy 0.65 sidecar 0.81 worker 0.90 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4156 /sachieved 4156 /s (100.0% ok)p50 3.6 ms p95 5.8 msin-flight 41.0 pool 335------------------------------envoy 2.25c sidecar 0.00cwrk hottest 0.82c mean 0.58c------------------------------mean request 3.81 ms before 1.39 envoy 0.63 sidecar 0.83 worker 0.96 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.7 ms p95 5.6 msin-flight 47.0 pool 335------------------------------envoy 2.72c sidecar 1.70cwrk hottest 0.81c mean 0.68c------------------------------mean request 3.89 ms before 1.35 envoy 0.67 sidecar 0.83 worker 1.04 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 7.6 ms p95 789.0 msin-flight 4625 pool 4625------------------------------envoy 2.61c sidecar 1.54cwrk hottest 0.83c mean 0.65c------------------------------mean request 156.09 ms before 39.24 envoy 71.66 sidecar 2.12 worker 43.07 resume 1.47 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5109 /sachieved 5092 /s (100.0% ok)p50 6.5 ms p95 35.5 msin-flight 319 pool 4625------------------------------envoy 2.53c sidecar 0.00cwrk hottest 0.89c mean 0.63c------------------------------mean request 10.81 ms before 3.77 envoy 2.26 sidecar 0.91 worker 3.86 resume 0.87 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 5992 /s (100.0% ok)p50 27.8 ms p95 100.7 msin-flight 423 pool 4625------------------------------envoy 2.82c sidecar 1.71cwrk hottest 0.88c mean 0.70c------------------------------mean request 37.32 ms before 16.43 envoy 7.15 sidecar 1.11 worker 12.63 resume 1.06 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6012 /s (100.0% ok)p50 28.2 ms p95 108.8 msin-flight 519 pool 4625------------------------------envoy 2.81c sidecar 1.64cwrk hottest 0.87c mean 0.70c------------------------------mean request 38.75 ms before 14.06 envoy 8.58 sidecar 1.08 worker 15.03 resume 1.04 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5977 /s (100.0% ok)p50 33.1 ms p95 153.2 msin-flight 575 pool 4625------------------------------envoy 2.88c sidecar 1.65cwrk hottest 0.87c mean 0.72c------------------------------mean request 49.23 ms before 22.51 envoy 9.30 sidecar 1.16 worker 16.26 resume 1.12 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6759 /sachieved 6217 /s (100.0% ok)p50 333.5 ms p95 1462.8 msin-flight 12k pool 12k------------------------------envoy 3.17c sidecar 1.54cwrk hottest 0.88c mean 0.79c------------------------------mean request 443.46 ms before 119.32 envoy 199.82 sidecar 2.54 worker 121.78 resume 2.44 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5467 /s (100.0% ok)p50 2864.3 ms p95 5747.1 msin-flight 39k pool 39k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3052.89 ms before 980.40 envoy 1244.18 sidecar 4.44 worker 823.86 resume 4.19 (control plane) +
+ + + +atenet-router capacity — 8c/2t +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + +R0 +R1 +R2 +R3 +R4 + +0 + +10k + +20k + +30k +requests / s + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + +0 + +2.0 + +4.0 + +6.0 + +8.0 +cores + + + + + + +envoy CPU limit, 8 cores — the variable under test + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + +0 + +500.0M + +1.0G + +1.5G +bytes + +50.0 + +100 + +150 + +200 +seconds since the arm started + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 375 /sachieved 375 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 39.0 pool 39.0------------------------------envoy 0.21c sidecar 0.15cwrk hottest 0.40c mean 0.27c------------------------------mean request 3.53 ms before 1.18 envoy 0.54 sidecar 0.79 worker 1.04 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.52c sidecar 0.48cwrk hottest 0.40c mean 0.26c------------------------------mean request 3.03 ms before 0.99 envoy 0.58 sidecar 0.78 worker 0.69 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.54c sidecar 0.48cwrk hottest 0.43c mean 0.27c------------------------------mean request 3.05 ms before 1.01 envoy 0.58 sidecar 0.79 worker 0.68 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1587 /sachieved 1586 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 10.0 pool 39.0------------------------------envoy 0.83c sidecar 0.53cwrk hottest 0.54c mean 0.43c------------------------------mean request 3.13 ms before 1.07 envoy 0.54 sidecar 0.80 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 11.0 pool 39.0------------------------------envoy 1.04c sidecar 0.88cwrk hottest 0.58c mean 0.52c------------------------------mean request 3.16 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 12.0 pool 39.0------------------------------envoy 1.03c sidecar 0.00cwrk hottest 0.57c mean 0.51c------------------------------mean request 3.15 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.71 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2201 /sachieved 2201 /s (100.0% ok)p50 3.2 ms p95 4.3 msin-flight 19.0 pool 39.0------------------------------envoy 1.12c sidecar 0.88cwrk hottest 0.75c mean 0.57c------------------------------mean request 3.28 ms before 1.17 envoy 0.53 sidecar 0.81 worker 0.77 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 5.1 msin-flight 24.0 pool 39.0------------------------------envoy 1.50c sidecar 1.17cwrk hottest 0.77c mean 0.75c------------------------------mean request 3.67 ms before 1.34 envoy 0.53 sidecar 0.81 worker 0.99 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 2999 /s (100.0% ok)p50 3.5 ms p95 5.6 msin-flight 410 pool 410------------------------------envoy 1.48c sidecar 1.19cwrk hottest 0.82c mean 0.74c------------------------------mean request 6.10 ms before 3.82 envoy -0.19 sidecar 1.50 worker 0.96 resume 1.46 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3412 /sachieved 3407 /s (100.0% ok)p50 4.7 ms p95 21.1 msin-flight 578 pool 577------------------------------envoy 1.64c sidecar 1.18cwrk hottest 0.92c mean 0.81c------------------------------mean request 8.62 ms before 2.61 envoy 2.34 sidecar 0.86 worker 2.81 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4005 /s (100.0% ok)p50 7.9 ms p95 23.9 msin-flight 208 pool 577------------------------------envoy 1.85c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.96 ms before 3.87 envoy 1.94 sidecar 0.86 worker 3.29 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 7.9 ms p95 30.6 msin-flight 318 pool 577------------------------------envoy 1.81c sidecar 1.32cwrk hottest 0.94c mean 0.91c------------------------------mean request 11.39 ms before 4.80 envoy 2.09 sidecar 0.87 worker 3.63 resume 0.83 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4292 /sachieved 4275 /s (100.0% ok)p50 7.5 ms p95 53.9 msin-flight 300 pool 577------------------------------envoy 1.83c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 20.50 ms before 10.05 envoy 3.36 sidecar 0.98 worker 6.12 resume 0.94 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 3732 /s (100.0% ok)p50 536.6 ms p95 4274.2 msin-flight 26k pool 26k------------------------------envoy 1.90c sidecar 1.07cwrk hottest 0.97c mean 0.95c------------------------------mean request 1311.04 ms before 698.40 envoy 346.99 sidecar 3.63 worker 262.03 resume 3.41 (control plane) +rung -1 0 qps setpoint------------------------------offered 375 /sachieved 375 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 39.0 pool 39.0------------------------------envoy 0.21c sidecar 0.15cwrk hottest 0.40c mean 0.27c------------------------------mean request 3.53 ms before 1.18 envoy 0.54 sidecar 0.79 worker 1.04 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.52c sidecar 0.48cwrk hottest 0.40c mean 0.26c------------------------------mean request 3.03 ms before 0.99 envoy 0.58 sidecar 0.78 worker 0.69 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.54c sidecar 0.48cwrk hottest 0.43c mean 0.27c------------------------------mean request 3.05 ms before 1.01 envoy 0.58 sidecar 0.79 worker 0.68 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1587 /sachieved 1586 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 10.0 pool 39.0------------------------------envoy 0.83c sidecar 0.53cwrk hottest 0.54c mean 0.43c------------------------------mean request 3.13 ms before 1.07 envoy 0.54 sidecar 0.80 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 11.0 pool 39.0------------------------------envoy 1.04c sidecar 0.88cwrk hottest 0.58c mean 0.52c------------------------------mean request 3.16 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 12.0 pool 39.0------------------------------envoy 1.03c sidecar 0.00cwrk hottest 0.57c mean 0.51c------------------------------mean request 3.15 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.71 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2201 /sachieved 2201 /s (100.0% ok)p50 3.2 ms p95 4.3 msin-flight 19.0 pool 39.0------------------------------envoy 1.12c sidecar 0.88cwrk hottest 0.75c mean 0.57c------------------------------mean request 3.28 ms before 1.17 envoy 0.53 sidecar 0.81 worker 0.77 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 5.1 msin-flight 24.0 pool 39.0------------------------------envoy 1.50c sidecar 1.17cwrk hottest 0.77c mean 0.75c------------------------------mean request 3.67 ms before 1.34 envoy 0.53 sidecar 0.81 worker 0.99 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 2999 /s (100.0% ok)p50 3.5 ms p95 5.6 msin-flight 410 pool 410------------------------------envoy 1.48c sidecar 1.19cwrk hottest 0.82c mean 0.74c------------------------------mean request 6.10 ms before 3.82 envoy -0.19 sidecar 1.50 worker 0.96 resume 1.46 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3412 /sachieved 3407 /s (100.0% ok)p50 4.7 ms p95 21.1 msin-flight 578 pool 577------------------------------envoy 1.64c sidecar 1.18cwrk hottest 0.92c mean 0.81c------------------------------mean request 8.62 ms before 2.61 envoy 2.34 sidecar 0.86 worker 2.81 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4005 /s (100.0% ok)p50 7.9 ms p95 23.9 msin-flight 208 pool 577------------------------------envoy 1.85c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.96 ms before 3.87 envoy 1.94 sidecar 0.86 worker 3.29 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 7.9 ms p95 30.6 msin-flight 318 pool 577------------------------------envoy 1.81c sidecar 1.32cwrk hottest 0.94c mean 0.91c------------------------------mean request 11.39 ms before 4.80 envoy 2.09 sidecar 0.87 worker 3.63 resume 0.83 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4292 /sachieved 4275 /s (100.0% ok)p50 7.5 ms p95 53.9 msin-flight 300 pool 577------------------------------envoy 1.83c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 20.50 ms before 10.05 envoy 3.36 sidecar 0.98 worker 6.12 resume 0.94 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 3732 /s (100.0% ok)p50 536.6 ms p95 4274.2 msin-flight 26k pool 26k------------------------------envoy 1.90c sidecar 1.07cwrk hottest 0.97c mean 0.95c------------------------------mean request 1311.04 ms before 698.40 envoy 346.99 sidecar 3.63 worker 262.03 resume 3.41 (control plane) +rung -1 0 qps setpoint------------------------------offered 375 /sachieved 375 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 39.0 pool 39.0------------------------------envoy 0.21c sidecar 0.15cwrk hottest 0.40c mean 0.27c------------------------------mean request 3.53 ms before 1.18 envoy 0.54 sidecar 0.79 worker 1.04 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.52c sidecar 0.48cwrk hottest 0.40c mean 0.26c------------------------------mean request 3.03 ms before 0.99 envoy 0.58 sidecar 0.78 worker 0.69 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.54c sidecar 0.48cwrk hottest 0.43c mean 0.27c------------------------------mean request 3.05 ms before 1.01 envoy 0.58 sidecar 0.79 worker 0.68 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1587 /sachieved 1586 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 10.0 pool 39.0------------------------------envoy 0.83c sidecar 0.53cwrk hottest 0.54c mean 0.43c------------------------------mean request 3.13 ms before 1.07 envoy 0.54 sidecar 0.80 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 11.0 pool 39.0------------------------------envoy 1.04c sidecar 0.88cwrk hottest 0.58c mean 0.52c------------------------------mean request 3.16 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 12.0 pool 39.0------------------------------envoy 1.03c sidecar 0.00cwrk hottest 0.57c mean 0.51c------------------------------mean request 3.15 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.71 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2201 /sachieved 2201 /s (100.0% ok)p50 3.2 ms p95 4.3 msin-flight 19.0 pool 39.0------------------------------envoy 1.12c sidecar 0.88cwrk hottest 0.75c mean 0.57c------------------------------mean request 3.28 ms before 1.17 envoy 0.53 sidecar 0.81 worker 0.77 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 5.1 msin-flight 24.0 pool 39.0------------------------------envoy 1.50c sidecar 1.17cwrk hottest 0.77c mean 0.75c------------------------------mean request 3.67 ms before 1.34 envoy 0.53 sidecar 0.81 worker 0.99 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 2999 /s (100.0% ok)p50 3.5 ms p95 5.6 msin-flight 410 pool 410------------------------------envoy 1.48c sidecar 1.19cwrk hottest 0.82c mean 0.74c------------------------------mean request 6.10 ms before 3.82 envoy -0.19 sidecar 1.50 worker 0.96 resume 1.46 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3412 /sachieved 3407 /s (100.0% ok)p50 4.7 ms p95 21.1 msin-flight 578 pool 577------------------------------envoy 1.64c sidecar 1.18cwrk hottest 0.92c mean 0.81c------------------------------mean request 8.62 ms before 2.61 envoy 2.34 sidecar 0.86 worker 2.81 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4005 /s (100.0% ok)p50 7.9 ms p95 23.9 msin-flight 208 pool 577------------------------------envoy 1.85c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.96 ms before 3.87 envoy 1.94 sidecar 0.86 worker 3.29 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 7.9 ms p95 30.6 msin-flight 318 pool 577------------------------------envoy 1.81c sidecar 1.32cwrk hottest 0.94c mean 0.91c------------------------------mean request 11.39 ms before 4.80 envoy 2.09 sidecar 0.87 worker 3.63 resume 0.83 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4292 /sachieved 4275 /s (100.0% ok)p50 7.5 ms p95 53.9 msin-flight 300 pool 577------------------------------envoy 1.83c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 20.50 ms before 10.05 envoy 3.36 sidecar 0.98 worker 6.12 resume 0.94 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 3732 /s (100.0% ok)p50 536.6 ms p95 4274.2 msin-flight 26k pool 26k------------------------------envoy 1.90c sidecar 1.07cwrk hottest 0.97c mean 0.95c------------------------------mean request 1311.04 ms before 698.40 envoy 346.99 sidecar 3.63 worker 262.03 resume 3.41 (control plane) +rung -1 0 qps setpoint------------------------------offered 375 /sachieved 375 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 39.0 pool 39.0------------------------------envoy 0.21c sidecar 0.15cwrk hottest 0.40c mean 0.27c------------------------------mean request 3.53 ms before 1.18 envoy 0.54 sidecar 0.79 worker 1.04 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.52c sidecar 0.48cwrk hottest 0.40c mean 0.26c------------------------------mean request 3.03 ms before 0.99 envoy 0.58 sidecar 0.78 worker 0.69 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.54c sidecar 0.48cwrk hottest 0.43c mean 0.27c------------------------------mean request 3.05 ms before 1.01 envoy 0.58 sidecar 0.79 worker 0.68 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1587 /sachieved 1586 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 10.0 pool 39.0------------------------------envoy 0.83c sidecar 0.53cwrk hottest 0.54c mean 0.43c------------------------------mean request 3.13 ms before 1.07 envoy 0.54 sidecar 0.80 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 11.0 pool 39.0------------------------------envoy 1.04c sidecar 0.88cwrk hottest 0.58c mean 0.52c------------------------------mean request 3.16 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 12.0 pool 39.0------------------------------envoy 1.03c sidecar 0.00cwrk hottest 0.57c mean 0.51c------------------------------mean request 3.15 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.71 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2201 /sachieved 2201 /s (100.0% ok)p50 3.2 ms p95 4.3 msin-flight 19.0 pool 39.0------------------------------envoy 1.12c sidecar 0.88cwrk hottest 0.75c mean 0.57c------------------------------mean request 3.28 ms before 1.17 envoy 0.53 sidecar 0.81 worker 0.77 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 5.1 msin-flight 24.0 pool 39.0------------------------------envoy 1.50c sidecar 1.17cwrk hottest 0.77c mean 0.75c------------------------------mean request 3.67 ms before 1.34 envoy 0.53 sidecar 0.81 worker 0.99 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 2999 /s (100.0% ok)p50 3.5 ms p95 5.6 msin-flight 410 pool 410------------------------------envoy 1.48c sidecar 1.19cwrk hottest 0.82c mean 0.74c------------------------------mean request 6.10 ms before 3.82 envoy -0.19 sidecar 1.50 worker 0.96 resume 1.46 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3412 /sachieved 3407 /s (100.0% ok)p50 4.7 ms p95 21.1 msin-flight 578 pool 577------------------------------envoy 1.64c sidecar 1.18cwrk hottest 0.92c mean 0.81c------------------------------mean request 8.62 ms before 2.61 envoy 2.34 sidecar 0.86 worker 2.81 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4005 /s (100.0% ok)p50 7.9 ms p95 23.9 msin-flight 208 pool 577------------------------------envoy 1.85c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.96 ms before 3.87 envoy 1.94 sidecar 0.86 worker 3.29 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 7.9 ms p95 30.6 msin-flight 318 pool 577------------------------------envoy 1.81c sidecar 1.32cwrk hottest 0.94c mean 0.91c------------------------------mean request 11.39 ms before 4.80 envoy 2.09 sidecar 0.87 worker 3.63 resume 0.83 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4292 /sachieved 4275 /s (100.0% ok)p50 7.5 ms p95 53.9 msin-flight 300 pool 577------------------------------envoy 1.83c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 20.50 ms before 10.05 envoy 3.36 sidecar 0.98 worker 6.12 resume 0.94 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 3732 /s (100.0% ok)p50 536.6 ms p95 4274.2 msin-flight 26k pool 26k------------------------------envoy 1.90c sidecar 1.07cwrk hottest 0.97c mean 0.95c------------------------------mean request 1311.04 ms before 698.40 envoy 346.99 sidecar 3.63 worker 262.03 resume 3.41 (control plane) +rung -1 0 qps setpoint------------------------------offered 375 /sachieved 375 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 39.0 pool 39.0------------------------------envoy 0.21c sidecar 0.15cwrk hottest 0.40c mean 0.27c------------------------------mean request 3.53 ms before 1.18 envoy 0.54 sidecar 0.79 worker 1.04 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.52c sidecar 0.48cwrk hottest 0.40c mean 0.26c------------------------------mean request 3.03 ms before 0.99 envoy 0.58 sidecar 0.78 worker 0.69 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.54c sidecar 0.48cwrk hottest 0.43c mean 0.27c------------------------------mean request 3.05 ms before 1.01 envoy 0.58 sidecar 0.79 worker 0.68 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1587 /sachieved 1586 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 10.0 pool 39.0------------------------------envoy 0.83c sidecar 0.53cwrk hottest 0.54c mean 0.43c------------------------------mean request 3.13 ms before 1.07 envoy 0.54 sidecar 0.80 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 11.0 pool 39.0------------------------------envoy 1.04c sidecar 0.88cwrk hottest 0.58c mean 0.52c------------------------------mean request 3.16 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 12.0 pool 39.0------------------------------envoy 1.03c sidecar 0.00cwrk hottest 0.57c mean 0.51c------------------------------mean request 3.15 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.71 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2201 /sachieved 2201 /s (100.0% ok)p50 3.2 ms p95 4.3 msin-flight 19.0 pool 39.0------------------------------envoy 1.12c sidecar 0.88cwrk hottest 0.75c mean 0.57c------------------------------mean request 3.28 ms before 1.17 envoy 0.53 sidecar 0.81 worker 0.77 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 5.1 msin-flight 24.0 pool 39.0------------------------------envoy 1.50c sidecar 1.17cwrk hottest 0.77c mean 0.75c------------------------------mean request 3.67 ms before 1.34 envoy 0.53 sidecar 0.81 worker 0.99 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 2999 /s (100.0% ok)p50 3.5 ms p95 5.6 msin-flight 410 pool 410------------------------------envoy 1.48c sidecar 1.19cwrk hottest 0.82c mean 0.74c------------------------------mean request 6.10 ms before 3.82 envoy -0.19 sidecar 1.50 worker 0.96 resume 1.46 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3412 /sachieved 3407 /s (100.0% ok)p50 4.7 ms p95 21.1 msin-flight 578 pool 577------------------------------envoy 1.64c sidecar 1.18cwrk hottest 0.92c mean 0.81c------------------------------mean request 8.62 ms before 2.61 envoy 2.34 sidecar 0.86 worker 2.81 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4005 /s (100.0% ok)p50 7.9 ms p95 23.9 msin-flight 208 pool 577------------------------------envoy 1.85c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.96 ms before 3.87 envoy 1.94 sidecar 0.86 worker 3.29 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 7.9 ms p95 30.6 msin-flight 318 pool 577------------------------------envoy 1.81c sidecar 1.32cwrk hottest 0.94c mean 0.91c------------------------------mean request 11.39 ms before 4.80 envoy 2.09 sidecar 0.87 worker 3.63 resume 0.83 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4292 /sachieved 4275 /s (100.0% ok)p50 7.5 ms p95 53.9 msin-flight 300 pool 577------------------------------envoy 1.83c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 20.50 ms before 10.05 envoy 3.36 sidecar 0.98 worker 6.12 resume 0.94 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 3732 /s (100.0% ok)p50 536.6 ms p95 4274.2 msin-flight 26k pool 26k------------------------------envoy 1.90c sidecar 1.07cwrk hottest 0.97c mean 0.95c------------------------------mean request 1311.04 ms before 698.40 envoy 346.99 sidecar 3.63 worker 262.03 resume 3.41 (control plane) +rung -1 0 qps setpoint------------------------------offered 375 /sachieved 375 /s (100.0% ok)p50 3.1 ms p95 4.5 msin-flight 39.0 pool 39.0------------------------------envoy 0.21c sidecar 0.15cwrk hottest 0.40c mean 0.27c------------------------------mean request 3.53 ms before 1.18 envoy 0.54 sidecar 0.79 worker 1.04 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.52c sidecar 0.48cwrk hottest 0.40c mean 0.26c------------------------------mean request 3.03 ms before 0.99 envoy 0.58 sidecar 0.78 worker 0.69 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 6.0 pool 39.0------------------------------envoy 0.54c sidecar 0.48cwrk hottest 0.43c mean 0.27c------------------------------mean request 3.05 ms before 1.01 envoy 0.58 sidecar 0.79 worker 0.68 resume 0.75 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1587 /sachieved 1586 /s (100.0% ok)p50 3.1 ms p95 4.0 msin-flight 10.0 pool 39.0------------------------------envoy 0.83c sidecar 0.53cwrk hottest 0.54c mean 0.43c------------------------------mean request 3.13 ms before 1.07 envoy 0.54 sidecar 0.80 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 11.0 pool 39.0------------------------------envoy 1.04c sidecar 0.88cwrk hottest 0.58c mean 0.52c------------------------------mean request 3.16 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.72 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 12.0 pool 39.0------------------------------envoy 1.03c sidecar 0.00cwrk hottest 0.57c mean 0.51c------------------------------mean request 3.15 ms before 1.09 envoy 0.54 sidecar 0.81 worker 0.71 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2201 /sachieved 2201 /s (100.0% ok)p50 3.2 ms p95 4.3 msin-flight 19.0 pool 39.0------------------------------envoy 1.12c sidecar 0.88cwrk hottest 0.75c mean 0.57c------------------------------mean request 3.28 ms before 1.17 envoy 0.53 sidecar 0.81 worker 0.77 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.5 ms p95 5.1 msin-flight 24.0 pool 39.0------------------------------envoy 1.50c sidecar 1.17cwrk hottest 0.77c mean 0.75c------------------------------mean request 3.67 ms before 1.34 envoy 0.53 sidecar 0.81 worker 0.99 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 2999 /s (100.0% ok)p50 3.5 ms p95 5.6 msin-flight 410 pool 410------------------------------envoy 1.48c sidecar 1.19cwrk hottest 0.82c mean 0.74c------------------------------mean request 6.10 ms before 3.82 envoy -0.19 sidecar 1.50 worker 0.96 resume 1.46 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3412 /sachieved 3407 /s (100.0% ok)p50 4.7 ms p95 21.1 msin-flight 578 pool 577------------------------------envoy 1.64c sidecar 1.18cwrk hottest 0.92c mean 0.81c------------------------------mean request 8.62 ms before 2.61 envoy 2.34 sidecar 0.86 worker 2.81 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4005 /s (100.0% ok)p50 7.9 ms p95 23.9 msin-flight 208 pool 577------------------------------envoy 1.85c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 9.96 ms before 3.87 envoy 1.94 sidecar 0.86 worker 3.29 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4001 /s (100.0% ok)p50 7.9 ms p95 30.6 msin-flight 318 pool 577------------------------------envoy 1.81c sidecar 1.32cwrk hottest 0.94c mean 0.91c------------------------------mean request 11.39 ms before 4.80 envoy 2.09 sidecar 0.87 worker 3.63 resume 0.83 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4292 /sachieved 4275 /s (100.0% ok)p50 7.5 ms p95 53.9 msin-flight 300 pool 577------------------------------envoy 1.83c sidecar 1.30cwrk hottest 0.94c mean 0.92c------------------------------mean request 20.50 ms before 10.05 envoy 3.36 sidecar 0.98 worker 6.12 resume 0.94 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 3732 /s (100.0% ok)p50 536.6 ms p95 4274.2 msin-flight 26k pool 26k------------------------------envoy 1.90c sidecar 1.07cwrk hottest 0.97c mean 0.95c------------------------------mean request 1311.04 ms before 698.40 envoy 346.99 sidecar 3.63 worker 262.03 resume 3.41 (control plane) +
+ + + +atenet-router capacity — 8c/4t +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 + +0 + +20k + +40k +requests / s + + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + +0 + +2.0 + +4.0 + +6.0 + +8.0 +cores + + + + + + + + +envoy CPU limit, 8 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + +0 + +1.0G + +2.0G +bytes + +50.0 + +100 + +150 + +200 + +250 + +300 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 497 /sachieved 497 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 10.0 pool 10.0------------------------------envoy 0.28c sidecar 0.00cwrk hottest 0.21c mean 0.14c------------------------------mean request 3.12 ms before 0.91 envoy 0.60 sidecar 0.78 worker 0.83 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 10.0------------------------------envoy 0.56c sidecar 0.39cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.95 envoy 0.70 sidecar 0.77 worker 0.56 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 13.0 pool 13.0------------------------------envoy 0.56c sidecar 0.48cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.96 envoy 0.73 sidecar 0.77 worker 0.50 resume 0.73 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1816 /sachieved 1816 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 120 pool 121------------------------------envoy 1.03c sidecar 0.73cwrk hottest 0.49c mean 0.25c------------------------------mean request 3.21 ms before 1.07 envoy 0.72 sidecar 0.87 worker 0.55 resume 0.83 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 66.0 pool 121------------------------------envoy 1.13c sidecar 0.00cwrk hottest 0.54c mean 0.29c------------------------------mean request 3.07 ms before 0.99 envoy 0.71 sidecar 0.80 worker 0.57 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 10.0 pool 121------------------------------envoy 1.11c sidecar 0.89cwrk hottest 0.54c mean 0.27c------------------------------mean request 3.03 ms before 0.98 envoy 0.69 sidecar 0.79 worker 0.56 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2708 /sachieved 2708 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 24.0 pool 121------------------------------envoy 1.52c sidecar 0.88cwrk hottest 0.66c mean 0.39c------------------------------mean request 3.15 ms before 1.05 envoy 0.62 sidecar 0.82 worker 0.66 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 30.0 pool 121------------------------------envoy 1.59c sidecar 1.21cwrk hottest 0.75c mean 0.40c------------------------------mean request 3.21 ms before 1.10 envoy 0.58 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3359 /sachieved 3359 /s (100.0% ok)p50 3.2 ms p95 4.9 msin-flight 31.0 pool 121------------------------------envoy 1.79c sidecar 1.21cwrk hottest 0.79c mean 0.45c------------------------------mean request 3.43 ms before 1.22 envoy 0.61 sidecar 0.81 worker 0.80 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 199 pool 199------------------------------envoy 2.18c sidecar 1.44cwrk hottest 0.74c mean 0.54c------------------------------mean request 3.63 ms before 1.24 envoy 0.60 sidecar 0.92 worker 0.87 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 34.0 pool 199------------------------------envoy 2.21c sidecar 1.50cwrk hottest 0.70c mean 0.55c------------------------------mean request 3.37 ms before 1.16 envoy 0.59 sidecar 0.81 worker 0.82 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4424 /sachieved 4424 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 74.0 pool 199------------------------------envoy 2.40c sidecar 1.52cwrk hottest 0.80c mean 0.61c------------------------------mean request 3.71 ms before 1.35 envoy 0.62 sidecar 0.81 worker 0.92 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.9 ms p95 9.1 msin-flight 1144 pool 1144------------------------------envoy 2.73c sidecar 1.69cwrk hottest 0.80c mean 0.68c------------------------------mean request 8.14 ms before 3.14 envoy 1.21 sidecar 1.45 worker 2.35 resume 1.41 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.8 ms p95 6.8 msin-flight 48.0 pool 1144------------------------------envoy 2.74c sidecar 1.77cwrk hottest 0.83c mean 0.69c------------------------------mean request 4.19 ms before 1.48 envoy 0.75 sidecar 0.81 worker 1.14 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5564 /sachieved 5563 /s (100.0% ok)p50 5.9 ms p95 596.3 msin-flight 4110 pool 4110------------------------------envoy 2.94c sidecar 1.68cwrk hottest 0.84c mean 0.73c------------------------------mean request 105.69 ms before 31.70 envoy 43.67 sidecar 1.33 worker 28.98 resume 1.27 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 7.0 ms p95 20.7 msin-flight 221 pool 4110------------------------------envoy 3.17c sidecar 1.92cwrk hottest 0.84c mean 0.79c------------------------------mean request 8.81 ms before 3.39 envoy 1.76 sidecar 0.87 worker 2.80 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5991 /s (99.5% ok)p50 8.4 ms p95 161.1 msin-flight 1935 pool 4110------------------------------envoy 3.14c sidecar 1.85cwrk hottest 0.85c mean 0.79c------------------------------mean request 26.78 ms before 10.23 envoy 5.13 sidecar 2.98 worker 8.44 resume 2.96 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6827 /sachieved 6246 /s (100.0% ok)p50 365.5 ms p95 1239.1 msin-flight 12k pool 12k------------------------------envoy 3.25c sidecar 1.75cwrk hottest 0.84c mean 0.81c------------------------------mean request 431.97 ms before 106.87 envoy 210.58 sidecar 2.26 worker 112.26 resume 2.18 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5484 /s (100.0% ok)p50 3060.6 ms p95 5954.7 msin-flight 41k pool 41k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3203.74 ms before 809.45 envoy 1420.77 sidecar 4.38 worker 969.14 resume 4.05 (control plane) +rung -1 0 qps setpoint------------------------------offered 497 /sachieved 497 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 10.0 pool 10.0------------------------------envoy 0.28c sidecar 0.00cwrk hottest 0.21c mean 0.14c------------------------------mean request 3.12 ms before 0.91 envoy 0.60 sidecar 0.78 worker 0.83 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 10.0------------------------------envoy 0.56c sidecar 0.39cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.95 envoy 0.70 sidecar 0.77 worker 0.56 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 13.0 pool 13.0------------------------------envoy 0.56c sidecar 0.48cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.96 envoy 0.73 sidecar 0.77 worker 0.50 resume 0.73 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1816 /sachieved 1816 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 120 pool 121------------------------------envoy 1.03c sidecar 0.73cwrk hottest 0.49c mean 0.25c------------------------------mean request 3.21 ms before 1.07 envoy 0.72 sidecar 0.87 worker 0.55 resume 0.83 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 66.0 pool 121------------------------------envoy 1.13c sidecar 0.00cwrk hottest 0.54c mean 0.29c------------------------------mean request 3.07 ms before 0.99 envoy 0.71 sidecar 0.80 worker 0.57 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 10.0 pool 121------------------------------envoy 1.11c sidecar 0.89cwrk hottest 0.54c mean 0.27c------------------------------mean request 3.03 ms before 0.98 envoy 0.69 sidecar 0.79 worker 0.56 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2708 /sachieved 2708 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 24.0 pool 121------------------------------envoy 1.52c sidecar 0.88cwrk hottest 0.66c mean 0.39c------------------------------mean request 3.15 ms before 1.05 envoy 0.62 sidecar 0.82 worker 0.66 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 30.0 pool 121------------------------------envoy 1.59c sidecar 1.21cwrk hottest 0.75c mean 0.40c------------------------------mean request 3.21 ms before 1.10 envoy 0.58 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3359 /sachieved 3359 /s (100.0% ok)p50 3.2 ms p95 4.9 msin-flight 31.0 pool 121------------------------------envoy 1.79c sidecar 1.21cwrk hottest 0.79c mean 0.45c------------------------------mean request 3.43 ms before 1.22 envoy 0.61 sidecar 0.81 worker 0.80 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 199 pool 199------------------------------envoy 2.18c sidecar 1.44cwrk hottest 0.74c mean 0.54c------------------------------mean request 3.63 ms before 1.24 envoy 0.60 sidecar 0.92 worker 0.87 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 34.0 pool 199------------------------------envoy 2.21c sidecar 1.50cwrk hottest 0.70c mean 0.55c------------------------------mean request 3.37 ms before 1.16 envoy 0.59 sidecar 0.81 worker 0.82 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4424 /sachieved 4424 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 74.0 pool 199------------------------------envoy 2.40c sidecar 1.52cwrk hottest 0.80c mean 0.61c------------------------------mean request 3.71 ms before 1.35 envoy 0.62 sidecar 0.81 worker 0.92 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.9 ms p95 9.1 msin-flight 1144 pool 1144------------------------------envoy 2.73c sidecar 1.69cwrk hottest 0.80c mean 0.68c------------------------------mean request 8.14 ms before 3.14 envoy 1.21 sidecar 1.45 worker 2.35 resume 1.41 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.8 ms p95 6.8 msin-flight 48.0 pool 1144------------------------------envoy 2.74c sidecar 1.77cwrk hottest 0.83c mean 0.69c------------------------------mean request 4.19 ms before 1.48 envoy 0.75 sidecar 0.81 worker 1.14 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5564 /sachieved 5563 /s (100.0% ok)p50 5.9 ms p95 596.3 msin-flight 4110 pool 4110------------------------------envoy 2.94c sidecar 1.68cwrk hottest 0.84c mean 0.73c------------------------------mean request 105.69 ms before 31.70 envoy 43.67 sidecar 1.33 worker 28.98 resume 1.27 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 7.0 ms p95 20.7 msin-flight 221 pool 4110------------------------------envoy 3.17c sidecar 1.92cwrk hottest 0.84c mean 0.79c------------------------------mean request 8.81 ms before 3.39 envoy 1.76 sidecar 0.87 worker 2.80 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5991 /s (99.5% ok)p50 8.4 ms p95 161.1 msin-flight 1935 pool 4110------------------------------envoy 3.14c sidecar 1.85cwrk hottest 0.85c mean 0.79c------------------------------mean request 26.78 ms before 10.23 envoy 5.13 sidecar 2.98 worker 8.44 resume 2.96 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6827 /sachieved 6246 /s (100.0% ok)p50 365.5 ms p95 1239.1 msin-flight 12k pool 12k------------------------------envoy 3.25c sidecar 1.75cwrk hottest 0.84c mean 0.81c------------------------------mean request 431.97 ms before 106.87 envoy 210.58 sidecar 2.26 worker 112.26 resume 2.18 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5484 /s (100.0% ok)p50 3060.6 ms p95 5954.7 msin-flight 41k pool 41k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3203.74 ms before 809.45 envoy 1420.77 sidecar 4.38 worker 969.14 resume 4.05 (control plane) +rung -1 0 qps setpoint------------------------------offered 497 /sachieved 497 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 10.0 pool 10.0------------------------------envoy 0.28c sidecar 0.00cwrk hottest 0.21c mean 0.14c------------------------------mean request 3.12 ms before 0.91 envoy 0.60 sidecar 0.78 worker 0.83 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 10.0------------------------------envoy 0.56c sidecar 0.39cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.95 envoy 0.70 sidecar 0.77 worker 0.56 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 13.0 pool 13.0------------------------------envoy 0.56c sidecar 0.48cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.96 envoy 0.73 sidecar 0.77 worker 0.50 resume 0.73 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1816 /sachieved 1816 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 120 pool 121------------------------------envoy 1.03c sidecar 0.73cwrk hottest 0.49c mean 0.25c------------------------------mean request 3.21 ms before 1.07 envoy 0.72 sidecar 0.87 worker 0.55 resume 0.83 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 66.0 pool 121------------------------------envoy 1.13c sidecar 0.00cwrk hottest 0.54c mean 0.29c------------------------------mean request 3.07 ms before 0.99 envoy 0.71 sidecar 0.80 worker 0.57 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 10.0 pool 121------------------------------envoy 1.11c sidecar 0.89cwrk hottest 0.54c mean 0.27c------------------------------mean request 3.03 ms before 0.98 envoy 0.69 sidecar 0.79 worker 0.56 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2708 /sachieved 2708 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 24.0 pool 121------------------------------envoy 1.52c sidecar 0.88cwrk hottest 0.66c mean 0.39c------------------------------mean request 3.15 ms before 1.05 envoy 0.62 sidecar 0.82 worker 0.66 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 30.0 pool 121------------------------------envoy 1.59c sidecar 1.21cwrk hottest 0.75c mean 0.40c------------------------------mean request 3.21 ms before 1.10 envoy 0.58 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3359 /sachieved 3359 /s (100.0% ok)p50 3.2 ms p95 4.9 msin-flight 31.0 pool 121------------------------------envoy 1.79c sidecar 1.21cwrk hottest 0.79c mean 0.45c------------------------------mean request 3.43 ms before 1.22 envoy 0.61 sidecar 0.81 worker 0.80 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 199 pool 199------------------------------envoy 2.18c sidecar 1.44cwrk hottest 0.74c mean 0.54c------------------------------mean request 3.63 ms before 1.24 envoy 0.60 sidecar 0.92 worker 0.87 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 34.0 pool 199------------------------------envoy 2.21c sidecar 1.50cwrk hottest 0.70c mean 0.55c------------------------------mean request 3.37 ms before 1.16 envoy 0.59 sidecar 0.81 worker 0.82 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4424 /sachieved 4424 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 74.0 pool 199------------------------------envoy 2.40c sidecar 1.52cwrk hottest 0.80c mean 0.61c------------------------------mean request 3.71 ms before 1.35 envoy 0.62 sidecar 0.81 worker 0.92 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.9 ms p95 9.1 msin-flight 1144 pool 1144------------------------------envoy 2.73c sidecar 1.69cwrk hottest 0.80c mean 0.68c------------------------------mean request 8.14 ms before 3.14 envoy 1.21 sidecar 1.45 worker 2.35 resume 1.41 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.8 ms p95 6.8 msin-flight 48.0 pool 1144------------------------------envoy 2.74c sidecar 1.77cwrk hottest 0.83c mean 0.69c------------------------------mean request 4.19 ms before 1.48 envoy 0.75 sidecar 0.81 worker 1.14 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5564 /sachieved 5563 /s (100.0% ok)p50 5.9 ms p95 596.3 msin-flight 4110 pool 4110------------------------------envoy 2.94c sidecar 1.68cwrk hottest 0.84c mean 0.73c------------------------------mean request 105.69 ms before 31.70 envoy 43.67 sidecar 1.33 worker 28.98 resume 1.27 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 7.0 ms p95 20.7 msin-flight 221 pool 4110------------------------------envoy 3.17c sidecar 1.92cwrk hottest 0.84c mean 0.79c------------------------------mean request 8.81 ms before 3.39 envoy 1.76 sidecar 0.87 worker 2.80 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5991 /s (99.5% ok)p50 8.4 ms p95 161.1 msin-flight 1935 pool 4110------------------------------envoy 3.14c sidecar 1.85cwrk hottest 0.85c mean 0.79c------------------------------mean request 26.78 ms before 10.23 envoy 5.13 sidecar 2.98 worker 8.44 resume 2.96 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6827 /sachieved 6246 /s (100.0% ok)p50 365.5 ms p95 1239.1 msin-flight 12k pool 12k------------------------------envoy 3.25c sidecar 1.75cwrk hottest 0.84c mean 0.81c------------------------------mean request 431.97 ms before 106.87 envoy 210.58 sidecar 2.26 worker 112.26 resume 2.18 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5484 /s (100.0% ok)p50 3060.6 ms p95 5954.7 msin-flight 41k pool 41k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3203.74 ms before 809.45 envoy 1420.77 sidecar 4.38 worker 969.14 resume 4.05 (control plane) +rung -1 0 qps setpoint------------------------------offered 497 /sachieved 497 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 10.0 pool 10.0------------------------------envoy 0.28c sidecar 0.00cwrk hottest 0.21c mean 0.14c------------------------------mean request 3.12 ms before 0.91 envoy 0.60 sidecar 0.78 worker 0.83 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 10.0------------------------------envoy 0.56c sidecar 0.39cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.95 envoy 0.70 sidecar 0.77 worker 0.56 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 13.0 pool 13.0------------------------------envoy 0.56c sidecar 0.48cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.96 envoy 0.73 sidecar 0.77 worker 0.50 resume 0.73 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1816 /sachieved 1816 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 120 pool 121------------------------------envoy 1.03c sidecar 0.73cwrk hottest 0.49c mean 0.25c------------------------------mean request 3.21 ms before 1.07 envoy 0.72 sidecar 0.87 worker 0.55 resume 0.83 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 66.0 pool 121------------------------------envoy 1.13c sidecar 0.00cwrk hottest 0.54c mean 0.29c------------------------------mean request 3.07 ms before 0.99 envoy 0.71 sidecar 0.80 worker 0.57 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 10.0 pool 121------------------------------envoy 1.11c sidecar 0.89cwrk hottest 0.54c mean 0.27c------------------------------mean request 3.03 ms before 0.98 envoy 0.69 sidecar 0.79 worker 0.56 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2708 /sachieved 2708 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 24.0 pool 121------------------------------envoy 1.52c sidecar 0.88cwrk hottest 0.66c mean 0.39c------------------------------mean request 3.15 ms before 1.05 envoy 0.62 sidecar 0.82 worker 0.66 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 30.0 pool 121------------------------------envoy 1.59c sidecar 1.21cwrk hottest 0.75c mean 0.40c------------------------------mean request 3.21 ms before 1.10 envoy 0.58 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3359 /sachieved 3359 /s (100.0% ok)p50 3.2 ms p95 4.9 msin-flight 31.0 pool 121------------------------------envoy 1.79c sidecar 1.21cwrk hottest 0.79c mean 0.45c------------------------------mean request 3.43 ms before 1.22 envoy 0.61 sidecar 0.81 worker 0.80 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 199 pool 199------------------------------envoy 2.18c sidecar 1.44cwrk hottest 0.74c mean 0.54c------------------------------mean request 3.63 ms before 1.24 envoy 0.60 sidecar 0.92 worker 0.87 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 34.0 pool 199------------------------------envoy 2.21c sidecar 1.50cwrk hottest 0.70c mean 0.55c------------------------------mean request 3.37 ms before 1.16 envoy 0.59 sidecar 0.81 worker 0.82 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4424 /sachieved 4424 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 74.0 pool 199------------------------------envoy 2.40c sidecar 1.52cwrk hottest 0.80c mean 0.61c------------------------------mean request 3.71 ms before 1.35 envoy 0.62 sidecar 0.81 worker 0.92 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.9 ms p95 9.1 msin-flight 1144 pool 1144------------------------------envoy 2.73c sidecar 1.69cwrk hottest 0.80c mean 0.68c------------------------------mean request 8.14 ms before 3.14 envoy 1.21 sidecar 1.45 worker 2.35 resume 1.41 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.8 ms p95 6.8 msin-flight 48.0 pool 1144------------------------------envoy 2.74c sidecar 1.77cwrk hottest 0.83c mean 0.69c------------------------------mean request 4.19 ms before 1.48 envoy 0.75 sidecar 0.81 worker 1.14 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5564 /sachieved 5563 /s (100.0% ok)p50 5.9 ms p95 596.3 msin-flight 4110 pool 4110------------------------------envoy 2.94c sidecar 1.68cwrk hottest 0.84c mean 0.73c------------------------------mean request 105.69 ms before 31.70 envoy 43.67 sidecar 1.33 worker 28.98 resume 1.27 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 7.0 ms p95 20.7 msin-flight 221 pool 4110------------------------------envoy 3.17c sidecar 1.92cwrk hottest 0.84c mean 0.79c------------------------------mean request 8.81 ms before 3.39 envoy 1.76 sidecar 0.87 worker 2.80 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5991 /s (99.5% ok)p50 8.4 ms p95 161.1 msin-flight 1935 pool 4110------------------------------envoy 3.14c sidecar 1.85cwrk hottest 0.85c mean 0.79c------------------------------mean request 26.78 ms before 10.23 envoy 5.13 sidecar 2.98 worker 8.44 resume 2.96 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6827 /sachieved 6246 /s (100.0% ok)p50 365.5 ms p95 1239.1 msin-flight 12k pool 12k------------------------------envoy 3.25c sidecar 1.75cwrk hottest 0.84c mean 0.81c------------------------------mean request 431.97 ms before 106.87 envoy 210.58 sidecar 2.26 worker 112.26 resume 2.18 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5484 /s (100.0% ok)p50 3060.6 ms p95 5954.7 msin-flight 41k pool 41k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3203.74 ms before 809.45 envoy 1420.77 sidecar 4.38 worker 969.14 resume 4.05 (control plane) +rung -1 0 qps setpoint------------------------------offered 497 /sachieved 497 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 10.0 pool 10.0------------------------------envoy 0.28c sidecar 0.00cwrk hottest 0.21c mean 0.14c------------------------------mean request 3.12 ms before 0.91 envoy 0.60 sidecar 0.78 worker 0.83 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 10.0------------------------------envoy 0.56c sidecar 0.39cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.95 envoy 0.70 sidecar 0.77 worker 0.56 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 13.0 pool 13.0------------------------------envoy 0.56c sidecar 0.48cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.96 envoy 0.73 sidecar 0.77 worker 0.50 resume 0.73 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1816 /sachieved 1816 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 120 pool 121------------------------------envoy 1.03c sidecar 0.73cwrk hottest 0.49c mean 0.25c------------------------------mean request 3.21 ms before 1.07 envoy 0.72 sidecar 0.87 worker 0.55 resume 0.83 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 66.0 pool 121------------------------------envoy 1.13c sidecar 0.00cwrk hottest 0.54c mean 0.29c------------------------------mean request 3.07 ms before 0.99 envoy 0.71 sidecar 0.80 worker 0.57 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 10.0 pool 121------------------------------envoy 1.11c sidecar 0.89cwrk hottest 0.54c mean 0.27c------------------------------mean request 3.03 ms before 0.98 envoy 0.69 sidecar 0.79 worker 0.56 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2708 /sachieved 2708 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 24.0 pool 121------------------------------envoy 1.52c sidecar 0.88cwrk hottest 0.66c mean 0.39c------------------------------mean request 3.15 ms before 1.05 envoy 0.62 sidecar 0.82 worker 0.66 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 30.0 pool 121------------------------------envoy 1.59c sidecar 1.21cwrk hottest 0.75c mean 0.40c------------------------------mean request 3.21 ms before 1.10 envoy 0.58 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3359 /sachieved 3359 /s (100.0% ok)p50 3.2 ms p95 4.9 msin-flight 31.0 pool 121------------------------------envoy 1.79c sidecar 1.21cwrk hottest 0.79c mean 0.45c------------------------------mean request 3.43 ms before 1.22 envoy 0.61 sidecar 0.81 worker 0.80 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 199 pool 199------------------------------envoy 2.18c sidecar 1.44cwrk hottest 0.74c mean 0.54c------------------------------mean request 3.63 ms before 1.24 envoy 0.60 sidecar 0.92 worker 0.87 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 34.0 pool 199------------------------------envoy 2.21c sidecar 1.50cwrk hottest 0.70c mean 0.55c------------------------------mean request 3.37 ms before 1.16 envoy 0.59 sidecar 0.81 worker 0.82 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4424 /sachieved 4424 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 74.0 pool 199------------------------------envoy 2.40c sidecar 1.52cwrk hottest 0.80c mean 0.61c------------------------------mean request 3.71 ms before 1.35 envoy 0.62 sidecar 0.81 worker 0.92 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.9 ms p95 9.1 msin-flight 1144 pool 1144------------------------------envoy 2.73c sidecar 1.69cwrk hottest 0.80c mean 0.68c------------------------------mean request 8.14 ms before 3.14 envoy 1.21 sidecar 1.45 worker 2.35 resume 1.41 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.8 ms p95 6.8 msin-flight 48.0 pool 1144------------------------------envoy 2.74c sidecar 1.77cwrk hottest 0.83c mean 0.69c------------------------------mean request 4.19 ms before 1.48 envoy 0.75 sidecar 0.81 worker 1.14 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5564 /sachieved 5563 /s (100.0% ok)p50 5.9 ms p95 596.3 msin-flight 4110 pool 4110------------------------------envoy 2.94c sidecar 1.68cwrk hottest 0.84c mean 0.73c------------------------------mean request 105.69 ms before 31.70 envoy 43.67 sidecar 1.33 worker 28.98 resume 1.27 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 7.0 ms p95 20.7 msin-flight 221 pool 4110------------------------------envoy 3.17c sidecar 1.92cwrk hottest 0.84c mean 0.79c------------------------------mean request 8.81 ms before 3.39 envoy 1.76 sidecar 0.87 worker 2.80 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5991 /s (99.5% ok)p50 8.4 ms p95 161.1 msin-flight 1935 pool 4110------------------------------envoy 3.14c sidecar 1.85cwrk hottest 0.85c mean 0.79c------------------------------mean request 26.78 ms before 10.23 envoy 5.13 sidecar 2.98 worker 8.44 resume 2.96 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6827 /sachieved 6246 /s (100.0% ok)p50 365.5 ms p95 1239.1 msin-flight 12k pool 12k------------------------------envoy 3.25c sidecar 1.75cwrk hottest 0.84c mean 0.81c------------------------------mean request 431.97 ms before 106.87 envoy 210.58 sidecar 2.26 worker 112.26 resume 2.18 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5484 /s (100.0% ok)p50 3060.6 ms p95 5954.7 msin-flight 41k pool 41k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3203.74 ms before 809.45 envoy 1420.77 sidecar 4.38 worker 969.14 resume 4.05 (control plane) +rung -1 0 qps setpoint------------------------------offered 497 /sachieved 497 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 10.0 pool 10.0------------------------------envoy 0.28c sidecar 0.00cwrk hottest 0.21c mean 0.14c------------------------------mean request 3.12 ms before 0.91 envoy 0.60 sidecar 0.78 worker 0.83 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 5.0 pool 10.0------------------------------envoy 0.56c sidecar 0.39cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.95 envoy 0.70 sidecar 0.77 worker 0.56 resume 0.73 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 2.9 ms p95 3.6 msin-flight 13.0 pool 13.0------------------------------envoy 0.56c sidecar 0.48cwrk hottest 0.27c mean 0.14c------------------------------mean request 2.97 ms before 0.96 envoy 0.73 sidecar 0.77 worker 0.50 resume 0.73 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 1816 /sachieved 1816 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 120 pool 121------------------------------envoy 1.03c sidecar 0.73cwrk hottest 0.49c mean 0.25c------------------------------mean request 3.21 ms before 1.07 envoy 0.72 sidecar 0.87 worker 0.55 resume 0.83 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 66.0 pool 121------------------------------envoy 1.13c sidecar 0.00cwrk hottest 0.54c mean 0.29c------------------------------mean request 3.07 ms before 0.99 envoy 0.71 sidecar 0.80 worker 0.57 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 10.0 pool 121------------------------------envoy 1.11c sidecar 0.89cwrk hottest 0.54c mean 0.27c------------------------------mean request 3.03 ms before 0.98 envoy 0.69 sidecar 0.79 worker 0.56 resume 0.75 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2708 /sachieved 2708 /s (100.0% ok)p50 3.1 ms p95 4.2 msin-flight 24.0 pool 121------------------------------envoy 1.52c sidecar 0.88cwrk hottest 0.66c mean 0.39c------------------------------mean request 3.15 ms before 1.05 envoy 0.62 sidecar 0.82 worker 0.66 resume 0.78 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 30.0 pool 121------------------------------envoy 1.59c sidecar 1.21cwrk hottest 0.75c mean 0.40c------------------------------mean request 3.21 ms before 1.10 envoy 0.58 sidecar 0.80 worker 0.73 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3359 /sachieved 3359 /s (100.0% ok)p50 3.2 ms p95 4.9 msin-flight 31.0 pool 121------------------------------envoy 1.79c sidecar 1.21cwrk hottest 0.79c mean 0.45c------------------------------mean request 3.43 ms before 1.22 envoy 0.61 sidecar 0.81 worker 0.80 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.0 msin-flight 199 pool 199------------------------------envoy 2.18c sidecar 1.44cwrk hottest 0.74c mean 0.54c------------------------------mean request 3.63 ms before 1.24 envoy 0.60 sidecar 0.92 worker 0.87 resume 0.82 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 4.6 msin-flight 34.0 pool 199------------------------------envoy 2.21c sidecar 1.50cwrk hottest 0.70c mean 0.55c------------------------------mean request 3.37 ms before 1.16 envoy 0.59 sidecar 0.81 worker 0.82 resume 0.77 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4424 /sachieved 4424 /s (100.0% ok)p50 3.4 ms p95 5.6 msin-flight 74.0 pool 199------------------------------envoy 2.40c sidecar 1.52cwrk hottest 0.80c mean 0.61c------------------------------mean request 3.71 ms before 1.35 envoy 0.62 sidecar 0.81 worker 0.92 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.9 ms p95 9.1 msin-flight 1144 pool 1144------------------------------envoy 2.73c sidecar 1.69cwrk hottest 0.80c mean 0.68c------------------------------mean request 8.14 ms before 3.14 envoy 1.21 sidecar 1.45 worker 2.35 resume 1.41 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.8 ms p95 6.8 msin-flight 48.0 pool 1144------------------------------envoy 2.74c sidecar 1.77cwrk hottest 0.83c mean 0.69c------------------------------mean request 4.19 ms before 1.48 envoy 0.75 sidecar 0.81 worker 1.14 resume 0.78 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5564 /sachieved 5563 /s (100.0% ok)p50 5.9 ms p95 596.3 msin-flight 4110 pool 4110------------------------------envoy 2.94c sidecar 1.68cwrk hottest 0.84c mean 0.73c------------------------------mean request 105.69 ms before 31.70 envoy 43.67 sidecar 1.33 worker 28.98 resume 1.27 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 7.0 ms p95 20.7 msin-flight 221 pool 4110------------------------------envoy 3.17c sidecar 1.92cwrk hottest 0.84c mean 0.79c------------------------------mean request 8.81 ms before 3.39 envoy 1.76 sidecar 0.87 worker 2.80 resume 0.83 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5991 /s (99.5% ok)p50 8.4 ms p95 161.1 msin-flight 1935 pool 4110------------------------------envoy 3.14c sidecar 1.85cwrk hottest 0.85c mean 0.79c------------------------------mean request 26.78 ms before 10.23 envoy 5.13 sidecar 2.98 worker 8.44 resume 2.96 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6827 /sachieved 6246 /s (100.0% ok)p50 365.5 ms p95 1239.1 msin-flight 12k pool 12k------------------------------envoy 3.25c sidecar 1.75cwrk hottest 0.84c mean 0.81c------------------------------mean request 431.97 ms before 106.87 envoy 210.58 sidecar 2.26 worker 112.26 resume 2.18 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 5484 /s (100.0% ok)p50 3060.6 ms p95 5954.7 msin-flight 41k pool 41k------------------------------envoy 3.40c sidecar 1.17cwrk hottest 0.86c mean 0.85c------------------------------mean request 3203.74 ms before 809.45 envoy 1420.77 sidecar 4.38 worker 969.14 resume 4.05 (control plane) +
+ + + +atenet-router capacity — 8c/8t +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 + +0 + +10k + +20k + +30k + +40k +requests / s + + + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + +0 + +2.0 + +4.0 + +6.0 + +8.0 +cores + + + + + + + + + +envoy CPU limit, 8 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + +0 + +1.0G + +2.0G +bytes + +50.0 + +100 + +150 + +200 + +250 + +300 + +350 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 296 /sachieved 296 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 45.0 pool 45.0------------------------------envoy 0.22c sidecar 0.03cwrk hottest 0.22c mean 0.08c------------------------------mean request 3.84 ms before 1.09 envoy 0.58 sidecar 0.79 worker 1.38 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 45.0------------------------------envoy 0.60c sidecar 0.00cwrk hottest 0.26c mean 0.07c------------------------------mean request 3.06 ms before 0.95 envoy 0.63 sidecar 0.79 worker 0.69 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.6 msin-flight 21.0 pool 45.0------------------------------envoy 0.56c sidecar 0.50cwrk hottest 0.23c mean 0.07c------------------------------mean request 3.00 ms before 0.95 envoy 0.66 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1172 /sachieved 1172 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 13.0 pool 45.0------------------------------envoy 0.67c sidecar 0.51cwrk hottest 0.26c mean 0.08c------------------------------mean request 2.99 ms before 0.95 envoy 0.69 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.7 msin-flight 9.0 pool 45.0------------------------------envoy 1.17c sidecar 0.94cwrk hottest 0.43c mean 0.15c------------------------------mean request 2.93 ms before 0.91 envoy 0.80 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 21.0 pool 45.0------------------------------envoy 1.16c sidecar 0.93cwrk hottest 0.47c mean 0.14c------------------------------mean request 2.95 ms before 0.92 envoy 0.78 sidecar 0.79 worker 0.46 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2207 /sachieved 2207 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 14.0 pool 45.0------------------------------envoy 1.28c sidecar 0.93cwrk hottest 0.42c mean 0.15c------------------------------mean request 2.96 ms before 0.94 envoy 0.79 sidecar 0.81 worker 0.43 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 21.0 pool 45.0------------------------------envoy 1.82c sidecar 1.25cwrk hottest 0.52c mean 0.23c------------------------------mean request 3.01 ms before 0.95 envoy 0.74 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 32.0 pool 45.0------------------------------envoy 1.83c sidecar 0.00cwrk hottest 0.53c mean 0.23c------------------------------mean request 3.04 ms before 0.96 envoy 0.69 sidecar 0.81 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 25.0 pool 32.0------------------------------envoy 1.79c sidecar 1.35cwrk hottest 0.49c mean 0.23c------------------------------mean request 3.07 ms before 0.98 envoy 0.69 sidecar 0.81 worker 0.59 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3907 /sachieved 3906 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 536 pool 536------------------------------envoy 2.32c sidecar 1.43cwrk hottest 0.76c mean 0.30c------------------------------mean request 4.15 ms before 1.14 envoy 0.87 sidecar 1.20 worker 0.94 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 276 pool 536------------------------------envoy 2.36c sidecar 1.65cwrk hottest 0.81c mean 0.29c------------------------------mean request 3.58 ms before 1.14 envoy 0.68 sidecar 0.97 worker 0.78 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4088 /sachieved 4088 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 63.0 pool 536------------------------------envoy 2.42c sidecar 1.62cwrk hottest 0.76c mean 0.30c------------------------------mean request 3.49 ms before 1.21 envoy 0.72 sidecar 0.82 worker 0.74 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 3.5 ms p95 16.7 msin-flight 65.0 pool 536------------------------------envoy 2.95c sidecar 1.70cwrk hottest 0.87c mean 0.37c------------------------------mean request 6.06 ms before 2.52 envoy 1.12 sidecar 0.83 worker 1.59 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.5 ms p95 17.2 msin-flight 75.0 pool 536------------------------------envoy 2.89c sidecar 1.90cwrk hottest 0.88c mean 0.36c------------------------------mean request 6.08 ms before 2.28 envoy 1.23 sidecar 0.83 worker 1.74 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5253 /sachieved 5250 /s (100.0% ok)p50 3.6 ms p95 21.2 msin-flight 85.0 pool 536------------------------------envoy 3.06c sidecar 1.95cwrk hottest 0.88c mean 0.40c------------------------------mean request 6.79 ms before 2.54 envoy 1.36 sidecar 0.83 worker 2.05 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 4.0 ms p95 24.7 msin-flight 90.0 pool 536------------------------------envoy 3.68c sidecar 0.00cwrk hottest 0.82c mean 0.46c------------------------------mean request 7.38 ms before 2.73 envoy 1.49 sidecar 0.85 worker 2.31 resume 0.81 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.7 ms p95 11.0 msin-flight 79.0 pool 536------------------------------envoy 3.65c sidecar 2.22cwrk hottest 0.80c mean 0.45c------------------------------mean request 4.78 ms before 1.58 envoy 0.98 sidecar 0.82 worker 1.39 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.9 ms p95 19.9 msin-flight 116 pool 275------------------------------envoy 3.65c sidecar 2.21cwrk hottest 0.82c mean 0.46c------------------------------mean request 6.63 ms before 2.55 envoy 1.29 sidecar 0.85 worker 1.94 resume 0.81 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6940 /sachieved 6940 /s (100.0% ok)p50 6.8 ms p95 22.5 msin-flight 642 pool 641------------------------------envoy 4.15c sidecar 2.26cwrk hottest 0.76c mean 0.52c------------------------------mean request 9.42 ms before 3.49 envoy 1.99 sidecar 0.90 worker 3.04 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (99.2% ok)p50 5.5 ms p95 77.5 msin-flight 1865 pool 1864------------------------------envoy 4.39c sidecar 2.42cwrk hottest 0.74c mean 0.55c------------------------------mean request 16.86 ms before 5.20 envoy 2.93 sidecar 3.47 worker 5.26 resume 3.46 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7150 /sachieved 7122 /s (99.6% ok)p50 7.0 ms p95 81.7 msin-flight 1822 pool 1864------------------------------envoy 4.39c sidecar 2.48cwrk hottest 0.70c mean 0.55c------------------------------mean request 18.59 ms before 7.73 envoy 3.00 sidecar 2.77 worker 5.10 resume 2.74 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 7604 /s (100.0% ok)p50 22.1 ms p95 312.3 msin-flight 13k pool 13k------------------------------envoy 4.77c sidecar 2.50cwrk hottest 0.67c mean 0.58c------------------------------mean request 52.42 ms before 12.00 envoy 14.64 sidecar 1.83 worker 23.94 resume 1.78 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 6543 /s (100.0% ok)p50 2758.7 ms p95 6534.9 msin-flight 34k pool 34k------------------------------envoy 4.72c sidecar 1.57cwrk hottest 0.63c mean 0.61c------------------------------mean request 2969.05 ms before 727.10 envoy 1393.75 sidecar 3.89 worker 844.31 resume 3.45 (control plane) +rung -1 0 qps setpoint------------------------------offered 296 /sachieved 296 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 45.0 pool 45.0------------------------------envoy 0.22c sidecar 0.03cwrk hottest 0.22c mean 0.08c------------------------------mean request 3.84 ms before 1.09 envoy 0.58 sidecar 0.79 worker 1.38 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 45.0------------------------------envoy 0.60c sidecar 0.00cwrk hottest 0.26c mean 0.07c------------------------------mean request 3.06 ms before 0.95 envoy 0.63 sidecar 0.79 worker 0.69 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.6 msin-flight 21.0 pool 45.0------------------------------envoy 0.56c sidecar 0.50cwrk hottest 0.23c mean 0.07c------------------------------mean request 3.00 ms before 0.95 envoy 0.66 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1172 /sachieved 1172 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 13.0 pool 45.0------------------------------envoy 0.67c sidecar 0.51cwrk hottest 0.26c mean 0.08c------------------------------mean request 2.99 ms before 0.95 envoy 0.69 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.7 msin-flight 9.0 pool 45.0------------------------------envoy 1.17c sidecar 0.94cwrk hottest 0.43c mean 0.15c------------------------------mean request 2.93 ms before 0.91 envoy 0.80 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 21.0 pool 45.0------------------------------envoy 1.16c sidecar 0.93cwrk hottest 0.47c mean 0.14c------------------------------mean request 2.95 ms before 0.92 envoy 0.78 sidecar 0.79 worker 0.46 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2207 /sachieved 2207 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 14.0 pool 45.0------------------------------envoy 1.28c sidecar 0.93cwrk hottest 0.42c mean 0.15c------------------------------mean request 2.96 ms before 0.94 envoy 0.79 sidecar 0.81 worker 0.43 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 21.0 pool 45.0------------------------------envoy 1.82c sidecar 1.25cwrk hottest 0.52c mean 0.23c------------------------------mean request 3.01 ms before 0.95 envoy 0.74 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 32.0 pool 45.0------------------------------envoy 1.83c sidecar 0.00cwrk hottest 0.53c mean 0.23c------------------------------mean request 3.04 ms before 0.96 envoy 0.69 sidecar 0.81 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 25.0 pool 32.0------------------------------envoy 1.79c sidecar 1.35cwrk hottest 0.49c mean 0.23c------------------------------mean request 3.07 ms before 0.98 envoy 0.69 sidecar 0.81 worker 0.59 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3907 /sachieved 3906 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 536 pool 536------------------------------envoy 2.32c sidecar 1.43cwrk hottest 0.76c mean 0.30c------------------------------mean request 4.15 ms before 1.14 envoy 0.87 sidecar 1.20 worker 0.94 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 276 pool 536------------------------------envoy 2.36c sidecar 1.65cwrk hottest 0.81c mean 0.29c------------------------------mean request 3.58 ms before 1.14 envoy 0.68 sidecar 0.97 worker 0.78 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4088 /sachieved 4088 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 63.0 pool 536------------------------------envoy 2.42c sidecar 1.62cwrk hottest 0.76c mean 0.30c------------------------------mean request 3.49 ms before 1.21 envoy 0.72 sidecar 0.82 worker 0.74 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 3.5 ms p95 16.7 msin-flight 65.0 pool 536------------------------------envoy 2.95c sidecar 1.70cwrk hottest 0.87c mean 0.37c------------------------------mean request 6.06 ms before 2.52 envoy 1.12 sidecar 0.83 worker 1.59 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.5 ms p95 17.2 msin-flight 75.0 pool 536------------------------------envoy 2.89c sidecar 1.90cwrk hottest 0.88c mean 0.36c------------------------------mean request 6.08 ms before 2.28 envoy 1.23 sidecar 0.83 worker 1.74 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5253 /sachieved 5250 /s (100.0% ok)p50 3.6 ms p95 21.2 msin-flight 85.0 pool 536------------------------------envoy 3.06c sidecar 1.95cwrk hottest 0.88c mean 0.40c------------------------------mean request 6.79 ms before 2.54 envoy 1.36 sidecar 0.83 worker 2.05 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 4.0 ms p95 24.7 msin-flight 90.0 pool 536------------------------------envoy 3.68c sidecar 0.00cwrk hottest 0.82c mean 0.46c------------------------------mean request 7.38 ms before 2.73 envoy 1.49 sidecar 0.85 worker 2.31 resume 0.81 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.7 ms p95 11.0 msin-flight 79.0 pool 536------------------------------envoy 3.65c sidecar 2.22cwrk hottest 0.80c mean 0.45c------------------------------mean request 4.78 ms before 1.58 envoy 0.98 sidecar 0.82 worker 1.39 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.9 ms p95 19.9 msin-flight 116 pool 275------------------------------envoy 3.65c sidecar 2.21cwrk hottest 0.82c mean 0.46c------------------------------mean request 6.63 ms before 2.55 envoy 1.29 sidecar 0.85 worker 1.94 resume 0.81 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6940 /sachieved 6940 /s (100.0% ok)p50 6.8 ms p95 22.5 msin-flight 642 pool 641------------------------------envoy 4.15c sidecar 2.26cwrk hottest 0.76c mean 0.52c------------------------------mean request 9.42 ms before 3.49 envoy 1.99 sidecar 0.90 worker 3.04 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (99.2% ok)p50 5.5 ms p95 77.5 msin-flight 1865 pool 1864------------------------------envoy 4.39c sidecar 2.42cwrk hottest 0.74c mean 0.55c------------------------------mean request 16.86 ms before 5.20 envoy 2.93 sidecar 3.47 worker 5.26 resume 3.46 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7150 /sachieved 7122 /s (99.6% ok)p50 7.0 ms p95 81.7 msin-flight 1822 pool 1864------------------------------envoy 4.39c sidecar 2.48cwrk hottest 0.70c mean 0.55c------------------------------mean request 18.59 ms before 7.73 envoy 3.00 sidecar 2.77 worker 5.10 resume 2.74 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 7604 /s (100.0% ok)p50 22.1 ms p95 312.3 msin-flight 13k pool 13k------------------------------envoy 4.77c sidecar 2.50cwrk hottest 0.67c mean 0.58c------------------------------mean request 52.42 ms before 12.00 envoy 14.64 sidecar 1.83 worker 23.94 resume 1.78 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 6543 /s (100.0% ok)p50 2758.7 ms p95 6534.9 msin-flight 34k pool 34k------------------------------envoy 4.72c sidecar 1.57cwrk hottest 0.63c mean 0.61c------------------------------mean request 2969.05 ms before 727.10 envoy 1393.75 sidecar 3.89 worker 844.31 resume 3.45 (control plane) +rung -1 0 qps setpoint------------------------------offered 296 /sachieved 296 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 45.0 pool 45.0------------------------------envoy 0.22c sidecar 0.03cwrk hottest 0.22c mean 0.08c------------------------------mean request 3.84 ms before 1.09 envoy 0.58 sidecar 0.79 worker 1.38 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 45.0------------------------------envoy 0.60c sidecar 0.00cwrk hottest 0.26c mean 0.07c------------------------------mean request 3.06 ms before 0.95 envoy 0.63 sidecar 0.79 worker 0.69 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.6 msin-flight 21.0 pool 45.0------------------------------envoy 0.56c sidecar 0.50cwrk hottest 0.23c mean 0.07c------------------------------mean request 3.00 ms before 0.95 envoy 0.66 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1172 /sachieved 1172 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 13.0 pool 45.0------------------------------envoy 0.67c sidecar 0.51cwrk hottest 0.26c mean 0.08c------------------------------mean request 2.99 ms before 0.95 envoy 0.69 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.7 msin-flight 9.0 pool 45.0------------------------------envoy 1.17c sidecar 0.94cwrk hottest 0.43c mean 0.15c------------------------------mean request 2.93 ms before 0.91 envoy 0.80 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 21.0 pool 45.0------------------------------envoy 1.16c sidecar 0.93cwrk hottest 0.47c mean 0.14c------------------------------mean request 2.95 ms before 0.92 envoy 0.78 sidecar 0.79 worker 0.46 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2207 /sachieved 2207 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 14.0 pool 45.0------------------------------envoy 1.28c sidecar 0.93cwrk hottest 0.42c mean 0.15c------------------------------mean request 2.96 ms before 0.94 envoy 0.79 sidecar 0.81 worker 0.43 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 21.0 pool 45.0------------------------------envoy 1.82c sidecar 1.25cwrk hottest 0.52c mean 0.23c------------------------------mean request 3.01 ms before 0.95 envoy 0.74 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 32.0 pool 45.0------------------------------envoy 1.83c sidecar 0.00cwrk hottest 0.53c mean 0.23c------------------------------mean request 3.04 ms before 0.96 envoy 0.69 sidecar 0.81 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 25.0 pool 32.0------------------------------envoy 1.79c sidecar 1.35cwrk hottest 0.49c mean 0.23c------------------------------mean request 3.07 ms before 0.98 envoy 0.69 sidecar 0.81 worker 0.59 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3907 /sachieved 3906 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 536 pool 536------------------------------envoy 2.32c sidecar 1.43cwrk hottest 0.76c mean 0.30c------------------------------mean request 4.15 ms before 1.14 envoy 0.87 sidecar 1.20 worker 0.94 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 276 pool 536------------------------------envoy 2.36c sidecar 1.65cwrk hottest 0.81c mean 0.29c------------------------------mean request 3.58 ms before 1.14 envoy 0.68 sidecar 0.97 worker 0.78 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4088 /sachieved 4088 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 63.0 pool 536------------------------------envoy 2.42c sidecar 1.62cwrk hottest 0.76c mean 0.30c------------------------------mean request 3.49 ms before 1.21 envoy 0.72 sidecar 0.82 worker 0.74 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 3.5 ms p95 16.7 msin-flight 65.0 pool 536------------------------------envoy 2.95c sidecar 1.70cwrk hottest 0.87c mean 0.37c------------------------------mean request 6.06 ms before 2.52 envoy 1.12 sidecar 0.83 worker 1.59 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.5 ms p95 17.2 msin-flight 75.0 pool 536------------------------------envoy 2.89c sidecar 1.90cwrk hottest 0.88c mean 0.36c------------------------------mean request 6.08 ms before 2.28 envoy 1.23 sidecar 0.83 worker 1.74 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5253 /sachieved 5250 /s (100.0% ok)p50 3.6 ms p95 21.2 msin-flight 85.0 pool 536------------------------------envoy 3.06c sidecar 1.95cwrk hottest 0.88c mean 0.40c------------------------------mean request 6.79 ms before 2.54 envoy 1.36 sidecar 0.83 worker 2.05 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 4.0 ms p95 24.7 msin-flight 90.0 pool 536------------------------------envoy 3.68c sidecar 0.00cwrk hottest 0.82c mean 0.46c------------------------------mean request 7.38 ms before 2.73 envoy 1.49 sidecar 0.85 worker 2.31 resume 0.81 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.7 ms p95 11.0 msin-flight 79.0 pool 536------------------------------envoy 3.65c sidecar 2.22cwrk hottest 0.80c mean 0.45c------------------------------mean request 4.78 ms before 1.58 envoy 0.98 sidecar 0.82 worker 1.39 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.9 ms p95 19.9 msin-flight 116 pool 275------------------------------envoy 3.65c sidecar 2.21cwrk hottest 0.82c mean 0.46c------------------------------mean request 6.63 ms before 2.55 envoy 1.29 sidecar 0.85 worker 1.94 resume 0.81 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6940 /sachieved 6940 /s (100.0% ok)p50 6.8 ms p95 22.5 msin-flight 642 pool 641------------------------------envoy 4.15c sidecar 2.26cwrk hottest 0.76c mean 0.52c------------------------------mean request 9.42 ms before 3.49 envoy 1.99 sidecar 0.90 worker 3.04 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (99.2% ok)p50 5.5 ms p95 77.5 msin-flight 1865 pool 1864------------------------------envoy 4.39c sidecar 2.42cwrk hottest 0.74c mean 0.55c------------------------------mean request 16.86 ms before 5.20 envoy 2.93 sidecar 3.47 worker 5.26 resume 3.46 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7150 /sachieved 7122 /s (99.6% ok)p50 7.0 ms p95 81.7 msin-flight 1822 pool 1864------------------------------envoy 4.39c sidecar 2.48cwrk hottest 0.70c mean 0.55c------------------------------mean request 18.59 ms before 7.73 envoy 3.00 sidecar 2.77 worker 5.10 resume 2.74 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 7604 /s (100.0% ok)p50 22.1 ms p95 312.3 msin-flight 13k pool 13k------------------------------envoy 4.77c sidecar 2.50cwrk hottest 0.67c mean 0.58c------------------------------mean request 52.42 ms before 12.00 envoy 14.64 sidecar 1.83 worker 23.94 resume 1.78 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 6543 /s (100.0% ok)p50 2758.7 ms p95 6534.9 msin-flight 34k pool 34k------------------------------envoy 4.72c sidecar 1.57cwrk hottest 0.63c mean 0.61c------------------------------mean request 2969.05 ms before 727.10 envoy 1393.75 sidecar 3.89 worker 844.31 resume 3.45 (control plane) +rung -1 0 qps setpoint------------------------------offered 296 /sachieved 296 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 45.0 pool 45.0------------------------------envoy 0.22c sidecar 0.03cwrk hottest 0.22c mean 0.08c------------------------------mean request 3.84 ms before 1.09 envoy 0.58 sidecar 0.79 worker 1.38 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 45.0------------------------------envoy 0.60c sidecar 0.00cwrk hottest 0.26c mean 0.07c------------------------------mean request 3.06 ms before 0.95 envoy 0.63 sidecar 0.79 worker 0.69 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.6 msin-flight 21.0 pool 45.0------------------------------envoy 0.56c sidecar 0.50cwrk hottest 0.23c mean 0.07c------------------------------mean request 3.00 ms before 0.95 envoy 0.66 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1172 /sachieved 1172 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 13.0 pool 45.0------------------------------envoy 0.67c sidecar 0.51cwrk hottest 0.26c mean 0.08c------------------------------mean request 2.99 ms before 0.95 envoy 0.69 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.7 msin-flight 9.0 pool 45.0------------------------------envoy 1.17c sidecar 0.94cwrk hottest 0.43c mean 0.15c------------------------------mean request 2.93 ms before 0.91 envoy 0.80 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 21.0 pool 45.0------------------------------envoy 1.16c sidecar 0.93cwrk hottest 0.47c mean 0.14c------------------------------mean request 2.95 ms before 0.92 envoy 0.78 sidecar 0.79 worker 0.46 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2207 /sachieved 2207 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 14.0 pool 45.0------------------------------envoy 1.28c sidecar 0.93cwrk hottest 0.42c mean 0.15c------------------------------mean request 2.96 ms before 0.94 envoy 0.79 sidecar 0.81 worker 0.43 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 21.0 pool 45.0------------------------------envoy 1.82c sidecar 1.25cwrk hottest 0.52c mean 0.23c------------------------------mean request 3.01 ms before 0.95 envoy 0.74 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 32.0 pool 45.0------------------------------envoy 1.83c sidecar 0.00cwrk hottest 0.53c mean 0.23c------------------------------mean request 3.04 ms before 0.96 envoy 0.69 sidecar 0.81 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 25.0 pool 32.0------------------------------envoy 1.79c sidecar 1.35cwrk hottest 0.49c mean 0.23c------------------------------mean request 3.07 ms before 0.98 envoy 0.69 sidecar 0.81 worker 0.59 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3907 /sachieved 3906 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 536 pool 536------------------------------envoy 2.32c sidecar 1.43cwrk hottest 0.76c mean 0.30c------------------------------mean request 4.15 ms before 1.14 envoy 0.87 sidecar 1.20 worker 0.94 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 276 pool 536------------------------------envoy 2.36c sidecar 1.65cwrk hottest 0.81c mean 0.29c------------------------------mean request 3.58 ms before 1.14 envoy 0.68 sidecar 0.97 worker 0.78 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4088 /sachieved 4088 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 63.0 pool 536------------------------------envoy 2.42c sidecar 1.62cwrk hottest 0.76c mean 0.30c------------------------------mean request 3.49 ms before 1.21 envoy 0.72 sidecar 0.82 worker 0.74 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 3.5 ms p95 16.7 msin-flight 65.0 pool 536------------------------------envoy 2.95c sidecar 1.70cwrk hottest 0.87c mean 0.37c------------------------------mean request 6.06 ms before 2.52 envoy 1.12 sidecar 0.83 worker 1.59 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.5 ms p95 17.2 msin-flight 75.0 pool 536------------------------------envoy 2.89c sidecar 1.90cwrk hottest 0.88c mean 0.36c------------------------------mean request 6.08 ms before 2.28 envoy 1.23 sidecar 0.83 worker 1.74 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5253 /sachieved 5250 /s (100.0% ok)p50 3.6 ms p95 21.2 msin-flight 85.0 pool 536------------------------------envoy 3.06c sidecar 1.95cwrk hottest 0.88c mean 0.40c------------------------------mean request 6.79 ms before 2.54 envoy 1.36 sidecar 0.83 worker 2.05 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 4.0 ms p95 24.7 msin-flight 90.0 pool 536------------------------------envoy 3.68c sidecar 0.00cwrk hottest 0.82c mean 0.46c------------------------------mean request 7.38 ms before 2.73 envoy 1.49 sidecar 0.85 worker 2.31 resume 0.81 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.7 ms p95 11.0 msin-flight 79.0 pool 536------------------------------envoy 3.65c sidecar 2.22cwrk hottest 0.80c mean 0.45c------------------------------mean request 4.78 ms before 1.58 envoy 0.98 sidecar 0.82 worker 1.39 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.9 ms p95 19.9 msin-flight 116 pool 275------------------------------envoy 3.65c sidecar 2.21cwrk hottest 0.82c mean 0.46c------------------------------mean request 6.63 ms before 2.55 envoy 1.29 sidecar 0.85 worker 1.94 resume 0.81 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6940 /sachieved 6940 /s (100.0% ok)p50 6.8 ms p95 22.5 msin-flight 642 pool 641------------------------------envoy 4.15c sidecar 2.26cwrk hottest 0.76c mean 0.52c------------------------------mean request 9.42 ms before 3.49 envoy 1.99 sidecar 0.90 worker 3.04 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (99.2% ok)p50 5.5 ms p95 77.5 msin-flight 1865 pool 1864------------------------------envoy 4.39c sidecar 2.42cwrk hottest 0.74c mean 0.55c------------------------------mean request 16.86 ms before 5.20 envoy 2.93 sidecar 3.47 worker 5.26 resume 3.46 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7150 /sachieved 7122 /s (99.6% ok)p50 7.0 ms p95 81.7 msin-flight 1822 pool 1864------------------------------envoy 4.39c sidecar 2.48cwrk hottest 0.70c mean 0.55c------------------------------mean request 18.59 ms before 7.73 envoy 3.00 sidecar 2.77 worker 5.10 resume 2.74 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 7604 /s (100.0% ok)p50 22.1 ms p95 312.3 msin-flight 13k pool 13k------------------------------envoy 4.77c sidecar 2.50cwrk hottest 0.67c mean 0.58c------------------------------mean request 52.42 ms before 12.00 envoy 14.64 sidecar 1.83 worker 23.94 resume 1.78 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 6543 /s (100.0% ok)p50 2758.7 ms p95 6534.9 msin-flight 34k pool 34k------------------------------envoy 4.72c sidecar 1.57cwrk hottest 0.63c mean 0.61c------------------------------mean request 2969.05 ms before 727.10 envoy 1393.75 sidecar 3.89 worker 844.31 resume 3.45 (control plane) +rung -1 0 qps setpoint------------------------------offered 296 /sachieved 296 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 45.0 pool 45.0------------------------------envoy 0.22c sidecar 0.03cwrk hottest 0.22c mean 0.08c------------------------------mean request 3.84 ms before 1.09 envoy 0.58 sidecar 0.79 worker 1.38 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 45.0------------------------------envoy 0.60c sidecar 0.00cwrk hottest 0.26c mean 0.07c------------------------------mean request 3.06 ms before 0.95 envoy 0.63 sidecar 0.79 worker 0.69 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.6 msin-flight 21.0 pool 45.0------------------------------envoy 0.56c sidecar 0.50cwrk hottest 0.23c mean 0.07c------------------------------mean request 3.00 ms before 0.95 envoy 0.66 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1172 /sachieved 1172 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 13.0 pool 45.0------------------------------envoy 0.67c sidecar 0.51cwrk hottest 0.26c mean 0.08c------------------------------mean request 2.99 ms before 0.95 envoy 0.69 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.7 msin-flight 9.0 pool 45.0------------------------------envoy 1.17c sidecar 0.94cwrk hottest 0.43c mean 0.15c------------------------------mean request 2.93 ms before 0.91 envoy 0.80 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 21.0 pool 45.0------------------------------envoy 1.16c sidecar 0.93cwrk hottest 0.47c mean 0.14c------------------------------mean request 2.95 ms before 0.92 envoy 0.78 sidecar 0.79 worker 0.46 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2207 /sachieved 2207 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 14.0 pool 45.0------------------------------envoy 1.28c sidecar 0.93cwrk hottest 0.42c mean 0.15c------------------------------mean request 2.96 ms before 0.94 envoy 0.79 sidecar 0.81 worker 0.43 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 21.0 pool 45.0------------------------------envoy 1.82c sidecar 1.25cwrk hottest 0.52c mean 0.23c------------------------------mean request 3.01 ms before 0.95 envoy 0.74 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 32.0 pool 45.0------------------------------envoy 1.83c sidecar 0.00cwrk hottest 0.53c mean 0.23c------------------------------mean request 3.04 ms before 0.96 envoy 0.69 sidecar 0.81 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 25.0 pool 32.0------------------------------envoy 1.79c sidecar 1.35cwrk hottest 0.49c mean 0.23c------------------------------mean request 3.07 ms before 0.98 envoy 0.69 sidecar 0.81 worker 0.59 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3907 /sachieved 3906 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 536 pool 536------------------------------envoy 2.32c sidecar 1.43cwrk hottest 0.76c mean 0.30c------------------------------mean request 4.15 ms before 1.14 envoy 0.87 sidecar 1.20 worker 0.94 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 276 pool 536------------------------------envoy 2.36c sidecar 1.65cwrk hottest 0.81c mean 0.29c------------------------------mean request 3.58 ms before 1.14 envoy 0.68 sidecar 0.97 worker 0.78 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4088 /sachieved 4088 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 63.0 pool 536------------------------------envoy 2.42c sidecar 1.62cwrk hottest 0.76c mean 0.30c------------------------------mean request 3.49 ms before 1.21 envoy 0.72 sidecar 0.82 worker 0.74 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 3.5 ms p95 16.7 msin-flight 65.0 pool 536------------------------------envoy 2.95c sidecar 1.70cwrk hottest 0.87c mean 0.37c------------------------------mean request 6.06 ms before 2.52 envoy 1.12 sidecar 0.83 worker 1.59 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.5 ms p95 17.2 msin-flight 75.0 pool 536------------------------------envoy 2.89c sidecar 1.90cwrk hottest 0.88c mean 0.36c------------------------------mean request 6.08 ms before 2.28 envoy 1.23 sidecar 0.83 worker 1.74 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5253 /sachieved 5250 /s (100.0% ok)p50 3.6 ms p95 21.2 msin-flight 85.0 pool 536------------------------------envoy 3.06c sidecar 1.95cwrk hottest 0.88c mean 0.40c------------------------------mean request 6.79 ms before 2.54 envoy 1.36 sidecar 0.83 worker 2.05 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 4.0 ms p95 24.7 msin-flight 90.0 pool 536------------------------------envoy 3.68c sidecar 0.00cwrk hottest 0.82c mean 0.46c------------------------------mean request 7.38 ms before 2.73 envoy 1.49 sidecar 0.85 worker 2.31 resume 0.81 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.7 ms p95 11.0 msin-flight 79.0 pool 536------------------------------envoy 3.65c sidecar 2.22cwrk hottest 0.80c mean 0.45c------------------------------mean request 4.78 ms before 1.58 envoy 0.98 sidecar 0.82 worker 1.39 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.9 ms p95 19.9 msin-flight 116 pool 275------------------------------envoy 3.65c sidecar 2.21cwrk hottest 0.82c mean 0.46c------------------------------mean request 6.63 ms before 2.55 envoy 1.29 sidecar 0.85 worker 1.94 resume 0.81 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6940 /sachieved 6940 /s (100.0% ok)p50 6.8 ms p95 22.5 msin-flight 642 pool 641------------------------------envoy 4.15c sidecar 2.26cwrk hottest 0.76c mean 0.52c------------------------------mean request 9.42 ms before 3.49 envoy 1.99 sidecar 0.90 worker 3.04 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (99.2% ok)p50 5.5 ms p95 77.5 msin-flight 1865 pool 1864------------------------------envoy 4.39c sidecar 2.42cwrk hottest 0.74c mean 0.55c------------------------------mean request 16.86 ms before 5.20 envoy 2.93 sidecar 3.47 worker 5.26 resume 3.46 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7150 /sachieved 7122 /s (99.6% ok)p50 7.0 ms p95 81.7 msin-flight 1822 pool 1864------------------------------envoy 4.39c sidecar 2.48cwrk hottest 0.70c mean 0.55c------------------------------mean request 18.59 ms before 7.73 envoy 3.00 sidecar 2.77 worker 5.10 resume 2.74 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 7604 /s (100.0% ok)p50 22.1 ms p95 312.3 msin-flight 13k pool 13k------------------------------envoy 4.77c sidecar 2.50cwrk hottest 0.67c mean 0.58c------------------------------mean request 52.42 ms before 12.00 envoy 14.64 sidecar 1.83 worker 23.94 resume 1.78 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 6543 /s (100.0% ok)p50 2758.7 ms p95 6534.9 msin-flight 34k pool 34k------------------------------envoy 4.72c sidecar 1.57cwrk hottest 0.63c mean 0.61c------------------------------mean request 2969.05 ms before 727.10 envoy 1393.75 sidecar 3.89 worker 844.31 resume 3.45 (control plane) +rung -1 0 qps setpoint------------------------------offered 296 /sachieved 296 /s (100.0% ok)p50 3.1 ms p95 6.4 msin-flight 45.0 pool 45.0------------------------------envoy 0.22c sidecar 0.03cwrk hottest 0.22c mean 0.08c------------------------------mean request 3.84 ms before 1.09 envoy 0.58 sidecar 0.79 worker 1.38 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 5.0 pool 45.0------------------------------envoy 0.60c sidecar 0.00cwrk hottest 0.26c mean 0.07c------------------------------mean request 3.06 ms before 0.95 envoy 0.63 sidecar 0.79 worker 0.69 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.6 msin-flight 21.0 pool 45.0------------------------------envoy 0.56c sidecar 0.50cwrk hottest 0.23c mean 0.07c------------------------------mean request 3.00 ms before 0.95 envoy 0.66 sidecar 0.80 worker 0.60 resume 0.76 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1172 /sachieved 1172 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 13.0 pool 45.0------------------------------envoy 0.67c sidecar 0.51cwrk hottest 0.26c mean 0.08c------------------------------mean request 2.99 ms before 0.95 envoy 0.69 sidecar 0.80 worker 0.56 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.8 ms p95 3.7 msin-flight 9.0 pool 45.0------------------------------envoy 1.17c sidecar 0.94cwrk hottest 0.43c mean 0.15c------------------------------mean request 2.93 ms before 0.91 envoy 0.80 sidecar 0.79 worker 0.43 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 21.0 pool 45.0------------------------------envoy 1.16c sidecar 0.93cwrk hottest 0.47c mean 0.14c------------------------------mean request 2.95 ms before 0.92 envoy 0.78 sidecar 0.79 worker 0.46 resume 0.75 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2207 /sachieved 2207 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 14.0 pool 45.0------------------------------envoy 1.28c sidecar 0.93cwrk hottest 0.42c mean 0.15c------------------------------mean request 2.96 ms before 0.94 envoy 0.79 sidecar 0.81 worker 0.43 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 21.0 pool 45.0------------------------------envoy 1.82c sidecar 1.25cwrk hottest 0.52c mean 0.23c------------------------------mean request 3.01 ms before 0.95 envoy 0.74 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 2.9 ms p95 4.0 msin-flight 32.0 pool 45.0------------------------------envoy 1.83c sidecar 0.00cwrk hottest 0.53c mean 0.23c------------------------------mean request 3.04 ms before 0.96 envoy 0.69 sidecar 0.81 worker 0.58 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.0 ms p95 4.0 msin-flight 25.0 pool 32.0------------------------------envoy 1.79c sidecar 1.35cwrk hottest 0.49c mean 0.23c------------------------------mean request 3.07 ms before 0.98 envoy 0.69 sidecar 0.81 worker 0.59 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3907 /sachieved 3906 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 536 pool 536------------------------------envoy 2.32c sidecar 1.43cwrk hottest 0.76c mean 0.30c------------------------------mean request 4.15 ms before 1.14 envoy 0.87 sidecar 1.20 worker 0.94 resume 0.79 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.1 ms p95 4.7 msin-flight 276 pool 536------------------------------envoy 2.36c sidecar 1.65cwrk hottest 0.81c mean 0.29c------------------------------mean request 3.58 ms before 1.14 envoy 0.68 sidecar 0.97 worker 0.78 resume 0.93 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4088 /sachieved 4088 /s (100.0% ok)p50 3.2 ms p95 5.6 msin-flight 63.0 pool 536------------------------------envoy 2.42c sidecar 1.62cwrk hottest 0.76c mean 0.30c------------------------------mean request 3.49 ms before 1.21 envoy 0.72 sidecar 0.82 worker 0.74 resume 0.78 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 5000 /sachieved 4999 /s (100.0% ok)p50 3.5 ms p95 16.7 msin-flight 65.0 pool 536------------------------------envoy 2.95c sidecar 1.70cwrk hottest 0.87c mean 0.37c------------------------------mean request 6.06 ms before 2.52 envoy 1.12 sidecar 0.83 worker 1.59 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.5 ms p95 17.2 msin-flight 75.0 pool 536------------------------------envoy 2.89c sidecar 1.90cwrk hottest 0.88c mean 0.36c------------------------------mean request 6.08 ms before 2.28 envoy 1.23 sidecar 0.83 worker 1.74 resume 0.79 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5253 /sachieved 5250 /s (100.0% ok)p50 3.6 ms p95 21.2 msin-flight 85.0 pool 536------------------------------envoy 3.06c sidecar 1.95cwrk hottest 0.88c mean 0.40c------------------------------mean request 6.79 ms before 2.54 envoy 1.36 sidecar 0.83 worker 2.05 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 6000 /sachieved 6002 /s (100.0% ok)p50 4.0 ms p95 24.7 msin-flight 90.0 pool 536------------------------------envoy 3.68c sidecar 0.00cwrk hottest 0.82c mean 0.46c------------------------------mean request 7.38 ms before 2.73 envoy 1.49 sidecar 0.85 worker 2.31 resume 0.81 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6001 /s (100.0% ok)p50 3.7 ms p95 11.0 msin-flight 79.0 pool 536------------------------------envoy 3.65c sidecar 2.22cwrk hottest 0.80c mean 0.45c------------------------------mean request 4.78 ms before 1.58 envoy 0.98 sidecar 0.82 worker 1.39 resume 0.78 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 3.9 ms p95 19.9 msin-flight 116 pool 275------------------------------envoy 3.65c sidecar 2.21cwrk hottest 0.82c mean 0.46c------------------------------mean request 6.63 ms before 2.55 envoy 1.29 sidecar 0.85 worker 1.94 resume 0.81 (control plane) +rung 6 7000 qps setpoint [warmup]------------------------------offered 6940 /sachieved 6940 /s (100.0% ok)p50 6.8 ms p95 22.5 msin-flight 642 pool 641------------------------------envoy 4.15c sidecar 2.26cwrk hottest 0.76c mean 0.52c------------------------------mean request 9.42 ms before 3.49 envoy 1.99 sidecar 0.90 worker 3.04 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (99.2% ok)p50 5.5 ms p95 77.5 msin-flight 1865 pool 1864------------------------------envoy 4.39c sidecar 2.42cwrk hottest 0.74c mean 0.55c------------------------------mean request 16.86 ms before 5.20 envoy 2.93 sidecar 3.47 worker 5.26 resume 3.46 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7150 /sachieved 7122 /s (99.6% ok)p50 7.0 ms p95 81.7 msin-flight 1822 pool 1864------------------------------envoy 4.39c sidecar 2.48cwrk hottest 0.70c mean 0.55c------------------------------mean request 18.59 ms before 7.73 envoy 3.00 sidecar 2.77 worker 5.10 resume 2.74 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 8000 /sachieved 7604 /s (100.0% ok)p50 22.1 ms p95 312.3 msin-flight 13k pool 13k------------------------------envoy 4.77c sidecar 2.50cwrk hottest 0.67c mean 0.58c------------------------------mean request 52.42 ms before 12.00 envoy 14.64 sidecar 1.83 worker 23.94 resume 1.78 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 6543 /s (100.0% ok)p50 2758.7 ms p95 6534.9 msin-flight 34k pool 34k------------------------------envoy 4.72c sidecar 1.57cwrk hottest 0.63c mean 0.61c------------------------------mean request 2969.05 ms before 727.10 envoy 1393.75 sidecar 3.89 worker 844.31 resume 3.45 (control plane) +
+ + + +atenet-router capacity — 16c +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 +R9 + +0 + +10k + +20k + +30k +requests / s + + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + + +0 + +5.0 + +10.0 + +15.0 +cores + + + + + + + +envoy CPU limit, 16 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + + +0 + +500.0M + +1.0G + +1.5G +bytes + +100 + +200 + +300 + +400 + +500 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung 0 1000 qps setpoint [warmup]------------------------------offered 890 /sachieved 890 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 35.0 pool 34.0------------------------------envoy 0.62c sidecar 0.12cwrk hottest 0.27c mean 0.04c------------------------------mean request 3.41 ms before 1.11 envoy 0.58 sidecar 0.82 worker 0.90 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.60c sidecar 0.52cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.10 ms before 1.02 envoy 0.56 sidecar 0.79 worker 0.73 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 29.0 pool 34.0------------------------------envoy 0.60c sidecar 0.51cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.18 ms before 1.06 envoy 0.60 sidecar 0.79 worker 0.72 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1434 /sachieved 1434 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 393 pool 393------------------------------envoy 0.90c sidecar 0.52cwrk hottest 0.37c mean 0.05c------------------------------mean request 5.50 ms before 2.56 envoy 0.91 sidecar 1.17 worker 0.86 resume 1.12 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 79.0 pool 393------------------------------envoy 1.25c sidecar 0.75cwrk hottest 0.34c mean 0.08c------------------------------mean request 3.09 ms before 0.95 envoy 0.82 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 393------------------------------envoy 1.21c sidecar 0.93cwrk hottest 0.43c mean 0.08c------------------------------mean request 3.04 ms before 0.97 envoy 0.72 sidecar 0.81 worker 0.54 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2240 /sachieved 2240 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 393------------------------------envoy 1.33c sidecar 0.93cwrk hottest 0.58c mean 0.08c------------------------------mean request 3.03 ms before 0.97 envoy 0.71 sidecar 0.81 worker 0.54 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 1041 pool 1047------------------------------envoy 1.86c sidecar 1.25cwrk hottest 0.73c mean 0.12c------------------------------mean request 8.11 ms before 3.20 envoy 1.64 sidecar 2.13 worker 1.15 resume 2.09 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.6 msin-flight 26.0 pool 1047------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.77c mean 0.11c------------------------------mean request 3.24 ms before 1.09 envoy 0.71 sidecar 0.81 worker 0.64 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3520 /sachieved 3519 /s (100.0% ok)p50 3.4 ms p95 6.7 msin-flight 299 pool 1047------------------------------envoy 2.05c sidecar 1.30cwrk hottest 0.86c mean 0.13c------------------------------mean request 4.32 ms before 1.29 envoy 0.86 sidecar 1.19 worker 0.99 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.9 ms p95 213.9 msin-flight 1594 pool 1591------------------------------envoy 2.54c sidecar 1.54cwrk hottest 0.82c mean 0.16c------------------------------mean request 26.47 ms before 13.94 envoy 5.21 sidecar 1.02 worker 6.31 resume 0.96 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.7 ms p95 11.2 msin-flight 426 pool 1591------------------------------envoy 2.23c sidecar 1.55cwrk hottest 0.91c mean 0.14c------------------------------mean request 5.77 ms before 2.31 envoy 1.18 sidecar 0.84 worker 1.44 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4321 /sachieved 4316 /s (100.0% ok)p50 4.0 ms p95 30.2 msin-flight 127 pool 1591------------------------------envoy 2.38c sidecar 1.56cwrk hottest 0.90c mean 0.15c------------------------------mean request 8.30 ms before 3.86 envoy 1.41 sidecar 0.85 worker 2.19 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5006 /s (100.0% ok)p50 6.3 ms p95 26.0 msin-flight 147 pool 1591------------------------------envoy 2.77c sidecar 0.00cwrk hottest 0.89c mean 0.17c------------------------------mean request 8.77 ms before 2.39 envoy 2.11 sidecar 0.84 worker 3.43 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 5.8 ms p95 22.9 msin-flight 194 pool 1591------------------------------envoy 2.78c sidecar 1.75cwrk hottest 0.88c mean 0.17c------------------------------mean request 8.21 ms before 3.32 envoy 1.58 sidecar 0.85 worker 2.45 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 4.2 ms p95 14.1 msin-flight 70.0 pool 1591------------------------------envoy 2.93c sidecar 1.78cwrk hottest 0.88c mean 0.18c------------------------------mean request 5.66 ms before 2.08 envoy 1.16 sidecar 0.83 worker 1.60 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5648 /sachieved 5645 /s (100.0% ok)p50 6.7 ms p95 32.6 msin-flight 209 pool 1591------------------------------envoy 3.17c sidecar 1.88cwrk hottest 0.86c mean 0.20c------------------------------mean request 12.08 ms before 5.38 envoy 2.22 sidecar 0.90 worker 3.60 resume 0.85 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5919 /s (100.0% ok)p50 15.5 ms p95 45.0 msin-flight 3112 pool 3113------------------------------envoy 3.37c sidecar 1.94cwrk hottest 0.83c mean 0.21c------------------------------mean request 18.03 ms before 7.86 envoy 3.20 sidecar 1.13 worker 5.84 resume 1.07 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6059 /s (100.0% ok)p50 4.8 ms p95 320.1 msin-flight 3095 pool 3113------------------------------envoy 4.14c sidecar 2.10cwrk hottest 0.77c mean 0.26c------------------------------mean request 40.70 ms before 13.77 envoy 10.93 sidecar 0.92 worker 15.08 resume 0.87 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6387 /sachieved 6378 /s (100.0% ok)p50 5.3 ms p95 49.5 msin-flight 415 pool 3113------------------------------envoy 4.06c sidecar 2.19cwrk hottest 0.80c mean 0.25c------------------------------mean request 12.90 ms before 6.48 envoy 2.13 sidecar 0.89 worker 3.41 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7006 /s (100.0% ok)p50 5.6 ms p95 51.6 msin-flight 245 pool 3113------------------------------envoy 4.43c sidecar 2.36cwrk hottest 0.77c mean 0.27c------------------------------mean request 17.19 ms before 6.72 envoy 3.59 sidecar 0.92 worker 5.96 resume 0.88 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7003 /s (99.5% ok)p50 5.3 ms p95 41.5 msin-flight 1473 pool 3113------------------------------envoy 4.43c sidecar 2.31cwrk hottest 0.77c mean 0.28c------------------------------mean request 15.96 ms before 3.85 envoy 3.82 sidecar 2.39 worker 5.90 resume 2.36 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7253 /sachieved 7244 /s (100.0% ok)p50 5.7 ms p95 19.3 msin-flight 233 pool 3113------------------------------envoy 4.92c sidecar 2.47cwrk hottest 0.70c mean 0.32c------------------------------mean request 7.64 ms before 2.88 envoy 1.64 sidecar 0.89 worker 2.23 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 8.5 ms p95 36.6 msin-flight 282 pool 3113------------------------------envoy 5.74c sidecar 2.76cwrk hottest 0.65c mean 0.36c------------------------------mean request 12.51 ms before 4.66 envoy 2.85 sidecar 0.93 worker 4.07 resume 0.89 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7996 /s (99.5% ok)p50 8.2 ms p95 70.9 msin-flight 1862 pool 3113------------------------------envoy 5.82c sidecar 2.73cwrk hottest 0.65c mean 0.36c------------------------------mean request 19.86 ms before 5.66 envoy 4.87 sidecar 2.18 worker 7.15 resume 2.15 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8722 /sachieved 8577 /s (100.0% ok)p50 46.9 ms p95 309.8 msin-flight 3448 pool 3448------------------------------envoy 6.87c sidecar 2.73cwrk hottest 0.63c mean 0.44c------------------------------mean request 79.93 ms before 38.83 envoy 15.02 sidecar 1.65 worker 24.43 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9157 /s (100.0% ok)p50 363.0 ms p95 611.0 msin-flight 5175 pool 5175------------------------------envoy 7.30c sidecar 2.57cwrk hottest 0.47c mean 0.45c------------------------------mean request 366.39 ms before 107.80 envoy 101.23 sidecar 2.68 worker 154.69 resume 1.71 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9021 /s (100.0% ok)p50 94.2 ms p95 272.6 msin-flight 2121 pool 5175------------------------------envoy 7.26c sidecar 2.32cwrk hottest 0.49c mean 0.45c------------------------------mean request 114.40 ms before 37.59 envoy 29.66 sidecar 1.19 worker 45.96 resume 1.14 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9524 /sachieved 8261 /s (100.0% ok)p50 225.5 ms p95 1698.8 msin-flight 28k pool 28k------------------------------envoy 7.04c sidecar 2.47cwrk hottest 0.50c mean 0.44c------------------------------mean request 548.22 ms before 194.58 envoy 182.24 sidecar 1.88 worker 169.52 resume 1.81 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 890 /sachieved 890 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 35.0 pool 34.0------------------------------envoy 0.62c sidecar 0.12cwrk hottest 0.27c mean 0.04c------------------------------mean request 3.41 ms before 1.11 envoy 0.58 sidecar 0.82 worker 0.90 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.60c sidecar 0.52cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.10 ms before 1.02 envoy 0.56 sidecar 0.79 worker 0.73 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 29.0 pool 34.0------------------------------envoy 0.60c sidecar 0.51cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.18 ms before 1.06 envoy 0.60 sidecar 0.79 worker 0.72 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1434 /sachieved 1434 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 393 pool 393------------------------------envoy 0.90c sidecar 0.52cwrk hottest 0.37c mean 0.05c------------------------------mean request 5.50 ms before 2.56 envoy 0.91 sidecar 1.17 worker 0.86 resume 1.12 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 79.0 pool 393------------------------------envoy 1.25c sidecar 0.75cwrk hottest 0.34c mean 0.08c------------------------------mean request 3.09 ms before 0.95 envoy 0.82 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 393------------------------------envoy 1.21c sidecar 0.93cwrk hottest 0.43c mean 0.08c------------------------------mean request 3.04 ms before 0.97 envoy 0.72 sidecar 0.81 worker 0.54 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2240 /sachieved 2240 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 393------------------------------envoy 1.33c sidecar 0.93cwrk hottest 0.58c mean 0.08c------------------------------mean request 3.03 ms before 0.97 envoy 0.71 sidecar 0.81 worker 0.54 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 1041 pool 1047------------------------------envoy 1.86c sidecar 1.25cwrk hottest 0.73c mean 0.12c------------------------------mean request 8.11 ms before 3.20 envoy 1.64 sidecar 2.13 worker 1.15 resume 2.09 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.6 msin-flight 26.0 pool 1047------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.77c mean 0.11c------------------------------mean request 3.24 ms before 1.09 envoy 0.71 sidecar 0.81 worker 0.64 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3520 /sachieved 3519 /s (100.0% ok)p50 3.4 ms p95 6.7 msin-flight 299 pool 1047------------------------------envoy 2.05c sidecar 1.30cwrk hottest 0.86c mean 0.13c------------------------------mean request 4.32 ms before 1.29 envoy 0.86 sidecar 1.19 worker 0.99 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.9 ms p95 213.9 msin-flight 1594 pool 1591------------------------------envoy 2.54c sidecar 1.54cwrk hottest 0.82c mean 0.16c------------------------------mean request 26.47 ms before 13.94 envoy 5.21 sidecar 1.02 worker 6.31 resume 0.96 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.7 ms p95 11.2 msin-flight 426 pool 1591------------------------------envoy 2.23c sidecar 1.55cwrk hottest 0.91c mean 0.14c------------------------------mean request 5.77 ms before 2.31 envoy 1.18 sidecar 0.84 worker 1.44 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4321 /sachieved 4316 /s (100.0% ok)p50 4.0 ms p95 30.2 msin-flight 127 pool 1591------------------------------envoy 2.38c sidecar 1.56cwrk hottest 0.90c mean 0.15c------------------------------mean request 8.30 ms before 3.86 envoy 1.41 sidecar 0.85 worker 2.19 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5006 /s (100.0% ok)p50 6.3 ms p95 26.0 msin-flight 147 pool 1591------------------------------envoy 2.77c sidecar 0.00cwrk hottest 0.89c mean 0.17c------------------------------mean request 8.77 ms before 2.39 envoy 2.11 sidecar 0.84 worker 3.43 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 5.8 ms p95 22.9 msin-flight 194 pool 1591------------------------------envoy 2.78c sidecar 1.75cwrk hottest 0.88c mean 0.17c------------------------------mean request 8.21 ms before 3.32 envoy 1.58 sidecar 0.85 worker 2.45 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 4.2 ms p95 14.1 msin-flight 70.0 pool 1591------------------------------envoy 2.93c sidecar 1.78cwrk hottest 0.88c mean 0.18c------------------------------mean request 5.66 ms before 2.08 envoy 1.16 sidecar 0.83 worker 1.60 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5648 /sachieved 5645 /s (100.0% ok)p50 6.7 ms p95 32.6 msin-flight 209 pool 1591------------------------------envoy 3.17c sidecar 1.88cwrk hottest 0.86c mean 0.20c------------------------------mean request 12.08 ms before 5.38 envoy 2.22 sidecar 0.90 worker 3.60 resume 0.85 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5919 /s (100.0% ok)p50 15.5 ms p95 45.0 msin-flight 3112 pool 3113------------------------------envoy 3.37c sidecar 1.94cwrk hottest 0.83c mean 0.21c------------------------------mean request 18.03 ms before 7.86 envoy 3.20 sidecar 1.13 worker 5.84 resume 1.07 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6059 /s (100.0% ok)p50 4.8 ms p95 320.1 msin-flight 3095 pool 3113------------------------------envoy 4.14c sidecar 2.10cwrk hottest 0.77c mean 0.26c------------------------------mean request 40.70 ms before 13.77 envoy 10.93 sidecar 0.92 worker 15.08 resume 0.87 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6387 /sachieved 6378 /s (100.0% ok)p50 5.3 ms p95 49.5 msin-flight 415 pool 3113------------------------------envoy 4.06c sidecar 2.19cwrk hottest 0.80c mean 0.25c------------------------------mean request 12.90 ms before 6.48 envoy 2.13 sidecar 0.89 worker 3.41 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7006 /s (100.0% ok)p50 5.6 ms p95 51.6 msin-flight 245 pool 3113------------------------------envoy 4.43c sidecar 2.36cwrk hottest 0.77c mean 0.27c------------------------------mean request 17.19 ms before 6.72 envoy 3.59 sidecar 0.92 worker 5.96 resume 0.88 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7003 /s (99.5% ok)p50 5.3 ms p95 41.5 msin-flight 1473 pool 3113------------------------------envoy 4.43c sidecar 2.31cwrk hottest 0.77c mean 0.28c------------------------------mean request 15.96 ms before 3.85 envoy 3.82 sidecar 2.39 worker 5.90 resume 2.36 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7253 /sachieved 7244 /s (100.0% ok)p50 5.7 ms p95 19.3 msin-flight 233 pool 3113------------------------------envoy 4.92c sidecar 2.47cwrk hottest 0.70c mean 0.32c------------------------------mean request 7.64 ms before 2.88 envoy 1.64 sidecar 0.89 worker 2.23 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 8.5 ms p95 36.6 msin-flight 282 pool 3113------------------------------envoy 5.74c sidecar 2.76cwrk hottest 0.65c mean 0.36c------------------------------mean request 12.51 ms before 4.66 envoy 2.85 sidecar 0.93 worker 4.07 resume 0.89 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7996 /s (99.5% ok)p50 8.2 ms p95 70.9 msin-flight 1862 pool 3113------------------------------envoy 5.82c sidecar 2.73cwrk hottest 0.65c mean 0.36c------------------------------mean request 19.86 ms before 5.66 envoy 4.87 sidecar 2.18 worker 7.15 resume 2.15 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8722 /sachieved 8577 /s (100.0% ok)p50 46.9 ms p95 309.8 msin-flight 3448 pool 3448------------------------------envoy 6.87c sidecar 2.73cwrk hottest 0.63c mean 0.44c------------------------------mean request 79.93 ms before 38.83 envoy 15.02 sidecar 1.65 worker 24.43 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9157 /s (100.0% ok)p50 363.0 ms p95 611.0 msin-flight 5175 pool 5175------------------------------envoy 7.30c sidecar 2.57cwrk hottest 0.47c mean 0.45c------------------------------mean request 366.39 ms before 107.80 envoy 101.23 sidecar 2.68 worker 154.69 resume 1.71 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9021 /s (100.0% ok)p50 94.2 ms p95 272.6 msin-flight 2121 pool 5175------------------------------envoy 7.26c sidecar 2.32cwrk hottest 0.49c mean 0.45c------------------------------mean request 114.40 ms before 37.59 envoy 29.66 sidecar 1.19 worker 45.96 resume 1.14 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9524 /sachieved 8261 /s (100.0% ok)p50 225.5 ms p95 1698.8 msin-flight 28k pool 28k------------------------------envoy 7.04c sidecar 2.47cwrk hottest 0.50c mean 0.44c------------------------------mean request 548.22 ms before 194.58 envoy 182.24 sidecar 1.88 worker 169.52 resume 1.81 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 890 /sachieved 890 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 35.0 pool 34.0------------------------------envoy 0.62c sidecar 0.12cwrk hottest 0.27c mean 0.04c------------------------------mean request 3.41 ms before 1.11 envoy 0.58 sidecar 0.82 worker 0.90 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.60c sidecar 0.52cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.10 ms before 1.02 envoy 0.56 sidecar 0.79 worker 0.73 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 29.0 pool 34.0------------------------------envoy 0.60c sidecar 0.51cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.18 ms before 1.06 envoy 0.60 sidecar 0.79 worker 0.72 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1434 /sachieved 1434 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 393 pool 393------------------------------envoy 0.90c sidecar 0.52cwrk hottest 0.37c mean 0.05c------------------------------mean request 5.50 ms before 2.56 envoy 0.91 sidecar 1.17 worker 0.86 resume 1.12 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 79.0 pool 393------------------------------envoy 1.25c sidecar 0.75cwrk hottest 0.34c mean 0.08c------------------------------mean request 3.09 ms before 0.95 envoy 0.82 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 393------------------------------envoy 1.21c sidecar 0.93cwrk hottest 0.43c mean 0.08c------------------------------mean request 3.04 ms before 0.97 envoy 0.72 sidecar 0.81 worker 0.54 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2240 /sachieved 2240 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 393------------------------------envoy 1.33c sidecar 0.93cwrk hottest 0.58c mean 0.08c------------------------------mean request 3.03 ms before 0.97 envoy 0.71 sidecar 0.81 worker 0.54 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 1041 pool 1047------------------------------envoy 1.86c sidecar 1.25cwrk hottest 0.73c mean 0.12c------------------------------mean request 8.11 ms before 3.20 envoy 1.64 sidecar 2.13 worker 1.15 resume 2.09 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.6 msin-flight 26.0 pool 1047------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.77c mean 0.11c------------------------------mean request 3.24 ms before 1.09 envoy 0.71 sidecar 0.81 worker 0.64 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3520 /sachieved 3519 /s (100.0% ok)p50 3.4 ms p95 6.7 msin-flight 299 pool 1047------------------------------envoy 2.05c sidecar 1.30cwrk hottest 0.86c mean 0.13c------------------------------mean request 4.32 ms before 1.29 envoy 0.86 sidecar 1.19 worker 0.99 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.9 ms p95 213.9 msin-flight 1594 pool 1591------------------------------envoy 2.54c sidecar 1.54cwrk hottest 0.82c mean 0.16c------------------------------mean request 26.47 ms before 13.94 envoy 5.21 sidecar 1.02 worker 6.31 resume 0.96 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.7 ms p95 11.2 msin-flight 426 pool 1591------------------------------envoy 2.23c sidecar 1.55cwrk hottest 0.91c mean 0.14c------------------------------mean request 5.77 ms before 2.31 envoy 1.18 sidecar 0.84 worker 1.44 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4321 /sachieved 4316 /s (100.0% ok)p50 4.0 ms p95 30.2 msin-flight 127 pool 1591------------------------------envoy 2.38c sidecar 1.56cwrk hottest 0.90c mean 0.15c------------------------------mean request 8.30 ms before 3.86 envoy 1.41 sidecar 0.85 worker 2.19 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5006 /s (100.0% ok)p50 6.3 ms p95 26.0 msin-flight 147 pool 1591------------------------------envoy 2.77c sidecar 0.00cwrk hottest 0.89c mean 0.17c------------------------------mean request 8.77 ms before 2.39 envoy 2.11 sidecar 0.84 worker 3.43 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 5.8 ms p95 22.9 msin-flight 194 pool 1591------------------------------envoy 2.78c sidecar 1.75cwrk hottest 0.88c mean 0.17c------------------------------mean request 8.21 ms before 3.32 envoy 1.58 sidecar 0.85 worker 2.45 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 4.2 ms p95 14.1 msin-flight 70.0 pool 1591------------------------------envoy 2.93c sidecar 1.78cwrk hottest 0.88c mean 0.18c------------------------------mean request 5.66 ms before 2.08 envoy 1.16 sidecar 0.83 worker 1.60 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5648 /sachieved 5645 /s (100.0% ok)p50 6.7 ms p95 32.6 msin-flight 209 pool 1591------------------------------envoy 3.17c sidecar 1.88cwrk hottest 0.86c mean 0.20c------------------------------mean request 12.08 ms before 5.38 envoy 2.22 sidecar 0.90 worker 3.60 resume 0.85 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5919 /s (100.0% ok)p50 15.5 ms p95 45.0 msin-flight 3112 pool 3113------------------------------envoy 3.37c sidecar 1.94cwrk hottest 0.83c mean 0.21c------------------------------mean request 18.03 ms before 7.86 envoy 3.20 sidecar 1.13 worker 5.84 resume 1.07 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6059 /s (100.0% ok)p50 4.8 ms p95 320.1 msin-flight 3095 pool 3113------------------------------envoy 4.14c sidecar 2.10cwrk hottest 0.77c mean 0.26c------------------------------mean request 40.70 ms before 13.77 envoy 10.93 sidecar 0.92 worker 15.08 resume 0.87 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6387 /sachieved 6378 /s (100.0% ok)p50 5.3 ms p95 49.5 msin-flight 415 pool 3113------------------------------envoy 4.06c sidecar 2.19cwrk hottest 0.80c mean 0.25c------------------------------mean request 12.90 ms before 6.48 envoy 2.13 sidecar 0.89 worker 3.41 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7006 /s (100.0% ok)p50 5.6 ms p95 51.6 msin-flight 245 pool 3113------------------------------envoy 4.43c sidecar 2.36cwrk hottest 0.77c mean 0.27c------------------------------mean request 17.19 ms before 6.72 envoy 3.59 sidecar 0.92 worker 5.96 resume 0.88 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7003 /s (99.5% ok)p50 5.3 ms p95 41.5 msin-flight 1473 pool 3113------------------------------envoy 4.43c sidecar 2.31cwrk hottest 0.77c mean 0.28c------------------------------mean request 15.96 ms before 3.85 envoy 3.82 sidecar 2.39 worker 5.90 resume 2.36 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7253 /sachieved 7244 /s (100.0% ok)p50 5.7 ms p95 19.3 msin-flight 233 pool 3113------------------------------envoy 4.92c sidecar 2.47cwrk hottest 0.70c mean 0.32c------------------------------mean request 7.64 ms before 2.88 envoy 1.64 sidecar 0.89 worker 2.23 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 8.5 ms p95 36.6 msin-flight 282 pool 3113------------------------------envoy 5.74c sidecar 2.76cwrk hottest 0.65c mean 0.36c------------------------------mean request 12.51 ms before 4.66 envoy 2.85 sidecar 0.93 worker 4.07 resume 0.89 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7996 /s (99.5% ok)p50 8.2 ms p95 70.9 msin-flight 1862 pool 3113------------------------------envoy 5.82c sidecar 2.73cwrk hottest 0.65c mean 0.36c------------------------------mean request 19.86 ms before 5.66 envoy 4.87 sidecar 2.18 worker 7.15 resume 2.15 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8722 /sachieved 8577 /s (100.0% ok)p50 46.9 ms p95 309.8 msin-flight 3448 pool 3448------------------------------envoy 6.87c sidecar 2.73cwrk hottest 0.63c mean 0.44c------------------------------mean request 79.93 ms before 38.83 envoy 15.02 sidecar 1.65 worker 24.43 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9157 /s (100.0% ok)p50 363.0 ms p95 611.0 msin-flight 5175 pool 5175------------------------------envoy 7.30c sidecar 2.57cwrk hottest 0.47c mean 0.45c------------------------------mean request 366.39 ms before 107.80 envoy 101.23 sidecar 2.68 worker 154.69 resume 1.71 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9021 /s (100.0% ok)p50 94.2 ms p95 272.6 msin-flight 2121 pool 5175------------------------------envoy 7.26c sidecar 2.32cwrk hottest 0.49c mean 0.45c------------------------------mean request 114.40 ms before 37.59 envoy 29.66 sidecar 1.19 worker 45.96 resume 1.14 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9524 /sachieved 8261 /s (100.0% ok)p50 225.5 ms p95 1698.8 msin-flight 28k pool 28k------------------------------envoy 7.04c sidecar 2.47cwrk hottest 0.50c mean 0.44c------------------------------mean request 548.22 ms before 194.58 envoy 182.24 sidecar 1.88 worker 169.52 resume 1.81 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 890 /sachieved 890 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 35.0 pool 34.0------------------------------envoy 0.62c sidecar 0.12cwrk hottest 0.27c mean 0.04c------------------------------mean request 3.41 ms before 1.11 envoy 0.58 sidecar 0.82 worker 0.90 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.60c sidecar 0.52cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.10 ms before 1.02 envoy 0.56 sidecar 0.79 worker 0.73 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 29.0 pool 34.0------------------------------envoy 0.60c sidecar 0.51cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.18 ms before 1.06 envoy 0.60 sidecar 0.79 worker 0.72 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1434 /sachieved 1434 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 393 pool 393------------------------------envoy 0.90c sidecar 0.52cwrk hottest 0.37c mean 0.05c------------------------------mean request 5.50 ms before 2.56 envoy 0.91 sidecar 1.17 worker 0.86 resume 1.12 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 79.0 pool 393------------------------------envoy 1.25c sidecar 0.75cwrk hottest 0.34c mean 0.08c------------------------------mean request 3.09 ms before 0.95 envoy 0.82 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 393------------------------------envoy 1.21c sidecar 0.93cwrk hottest 0.43c mean 0.08c------------------------------mean request 3.04 ms before 0.97 envoy 0.72 sidecar 0.81 worker 0.54 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2240 /sachieved 2240 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 393------------------------------envoy 1.33c sidecar 0.93cwrk hottest 0.58c mean 0.08c------------------------------mean request 3.03 ms before 0.97 envoy 0.71 sidecar 0.81 worker 0.54 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 1041 pool 1047------------------------------envoy 1.86c sidecar 1.25cwrk hottest 0.73c mean 0.12c------------------------------mean request 8.11 ms before 3.20 envoy 1.64 sidecar 2.13 worker 1.15 resume 2.09 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.6 msin-flight 26.0 pool 1047------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.77c mean 0.11c------------------------------mean request 3.24 ms before 1.09 envoy 0.71 sidecar 0.81 worker 0.64 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3520 /sachieved 3519 /s (100.0% ok)p50 3.4 ms p95 6.7 msin-flight 299 pool 1047------------------------------envoy 2.05c sidecar 1.30cwrk hottest 0.86c mean 0.13c------------------------------mean request 4.32 ms before 1.29 envoy 0.86 sidecar 1.19 worker 0.99 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.9 ms p95 213.9 msin-flight 1594 pool 1591------------------------------envoy 2.54c sidecar 1.54cwrk hottest 0.82c mean 0.16c------------------------------mean request 26.47 ms before 13.94 envoy 5.21 sidecar 1.02 worker 6.31 resume 0.96 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.7 ms p95 11.2 msin-flight 426 pool 1591------------------------------envoy 2.23c sidecar 1.55cwrk hottest 0.91c mean 0.14c------------------------------mean request 5.77 ms before 2.31 envoy 1.18 sidecar 0.84 worker 1.44 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4321 /sachieved 4316 /s (100.0% ok)p50 4.0 ms p95 30.2 msin-flight 127 pool 1591------------------------------envoy 2.38c sidecar 1.56cwrk hottest 0.90c mean 0.15c------------------------------mean request 8.30 ms before 3.86 envoy 1.41 sidecar 0.85 worker 2.19 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5006 /s (100.0% ok)p50 6.3 ms p95 26.0 msin-flight 147 pool 1591------------------------------envoy 2.77c sidecar 0.00cwrk hottest 0.89c mean 0.17c------------------------------mean request 8.77 ms before 2.39 envoy 2.11 sidecar 0.84 worker 3.43 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 5.8 ms p95 22.9 msin-flight 194 pool 1591------------------------------envoy 2.78c sidecar 1.75cwrk hottest 0.88c mean 0.17c------------------------------mean request 8.21 ms before 3.32 envoy 1.58 sidecar 0.85 worker 2.45 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 4.2 ms p95 14.1 msin-flight 70.0 pool 1591------------------------------envoy 2.93c sidecar 1.78cwrk hottest 0.88c mean 0.18c------------------------------mean request 5.66 ms before 2.08 envoy 1.16 sidecar 0.83 worker 1.60 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5648 /sachieved 5645 /s (100.0% ok)p50 6.7 ms p95 32.6 msin-flight 209 pool 1591------------------------------envoy 3.17c sidecar 1.88cwrk hottest 0.86c mean 0.20c------------------------------mean request 12.08 ms before 5.38 envoy 2.22 sidecar 0.90 worker 3.60 resume 0.85 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5919 /s (100.0% ok)p50 15.5 ms p95 45.0 msin-flight 3112 pool 3113------------------------------envoy 3.37c sidecar 1.94cwrk hottest 0.83c mean 0.21c------------------------------mean request 18.03 ms before 7.86 envoy 3.20 sidecar 1.13 worker 5.84 resume 1.07 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6059 /s (100.0% ok)p50 4.8 ms p95 320.1 msin-flight 3095 pool 3113------------------------------envoy 4.14c sidecar 2.10cwrk hottest 0.77c mean 0.26c------------------------------mean request 40.70 ms before 13.77 envoy 10.93 sidecar 0.92 worker 15.08 resume 0.87 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6387 /sachieved 6378 /s (100.0% ok)p50 5.3 ms p95 49.5 msin-flight 415 pool 3113------------------------------envoy 4.06c sidecar 2.19cwrk hottest 0.80c mean 0.25c------------------------------mean request 12.90 ms before 6.48 envoy 2.13 sidecar 0.89 worker 3.41 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7006 /s (100.0% ok)p50 5.6 ms p95 51.6 msin-flight 245 pool 3113------------------------------envoy 4.43c sidecar 2.36cwrk hottest 0.77c mean 0.27c------------------------------mean request 17.19 ms before 6.72 envoy 3.59 sidecar 0.92 worker 5.96 resume 0.88 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7003 /s (99.5% ok)p50 5.3 ms p95 41.5 msin-flight 1473 pool 3113------------------------------envoy 4.43c sidecar 2.31cwrk hottest 0.77c mean 0.28c------------------------------mean request 15.96 ms before 3.85 envoy 3.82 sidecar 2.39 worker 5.90 resume 2.36 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7253 /sachieved 7244 /s (100.0% ok)p50 5.7 ms p95 19.3 msin-flight 233 pool 3113------------------------------envoy 4.92c sidecar 2.47cwrk hottest 0.70c mean 0.32c------------------------------mean request 7.64 ms before 2.88 envoy 1.64 sidecar 0.89 worker 2.23 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 8.5 ms p95 36.6 msin-flight 282 pool 3113------------------------------envoy 5.74c sidecar 2.76cwrk hottest 0.65c mean 0.36c------------------------------mean request 12.51 ms before 4.66 envoy 2.85 sidecar 0.93 worker 4.07 resume 0.89 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7996 /s (99.5% ok)p50 8.2 ms p95 70.9 msin-flight 1862 pool 3113------------------------------envoy 5.82c sidecar 2.73cwrk hottest 0.65c mean 0.36c------------------------------mean request 19.86 ms before 5.66 envoy 4.87 sidecar 2.18 worker 7.15 resume 2.15 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8722 /sachieved 8577 /s (100.0% ok)p50 46.9 ms p95 309.8 msin-flight 3448 pool 3448------------------------------envoy 6.87c sidecar 2.73cwrk hottest 0.63c mean 0.44c------------------------------mean request 79.93 ms before 38.83 envoy 15.02 sidecar 1.65 worker 24.43 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9157 /s (100.0% ok)p50 363.0 ms p95 611.0 msin-flight 5175 pool 5175------------------------------envoy 7.30c sidecar 2.57cwrk hottest 0.47c mean 0.45c------------------------------mean request 366.39 ms before 107.80 envoy 101.23 sidecar 2.68 worker 154.69 resume 1.71 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9021 /s (100.0% ok)p50 94.2 ms p95 272.6 msin-flight 2121 pool 5175------------------------------envoy 7.26c sidecar 2.32cwrk hottest 0.49c mean 0.45c------------------------------mean request 114.40 ms before 37.59 envoy 29.66 sidecar 1.19 worker 45.96 resume 1.14 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9524 /sachieved 8261 /s (100.0% ok)p50 225.5 ms p95 1698.8 msin-flight 28k pool 28k------------------------------envoy 7.04c sidecar 2.47cwrk hottest 0.50c mean 0.44c------------------------------mean request 548.22 ms before 194.58 envoy 182.24 sidecar 1.88 worker 169.52 resume 1.81 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 890 /sachieved 890 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 35.0 pool 34.0------------------------------envoy 0.62c sidecar 0.12cwrk hottest 0.27c mean 0.04c------------------------------mean request 3.41 ms before 1.11 envoy 0.58 sidecar 0.82 worker 0.90 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.60c sidecar 0.52cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.10 ms before 1.02 envoy 0.56 sidecar 0.79 worker 0.73 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 29.0 pool 34.0------------------------------envoy 0.60c sidecar 0.51cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.18 ms before 1.06 envoy 0.60 sidecar 0.79 worker 0.72 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1434 /sachieved 1434 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 393 pool 393------------------------------envoy 0.90c sidecar 0.52cwrk hottest 0.37c mean 0.05c------------------------------mean request 5.50 ms before 2.56 envoy 0.91 sidecar 1.17 worker 0.86 resume 1.12 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 79.0 pool 393------------------------------envoy 1.25c sidecar 0.75cwrk hottest 0.34c mean 0.08c------------------------------mean request 3.09 ms before 0.95 envoy 0.82 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 393------------------------------envoy 1.21c sidecar 0.93cwrk hottest 0.43c mean 0.08c------------------------------mean request 3.04 ms before 0.97 envoy 0.72 sidecar 0.81 worker 0.54 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2240 /sachieved 2240 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 393------------------------------envoy 1.33c sidecar 0.93cwrk hottest 0.58c mean 0.08c------------------------------mean request 3.03 ms before 0.97 envoy 0.71 sidecar 0.81 worker 0.54 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 1041 pool 1047------------------------------envoy 1.86c sidecar 1.25cwrk hottest 0.73c mean 0.12c------------------------------mean request 8.11 ms before 3.20 envoy 1.64 sidecar 2.13 worker 1.15 resume 2.09 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.6 msin-flight 26.0 pool 1047------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.77c mean 0.11c------------------------------mean request 3.24 ms before 1.09 envoy 0.71 sidecar 0.81 worker 0.64 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3520 /sachieved 3519 /s (100.0% ok)p50 3.4 ms p95 6.7 msin-flight 299 pool 1047------------------------------envoy 2.05c sidecar 1.30cwrk hottest 0.86c mean 0.13c------------------------------mean request 4.32 ms before 1.29 envoy 0.86 sidecar 1.19 worker 0.99 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.9 ms p95 213.9 msin-flight 1594 pool 1591------------------------------envoy 2.54c sidecar 1.54cwrk hottest 0.82c mean 0.16c------------------------------mean request 26.47 ms before 13.94 envoy 5.21 sidecar 1.02 worker 6.31 resume 0.96 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.7 ms p95 11.2 msin-flight 426 pool 1591------------------------------envoy 2.23c sidecar 1.55cwrk hottest 0.91c mean 0.14c------------------------------mean request 5.77 ms before 2.31 envoy 1.18 sidecar 0.84 worker 1.44 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4321 /sachieved 4316 /s (100.0% ok)p50 4.0 ms p95 30.2 msin-flight 127 pool 1591------------------------------envoy 2.38c sidecar 1.56cwrk hottest 0.90c mean 0.15c------------------------------mean request 8.30 ms before 3.86 envoy 1.41 sidecar 0.85 worker 2.19 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5006 /s (100.0% ok)p50 6.3 ms p95 26.0 msin-flight 147 pool 1591------------------------------envoy 2.77c sidecar 0.00cwrk hottest 0.89c mean 0.17c------------------------------mean request 8.77 ms before 2.39 envoy 2.11 sidecar 0.84 worker 3.43 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 5.8 ms p95 22.9 msin-flight 194 pool 1591------------------------------envoy 2.78c sidecar 1.75cwrk hottest 0.88c mean 0.17c------------------------------mean request 8.21 ms before 3.32 envoy 1.58 sidecar 0.85 worker 2.45 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 4.2 ms p95 14.1 msin-flight 70.0 pool 1591------------------------------envoy 2.93c sidecar 1.78cwrk hottest 0.88c mean 0.18c------------------------------mean request 5.66 ms before 2.08 envoy 1.16 sidecar 0.83 worker 1.60 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5648 /sachieved 5645 /s (100.0% ok)p50 6.7 ms p95 32.6 msin-flight 209 pool 1591------------------------------envoy 3.17c sidecar 1.88cwrk hottest 0.86c mean 0.20c------------------------------mean request 12.08 ms before 5.38 envoy 2.22 sidecar 0.90 worker 3.60 resume 0.85 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5919 /s (100.0% ok)p50 15.5 ms p95 45.0 msin-flight 3112 pool 3113------------------------------envoy 3.37c sidecar 1.94cwrk hottest 0.83c mean 0.21c------------------------------mean request 18.03 ms before 7.86 envoy 3.20 sidecar 1.13 worker 5.84 resume 1.07 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6059 /s (100.0% ok)p50 4.8 ms p95 320.1 msin-flight 3095 pool 3113------------------------------envoy 4.14c sidecar 2.10cwrk hottest 0.77c mean 0.26c------------------------------mean request 40.70 ms before 13.77 envoy 10.93 sidecar 0.92 worker 15.08 resume 0.87 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6387 /sachieved 6378 /s (100.0% ok)p50 5.3 ms p95 49.5 msin-flight 415 pool 3113------------------------------envoy 4.06c sidecar 2.19cwrk hottest 0.80c mean 0.25c------------------------------mean request 12.90 ms before 6.48 envoy 2.13 sidecar 0.89 worker 3.41 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7006 /s (100.0% ok)p50 5.6 ms p95 51.6 msin-flight 245 pool 3113------------------------------envoy 4.43c sidecar 2.36cwrk hottest 0.77c mean 0.27c------------------------------mean request 17.19 ms before 6.72 envoy 3.59 sidecar 0.92 worker 5.96 resume 0.88 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7003 /s (99.5% ok)p50 5.3 ms p95 41.5 msin-flight 1473 pool 3113------------------------------envoy 4.43c sidecar 2.31cwrk hottest 0.77c mean 0.28c------------------------------mean request 15.96 ms before 3.85 envoy 3.82 sidecar 2.39 worker 5.90 resume 2.36 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7253 /sachieved 7244 /s (100.0% ok)p50 5.7 ms p95 19.3 msin-flight 233 pool 3113------------------------------envoy 4.92c sidecar 2.47cwrk hottest 0.70c mean 0.32c------------------------------mean request 7.64 ms before 2.88 envoy 1.64 sidecar 0.89 worker 2.23 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 8.5 ms p95 36.6 msin-flight 282 pool 3113------------------------------envoy 5.74c sidecar 2.76cwrk hottest 0.65c mean 0.36c------------------------------mean request 12.51 ms before 4.66 envoy 2.85 sidecar 0.93 worker 4.07 resume 0.89 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7996 /s (99.5% ok)p50 8.2 ms p95 70.9 msin-flight 1862 pool 3113------------------------------envoy 5.82c sidecar 2.73cwrk hottest 0.65c mean 0.36c------------------------------mean request 19.86 ms before 5.66 envoy 4.87 sidecar 2.18 worker 7.15 resume 2.15 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8722 /sachieved 8577 /s (100.0% ok)p50 46.9 ms p95 309.8 msin-flight 3448 pool 3448------------------------------envoy 6.87c sidecar 2.73cwrk hottest 0.63c mean 0.44c------------------------------mean request 79.93 ms before 38.83 envoy 15.02 sidecar 1.65 worker 24.43 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9157 /s (100.0% ok)p50 363.0 ms p95 611.0 msin-flight 5175 pool 5175------------------------------envoy 7.30c sidecar 2.57cwrk hottest 0.47c mean 0.45c------------------------------mean request 366.39 ms before 107.80 envoy 101.23 sidecar 2.68 worker 154.69 resume 1.71 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9021 /s (100.0% ok)p50 94.2 ms p95 272.6 msin-flight 2121 pool 5175------------------------------envoy 7.26c sidecar 2.32cwrk hottest 0.49c mean 0.45c------------------------------mean request 114.40 ms before 37.59 envoy 29.66 sidecar 1.19 worker 45.96 resume 1.14 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9524 /sachieved 8261 /s (100.0% ok)p50 225.5 ms p95 1698.8 msin-flight 28k pool 28k------------------------------envoy 7.04c sidecar 2.47cwrk hottest 0.50c mean 0.44c------------------------------mean request 548.22 ms before 194.58 envoy 182.24 sidecar 1.88 worker 169.52 resume 1.81 (control plane) +rung 0 1000 qps setpoint [warmup]------------------------------offered 890 /sachieved 890 /s (100.0% ok)p50 3.1 ms p95 4.8 msin-flight 35.0 pool 34.0------------------------------envoy 0.62c sidecar 0.12cwrk hottest 0.27c mean 0.04c------------------------------mean request 3.41 ms before 1.11 envoy 0.58 sidecar 0.82 worker 0.90 resume 0.77 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.8 msin-flight 9.0 pool 34.0------------------------------envoy 0.60c sidecar 0.52cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.10 ms before 1.02 envoy 0.56 sidecar 0.79 worker 0.73 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.1 ms p95 4.1 msin-flight 29.0 pool 34.0------------------------------envoy 0.60c sidecar 0.51cwrk hottest 0.35c mean 0.04c------------------------------mean request 3.18 ms before 1.06 envoy 0.60 sidecar 0.79 worker 0.72 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1434 /sachieved 1434 /s (100.0% ok)p50 3.0 ms p95 4.3 msin-flight 393 pool 393------------------------------envoy 0.90c sidecar 0.52cwrk hottest 0.37c mean 0.05c------------------------------mean request 5.50 ms before 2.56 envoy 0.91 sidecar 1.17 worker 0.86 resume 1.12 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 79.0 pool 393------------------------------envoy 1.25c sidecar 0.75cwrk hottest 0.34c mean 0.08c------------------------------mean request 3.09 ms before 0.95 envoy 0.82 sidecar 0.80 worker 0.52 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 3.0 ms p95 3.9 msin-flight 14.0 pool 393------------------------------envoy 1.21c sidecar 0.93cwrk hottest 0.43c mean 0.08c------------------------------mean request 3.04 ms before 0.97 envoy 0.72 sidecar 0.81 worker 0.54 resume 0.77 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2240 /sachieved 2240 /s (100.0% ok)p50 2.9 ms p95 3.9 msin-flight 16.0 pool 393------------------------------envoy 1.33c sidecar 0.93cwrk hottest 0.58c mean 0.08c------------------------------mean request 3.03 ms before 0.97 envoy 0.71 sidecar 0.81 worker 0.54 resume 0.76 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.9 msin-flight 1041 pool 1047------------------------------envoy 1.86c sidecar 1.25cwrk hottest 0.73c mean 0.12c------------------------------mean request 8.11 ms before 3.20 envoy 1.64 sidecar 2.13 worker 1.15 resume 2.09 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.6 msin-flight 26.0 pool 1047------------------------------envoy 1.82c sidecar 1.30cwrk hottest 0.77c mean 0.11c------------------------------mean request 3.24 ms before 1.09 envoy 0.71 sidecar 0.81 worker 0.64 resume 0.77 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3520 /sachieved 3519 /s (100.0% ok)p50 3.4 ms p95 6.7 msin-flight 299 pool 1047------------------------------envoy 2.05c sidecar 1.30cwrk hottest 0.86c mean 0.13c------------------------------mean request 4.32 ms before 1.29 envoy 0.86 sidecar 1.19 worker 0.99 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.9 ms p95 213.9 msin-flight 1594 pool 1591------------------------------envoy 2.54c sidecar 1.54cwrk hottest 0.82c mean 0.16c------------------------------mean request 26.47 ms before 13.94 envoy 5.21 sidecar 1.02 worker 6.31 resume 0.96 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.7 ms p95 11.2 msin-flight 426 pool 1591------------------------------envoy 2.23c sidecar 1.55cwrk hottest 0.91c mean 0.14c------------------------------mean request 5.77 ms before 2.31 envoy 1.18 sidecar 0.84 worker 1.44 resume 0.80 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4321 /sachieved 4316 /s (100.0% ok)p50 4.0 ms p95 30.2 msin-flight 127 pool 1591------------------------------envoy 2.38c sidecar 1.56cwrk hottest 0.90c mean 0.15c------------------------------mean request 8.30 ms before 3.86 envoy 1.41 sidecar 0.85 worker 2.19 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5006 /s (100.0% ok)p50 6.3 ms p95 26.0 msin-flight 147 pool 1591------------------------------envoy 2.77c sidecar 0.00cwrk hottest 0.89c mean 0.17c------------------------------mean request 8.77 ms before 2.39 envoy 2.11 sidecar 0.84 worker 3.43 resume 0.80 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5001 /s (100.0% ok)p50 5.8 ms p95 22.9 msin-flight 194 pool 1591------------------------------envoy 2.78c sidecar 1.75cwrk hottest 0.88c mean 0.17c------------------------------mean request 8.21 ms before 3.32 envoy 1.58 sidecar 0.85 worker 2.45 resume 0.81 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 4.2 ms p95 14.1 msin-flight 70.0 pool 1591------------------------------envoy 2.93c sidecar 1.78cwrk hottest 0.88c mean 0.18c------------------------------mean request 5.66 ms before 2.08 envoy 1.16 sidecar 0.83 worker 1.60 resume 0.79 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5648 /sachieved 5645 /s (100.0% ok)p50 6.7 ms p95 32.6 msin-flight 209 pool 1591------------------------------envoy 3.17c sidecar 1.88cwrk hottest 0.86c mean 0.20c------------------------------mean request 12.08 ms before 5.38 envoy 2.22 sidecar 0.90 worker 3.60 resume 0.85 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 5919 /s (100.0% ok)p50 15.5 ms p95 45.0 msin-flight 3112 pool 3113------------------------------envoy 3.37c sidecar 1.94cwrk hottest 0.83c mean 0.21c------------------------------mean request 18.03 ms before 7.86 envoy 3.20 sidecar 1.13 worker 5.84 resume 1.07 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6059 /s (100.0% ok)p50 4.8 ms p95 320.1 msin-flight 3095 pool 3113------------------------------envoy 4.14c sidecar 2.10cwrk hottest 0.77c mean 0.26c------------------------------mean request 40.70 ms before 13.77 envoy 10.93 sidecar 0.92 worker 15.08 resume 0.87 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6387 /sachieved 6378 /s (100.0% ok)p50 5.3 ms p95 49.5 msin-flight 415 pool 3113------------------------------envoy 4.06c sidecar 2.19cwrk hottest 0.80c mean 0.25c------------------------------mean request 12.90 ms before 6.48 envoy 2.13 sidecar 0.89 worker 3.41 resume 0.85 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7006 /s (100.0% ok)p50 5.6 ms p95 51.6 msin-flight 245 pool 3113------------------------------envoy 4.43c sidecar 2.36cwrk hottest 0.77c mean 0.27c------------------------------mean request 17.19 ms before 6.72 envoy 3.59 sidecar 0.92 worker 5.96 resume 0.88 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7003 /s (99.5% ok)p50 5.3 ms p95 41.5 msin-flight 1473 pool 3113------------------------------envoy 4.43c sidecar 2.31cwrk hottest 0.77c mean 0.28c------------------------------mean request 15.96 ms before 3.85 envoy 3.82 sidecar 2.39 worker 5.90 resume 2.36 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7253 /sachieved 7244 /s (100.0% ok)p50 5.7 ms p95 19.3 msin-flight 233 pool 3113------------------------------envoy 4.92c sidecar 2.47cwrk hottest 0.70c mean 0.32c------------------------------mean request 7.64 ms before 2.88 envoy 1.64 sidecar 0.89 worker 2.23 resume 0.84 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 8004 /s (100.0% ok)p50 8.5 ms p95 36.6 msin-flight 282 pool 3113------------------------------envoy 5.74c sidecar 2.76cwrk hottest 0.65c mean 0.36c------------------------------mean request 12.51 ms before 4.66 envoy 2.85 sidecar 0.93 worker 4.07 resume 0.89 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7996 /s (99.5% ok)p50 8.2 ms p95 70.9 msin-flight 1862 pool 3113------------------------------envoy 5.82c sidecar 2.73cwrk hottest 0.65c mean 0.36c------------------------------mean request 19.86 ms before 5.66 envoy 4.87 sidecar 2.18 worker 7.15 resume 2.15 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8722 /sachieved 8577 /s (100.0% ok)p50 46.9 ms p95 309.8 msin-flight 3448 pool 3448------------------------------envoy 6.87c sidecar 2.73cwrk hottest 0.63c mean 0.44c------------------------------mean request 79.93 ms before 38.83 envoy 15.02 sidecar 1.65 worker 24.43 resume 1.60 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9157 /s (100.0% ok)p50 363.0 ms p95 611.0 msin-flight 5175 pool 5175------------------------------envoy 7.30c sidecar 2.57cwrk hottest 0.47c mean 0.45c------------------------------mean request 366.39 ms before 107.80 envoy 101.23 sidecar 2.68 worker 154.69 resume 1.71 (control plane) +rung 8 9000 qps setpoint------------------------------offered 9000 /sachieved 9021 /s (100.0% ok)p50 94.2 ms p95 272.6 msin-flight 2121 pool 5175------------------------------envoy 7.26c sidecar 2.32cwrk hottest 0.49c mean 0.45c------------------------------mean request 114.40 ms before 37.59 envoy 29.66 sidecar 1.19 worker 45.96 resume 1.14 (control plane) +rung 9 10k qps setpoint [warmup]------------------------------offered 9524 /sachieved 8261 /s (100.0% ok)p50 225.5 ms p95 1698.8 msin-flight 28k pool 28k------------------------------envoy 7.04c sidecar 2.47cwrk hottest 0.50c mean 0.44c------------------------------mean request 548.22 ms before 194.58 envoy 182.24 sidecar 1.88 worker 169.52 resume 1.81 (control plane) +
+ + + +atenet-router capacity — 16c pinned +substrate-routercap · c3-standard-88 · window boundaries are cAdvisor's, so one vertical line is one interval in every panel + +Offered vs achieved throughput, and concurrency + + + + + + + + + +R0 +R1 +R2 +R3 +R4 +R5 +R6 +R7 +R8 + +0 + +10k + +20k + +30k + +40k +requests / s + + + + + + +envoy sheds above 20,000 in flight + + +router kernel out of upstream source ports at 28,232 (measured) + + + + + + + + + + + + + + + + + + + + + + + + + + + +offered + + + + + + + + + + + + + + + + + + + + + + + + + + + +achieved + + + + + + + + + + + + + + + + + + + + + + + + + + + +success + + + + + + + + + + + + + + + + + + + + + + + + + + + +in-flight (max) + + + + + + + + + + + + + + + + + + + + + + + + + + + +client pool + +Client latency from scheduled send time (log scale) + + + + + + + + + + +10.0 + +100 + +1000 +milliseconds + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +p50 + + + + + + + + + + + + + + + + + + + + + + + + + + + +p95 + +Where the mean request spent its time + + + + + + + + + + +0 + +20.0 + +40.0 + +60.0 + +80.0 + +100 +% of mean request + + + + + + + + + + +worker leg + +sidecar + +Envoy itself + +before Envoy + +below the floor + +Router CPU, per container + + + + + + + + + + +0 + +5.0 + +10.0 + +15.0 +cores + + + + + + +envoy CPU limit, 16 cores — the variable under test + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router + +Per-thread CPU: hottest worker vs the mean + + + + + + + + + + +0 + +0.25 + +0.5 + +0.75 + +1.0 +cores of one thread + + + + + + +one core — a single thread's ceiling + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +hottest worker + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +mean worker + +Router memory working set, per container + + + + + + + + + + +0 + +500.0M + +1.0G + +1.5G + +2.0G +bytes + +100 + +200 + +300 + +400 +seconds since the arm started + + + + + + + + + + + + + + + + + + + + + + + + + + + +envoy + + + + + + + + + + + + + + + + + + + + + + + + + +atenet-router +rung -1 0 qps setpoint------------------------------offered 449 /sachieved 449 /s (100.0% ok)p50 3.1 ms p95 5.6 msin-flight 28.0 pool 28.0------------------------------envoy 0.32c sidecar 0.00c------------------------------mean request 3.66 ms before 0.94 envoy 0.62 sidecar 0.83 worker 1.27 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 7.0 pool 28.0------------------------------envoy 0.59c sidecar 0.43cwrk hottest 0.24c mean 0.04c------------------------------mean request 3.03 ms before 0.98 envoy 0.68 sidecar 0.80 worker 0.57 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 5.0 pool 28.0------------------------------envoy 0.61c sidecar 0.49cwrk hottest 0.26c mean 0.04c------------------------------mean request 3.01 ms before 0.97 envoy 0.68 sidecar 0.78 worker 0.58 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1169 /sachieved 1169 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 0.69c sidecar 0.49cwrk hottest 0.25c mean 0.05c------------------------------mean request 3.00 ms before 0.97 envoy 0.71 sidecar 0.78 worker 0.53 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 10.0 pool 28.0------------------------------envoy 1.17c sidecar 0.00cwrk hottest 0.34c mean 0.07c------------------------------mean request 2.94 ms before 0.91 envoy 0.77 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 1.24c sidecar 0.90cwrk hottest 0.32c mean 0.08c------------------------------mean request 2.96 ms before 0.92 envoy 0.75 sidecar 0.80 worker 0.49 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 10.0 pool 28.0------------------------------envoy 1.22c sidecar 0.92cwrk hottest 0.34c mean 0.08c------------------------------mean request 2.99 ms before 0.94 envoy 0.78 sidecar 0.82 worker 0.44 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2560 /sachieved 2560 /s (100.0% ok)p50 3.0 ms p95 8.9 msin-flight 939 pool 939------------------------------envoy 1.66c sidecar 0.97cwrk hottest 0.40c mean 0.10c------------------------------mean request 9.50 ms before 4.14 envoy 2.17 sidecar 1.10 worker 2.08 resume 1.06 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 22.0 pool 939------------------------------envoy 1.79c sidecar 1.29cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.20 ms before 1.07 envoy 0.64 sidecar 0.81 worker 0.68 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 4.4 msin-flight 19.0 pool 939------------------------------envoy 1.67c sidecar 1.24cwrk hottest 0.67c mean 0.10c------------------------------mean request 3.28 ms before 1.13 envoy 0.58 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3756 /sachieved 3756 /s (100.0% ok)p50 3.3 ms p95 5.6 msin-flight 32.0 pool 939------------------------------envoy 2.22c sidecar 1.30cwrk hottest 0.79c mean 0.14c------------------------------mean request 3.57 ms before 1.21 envoy 0.70 sidecar 0.82 worker 0.83 resume 0.78 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.7 msin-flight 378 pool 939------------------------------envoy 2.47c sidecar 1.59cwrk hottest 0.78c mean 0.15c------------------------------mean request 4.20 ms before 1.11 envoy 0.85 sidecar 1.33 worker 0.91 resume 1.29 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 6.5 msin-flight 39.0 pool 939------------------------------envoy 2.47c sidecar 1.69cwrk hottest 0.82c mean 0.15c------------------------------mean request 3.69 ms before 1.25 envoy 0.79 sidecar 0.81 worker 0.83 resume 0.77 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4848 /sachieved 4848 /s (100.0% ok)p50 3.4 ms p95 8.4 msin-flight 44.0 pool 939------------------------------envoy 3.04c sidecar 1.73cwrk hottest 0.82c mean 0.19c------------------------------mean request 4.10 ms before 1.42 envoy 0.87 sidecar 0.81 worker 1.01 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.3 ms p95 6.0 msin-flight 67.0 pool 939------------------------------envoy 3.25c sidecar 1.92cwrk hottest 0.76c mean 0.20c------------------------------mean request 3.70 ms before 1.23 envoy 0.77 sidecar 0.82 worker 0.87 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.4 ms p95 7.5 msin-flight 1383 pool 1383------------------------------envoy 3.25c sidecar 1.95cwrk hottest 0.65c mean 0.21c------------------------------mean request 9.94 ms before 3.76 envoy 1.18 sidecar 2.66 worker 2.34 resume 2.62 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5836 /sachieved 5836 /s (100.0% ok)p50 4.2 ms p95 7.8 msin-flight 79.0 pool 1383------------------------------envoy 3.91c sidecar 2.07cwrk hottest 0.64c mean 0.25c------------------------------mean request 4.60 ms before 1.53 envoy 1.00 sidecar 0.84 worker 1.24 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 4.3 ms p95 8.8 msin-flight 321 pool 1383------------------------------envoy 4.12c sidecar 2.21cwrk hottest 0.68c mean 0.26c------------------------------mean request 5.01 ms before 1.68 envoy 1.10 sidecar 0.88 worker 1.36 resume 0.84 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6309 /sachieved 6306 /s (100.0% ok)p50 4.3 ms p95 24.1 msin-flight 750 pool 1383------------------------------envoy 4.49c sidecar 2.29cwrk hottest 0.69c mean 0.29c------------------------------mean request 8.32 ms before 3.67 envoy 1.49 sidecar 1.08 worker 2.08 resume 1.04 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7005 /s (100.0% ok)p50 5.6 ms p95 51.7 msin-flight 1273 pool 1383------------------------------envoy 5.07c sidecar 2.43cwrk hottest 0.65c mean 0.32c------------------------------mean request 13.36 ms before 4.83 envoy 3.15 sidecar 0.91 worker 4.48 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.9 ms p95 13.6 msin-flight 128 pool 1383------------------------------envoy 5.05c sidecar 2.52cwrk hottest 0.65c mean 0.32c------------------------------mean request 6.13 ms before 2.07 envoy 1.41 sidecar 0.88 worker 1.77 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 5.3 ms p95 12.2 msin-flight 925 pool 1383------------------------------envoy 4.99c sidecar 2.59cwrk hottest 0.63c mean 0.31c------------------------------mean request 6.16 ms before 2.11 envoy 1.38 sidecar 0.90 worker 1.77 resume 0.85 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7964 /sachieved 7909 /s (100.0% ok)p50 144.9 ms p95 260.9 msin-flight 2058 pool 2058------------------------------envoy 6.09c sidecar 2.41cwrk hottest 0.48c mean 0.38c------------------------------mean request 145.66 ms before 60.25 envoy 32.22 sidecar 1.27 worker 51.92 resume 1.21 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7825 /s (100.0% ok)p50 146.2 ms p95 484.7 msin-flight 4583 pool 4582------------------------------envoy 5.93c sidecar 2.42cwrk hottest 0.43c mean 0.37c------------------------------mean request 179.36 ms before 107.05 envoy 27.38 sidecar 1.36 worker 43.56 resume 1.30 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8648 /sachieved 7279 /s (99.4% ok)p50 1088.4 ms p95 2827.0 msin-flight 34k pool 34k------------------------------envoy 5.91c sidecar 2.27cwrk hottest 0.40c mean 0.37c------------------------------mean request 1331.75 ms before 573.44 envoy 449.57 sidecar 3.64 worker 305.09 resume 3.43 (control plane) +rung -1 0 qps setpoint------------------------------offered 449 /sachieved 449 /s (100.0% ok)p50 3.1 ms p95 5.6 msin-flight 28.0 pool 28.0------------------------------envoy 0.32c sidecar 0.00c------------------------------mean request 3.66 ms before 0.94 envoy 0.62 sidecar 0.83 worker 1.27 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 7.0 pool 28.0------------------------------envoy 0.59c sidecar 0.43cwrk hottest 0.24c mean 0.04c------------------------------mean request 3.03 ms before 0.98 envoy 0.68 sidecar 0.80 worker 0.57 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 5.0 pool 28.0------------------------------envoy 0.61c sidecar 0.49cwrk hottest 0.26c mean 0.04c------------------------------mean request 3.01 ms before 0.97 envoy 0.68 sidecar 0.78 worker 0.58 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1169 /sachieved 1169 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 0.69c sidecar 0.49cwrk hottest 0.25c mean 0.05c------------------------------mean request 3.00 ms before 0.97 envoy 0.71 sidecar 0.78 worker 0.53 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 10.0 pool 28.0------------------------------envoy 1.17c sidecar 0.00cwrk hottest 0.34c mean 0.07c------------------------------mean request 2.94 ms before 0.91 envoy 0.77 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 1.24c sidecar 0.90cwrk hottest 0.32c mean 0.08c------------------------------mean request 2.96 ms before 0.92 envoy 0.75 sidecar 0.80 worker 0.49 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 10.0 pool 28.0------------------------------envoy 1.22c sidecar 0.92cwrk hottest 0.34c mean 0.08c------------------------------mean request 2.99 ms before 0.94 envoy 0.78 sidecar 0.82 worker 0.44 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2560 /sachieved 2560 /s (100.0% ok)p50 3.0 ms p95 8.9 msin-flight 939 pool 939------------------------------envoy 1.66c sidecar 0.97cwrk hottest 0.40c mean 0.10c------------------------------mean request 9.50 ms before 4.14 envoy 2.17 sidecar 1.10 worker 2.08 resume 1.06 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 22.0 pool 939------------------------------envoy 1.79c sidecar 1.29cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.20 ms before 1.07 envoy 0.64 sidecar 0.81 worker 0.68 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 4.4 msin-flight 19.0 pool 939------------------------------envoy 1.67c sidecar 1.24cwrk hottest 0.67c mean 0.10c------------------------------mean request 3.28 ms before 1.13 envoy 0.58 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3756 /sachieved 3756 /s (100.0% ok)p50 3.3 ms p95 5.6 msin-flight 32.0 pool 939------------------------------envoy 2.22c sidecar 1.30cwrk hottest 0.79c mean 0.14c------------------------------mean request 3.57 ms before 1.21 envoy 0.70 sidecar 0.82 worker 0.83 resume 0.78 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.7 msin-flight 378 pool 939------------------------------envoy 2.47c sidecar 1.59cwrk hottest 0.78c mean 0.15c------------------------------mean request 4.20 ms before 1.11 envoy 0.85 sidecar 1.33 worker 0.91 resume 1.29 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 6.5 msin-flight 39.0 pool 939------------------------------envoy 2.47c sidecar 1.69cwrk hottest 0.82c mean 0.15c------------------------------mean request 3.69 ms before 1.25 envoy 0.79 sidecar 0.81 worker 0.83 resume 0.77 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4848 /sachieved 4848 /s (100.0% ok)p50 3.4 ms p95 8.4 msin-flight 44.0 pool 939------------------------------envoy 3.04c sidecar 1.73cwrk hottest 0.82c mean 0.19c------------------------------mean request 4.10 ms before 1.42 envoy 0.87 sidecar 0.81 worker 1.01 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.3 ms p95 6.0 msin-flight 67.0 pool 939------------------------------envoy 3.25c sidecar 1.92cwrk hottest 0.76c mean 0.20c------------------------------mean request 3.70 ms before 1.23 envoy 0.77 sidecar 0.82 worker 0.87 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.4 ms p95 7.5 msin-flight 1383 pool 1383------------------------------envoy 3.25c sidecar 1.95cwrk hottest 0.65c mean 0.21c------------------------------mean request 9.94 ms before 3.76 envoy 1.18 sidecar 2.66 worker 2.34 resume 2.62 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5836 /sachieved 5836 /s (100.0% ok)p50 4.2 ms p95 7.8 msin-flight 79.0 pool 1383------------------------------envoy 3.91c sidecar 2.07cwrk hottest 0.64c mean 0.25c------------------------------mean request 4.60 ms before 1.53 envoy 1.00 sidecar 0.84 worker 1.24 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 4.3 ms p95 8.8 msin-flight 321 pool 1383------------------------------envoy 4.12c sidecar 2.21cwrk hottest 0.68c mean 0.26c------------------------------mean request 5.01 ms before 1.68 envoy 1.10 sidecar 0.88 worker 1.36 resume 0.84 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6309 /sachieved 6306 /s (100.0% ok)p50 4.3 ms p95 24.1 msin-flight 750 pool 1383------------------------------envoy 4.49c sidecar 2.29cwrk hottest 0.69c mean 0.29c------------------------------mean request 8.32 ms before 3.67 envoy 1.49 sidecar 1.08 worker 2.08 resume 1.04 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7005 /s (100.0% ok)p50 5.6 ms p95 51.7 msin-flight 1273 pool 1383------------------------------envoy 5.07c sidecar 2.43cwrk hottest 0.65c mean 0.32c------------------------------mean request 13.36 ms before 4.83 envoy 3.15 sidecar 0.91 worker 4.48 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.9 ms p95 13.6 msin-flight 128 pool 1383------------------------------envoy 5.05c sidecar 2.52cwrk hottest 0.65c mean 0.32c------------------------------mean request 6.13 ms before 2.07 envoy 1.41 sidecar 0.88 worker 1.77 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 5.3 ms p95 12.2 msin-flight 925 pool 1383------------------------------envoy 4.99c sidecar 2.59cwrk hottest 0.63c mean 0.31c------------------------------mean request 6.16 ms before 2.11 envoy 1.38 sidecar 0.90 worker 1.77 resume 0.85 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7964 /sachieved 7909 /s (100.0% ok)p50 144.9 ms p95 260.9 msin-flight 2058 pool 2058------------------------------envoy 6.09c sidecar 2.41cwrk hottest 0.48c mean 0.38c------------------------------mean request 145.66 ms before 60.25 envoy 32.22 sidecar 1.27 worker 51.92 resume 1.21 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7825 /s (100.0% ok)p50 146.2 ms p95 484.7 msin-flight 4583 pool 4582------------------------------envoy 5.93c sidecar 2.42cwrk hottest 0.43c mean 0.37c------------------------------mean request 179.36 ms before 107.05 envoy 27.38 sidecar 1.36 worker 43.56 resume 1.30 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8648 /sachieved 7279 /s (99.4% ok)p50 1088.4 ms p95 2827.0 msin-flight 34k pool 34k------------------------------envoy 5.91c sidecar 2.27cwrk hottest 0.40c mean 0.37c------------------------------mean request 1331.75 ms before 573.44 envoy 449.57 sidecar 3.64 worker 305.09 resume 3.43 (control plane) +rung -1 0 qps setpoint------------------------------offered 449 /sachieved 449 /s (100.0% ok)p50 3.1 ms p95 5.6 msin-flight 28.0 pool 28.0------------------------------envoy 0.32c sidecar 0.00c------------------------------mean request 3.66 ms before 0.94 envoy 0.62 sidecar 0.83 worker 1.27 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 7.0 pool 28.0------------------------------envoy 0.59c sidecar 0.43cwrk hottest 0.24c mean 0.04c------------------------------mean request 3.03 ms before 0.98 envoy 0.68 sidecar 0.80 worker 0.57 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 5.0 pool 28.0------------------------------envoy 0.61c sidecar 0.49cwrk hottest 0.26c mean 0.04c------------------------------mean request 3.01 ms before 0.97 envoy 0.68 sidecar 0.78 worker 0.58 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1169 /sachieved 1169 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 0.69c sidecar 0.49cwrk hottest 0.25c mean 0.05c------------------------------mean request 3.00 ms before 0.97 envoy 0.71 sidecar 0.78 worker 0.53 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 10.0 pool 28.0------------------------------envoy 1.17c sidecar 0.00cwrk hottest 0.34c mean 0.07c------------------------------mean request 2.94 ms before 0.91 envoy 0.77 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 1.24c sidecar 0.90cwrk hottest 0.32c mean 0.08c------------------------------mean request 2.96 ms before 0.92 envoy 0.75 sidecar 0.80 worker 0.49 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 10.0 pool 28.0------------------------------envoy 1.22c sidecar 0.92cwrk hottest 0.34c mean 0.08c------------------------------mean request 2.99 ms before 0.94 envoy 0.78 sidecar 0.82 worker 0.44 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2560 /sachieved 2560 /s (100.0% ok)p50 3.0 ms p95 8.9 msin-flight 939 pool 939------------------------------envoy 1.66c sidecar 0.97cwrk hottest 0.40c mean 0.10c------------------------------mean request 9.50 ms before 4.14 envoy 2.17 sidecar 1.10 worker 2.08 resume 1.06 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 22.0 pool 939------------------------------envoy 1.79c sidecar 1.29cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.20 ms before 1.07 envoy 0.64 sidecar 0.81 worker 0.68 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 4.4 msin-flight 19.0 pool 939------------------------------envoy 1.67c sidecar 1.24cwrk hottest 0.67c mean 0.10c------------------------------mean request 3.28 ms before 1.13 envoy 0.58 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3756 /sachieved 3756 /s (100.0% ok)p50 3.3 ms p95 5.6 msin-flight 32.0 pool 939------------------------------envoy 2.22c sidecar 1.30cwrk hottest 0.79c mean 0.14c------------------------------mean request 3.57 ms before 1.21 envoy 0.70 sidecar 0.82 worker 0.83 resume 0.78 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.7 msin-flight 378 pool 939------------------------------envoy 2.47c sidecar 1.59cwrk hottest 0.78c mean 0.15c------------------------------mean request 4.20 ms before 1.11 envoy 0.85 sidecar 1.33 worker 0.91 resume 1.29 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 6.5 msin-flight 39.0 pool 939------------------------------envoy 2.47c sidecar 1.69cwrk hottest 0.82c mean 0.15c------------------------------mean request 3.69 ms before 1.25 envoy 0.79 sidecar 0.81 worker 0.83 resume 0.77 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4848 /sachieved 4848 /s (100.0% ok)p50 3.4 ms p95 8.4 msin-flight 44.0 pool 939------------------------------envoy 3.04c sidecar 1.73cwrk hottest 0.82c mean 0.19c------------------------------mean request 4.10 ms before 1.42 envoy 0.87 sidecar 0.81 worker 1.01 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.3 ms p95 6.0 msin-flight 67.0 pool 939------------------------------envoy 3.25c sidecar 1.92cwrk hottest 0.76c mean 0.20c------------------------------mean request 3.70 ms before 1.23 envoy 0.77 sidecar 0.82 worker 0.87 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.4 ms p95 7.5 msin-flight 1383 pool 1383------------------------------envoy 3.25c sidecar 1.95cwrk hottest 0.65c mean 0.21c------------------------------mean request 9.94 ms before 3.76 envoy 1.18 sidecar 2.66 worker 2.34 resume 2.62 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5836 /sachieved 5836 /s (100.0% ok)p50 4.2 ms p95 7.8 msin-flight 79.0 pool 1383------------------------------envoy 3.91c sidecar 2.07cwrk hottest 0.64c mean 0.25c------------------------------mean request 4.60 ms before 1.53 envoy 1.00 sidecar 0.84 worker 1.24 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 4.3 ms p95 8.8 msin-flight 321 pool 1383------------------------------envoy 4.12c sidecar 2.21cwrk hottest 0.68c mean 0.26c------------------------------mean request 5.01 ms before 1.68 envoy 1.10 sidecar 0.88 worker 1.36 resume 0.84 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6309 /sachieved 6306 /s (100.0% ok)p50 4.3 ms p95 24.1 msin-flight 750 pool 1383------------------------------envoy 4.49c sidecar 2.29cwrk hottest 0.69c mean 0.29c------------------------------mean request 8.32 ms before 3.67 envoy 1.49 sidecar 1.08 worker 2.08 resume 1.04 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7005 /s (100.0% ok)p50 5.6 ms p95 51.7 msin-flight 1273 pool 1383------------------------------envoy 5.07c sidecar 2.43cwrk hottest 0.65c mean 0.32c------------------------------mean request 13.36 ms before 4.83 envoy 3.15 sidecar 0.91 worker 4.48 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.9 ms p95 13.6 msin-flight 128 pool 1383------------------------------envoy 5.05c sidecar 2.52cwrk hottest 0.65c mean 0.32c------------------------------mean request 6.13 ms before 2.07 envoy 1.41 sidecar 0.88 worker 1.77 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 5.3 ms p95 12.2 msin-flight 925 pool 1383------------------------------envoy 4.99c sidecar 2.59cwrk hottest 0.63c mean 0.31c------------------------------mean request 6.16 ms before 2.11 envoy 1.38 sidecar 0.90 worker 1.77 resume 0.85 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7964 /sachieved 7909 /s (100.0% ok)p50 144.9 ms p95 260.9 msin-flight 2058 pool 2058------------------------------envoy 6.09c sidecar 2.41cwrk hottest 0.48c mean 0.38c------------------------------mean request 145.66 ms before 60.25 envoy 32.22 sidecar 1.27 worker 51.92 resume 1.21 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7825 /s (100.0% ok)p50 146.2 ms p95 484.7 msin-flight 4583 pool 4582------------------------------envoy 5.93c sidecar 2.42cwrk hottest 0.43c mean 0.37c------------------------------mean request 179.36 ms before 107.05 envoy 27.38 sidecar 1.36 worker 43.56 resume 1.30 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8648 /sachieved 7279 /s (99.4% ok)p50 1088.4 ms p95 2827.0 msin-flight 34k pool 34k------------------------------envoy 5.91c sidecar 2.27cwrk hottest 0.40c mean 0.37c------------------------------mean request 1331.75 ms before 573.44 envoy 449.57 sidecar 3.64 worker 305.09 resume 3.43 (control plane) +rung -1 0 qps setpoint------------------------------offered 449 /sachieved 449 /s (100.0% ok)p50 3.1 ms p95 5.6 msin-flight 28.0 pool 28.0------------------------------envoy 0.32c sidecar 0.00c------------------------------mean request 3.66 ms before 0.94 envoy 0.62 sidecar 0.83 worker 1.27 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 7.0 pool 28.0------------------------------envoy 0.59c sidecar 0.43cwrk hottest 0.24c mean 0.04c------------------------------mean request 3.03 ms before 0.98 envoy 0.68 sidecar 0.80 worker 0.57 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 5.0 pool 28.0------------------------------envoy 0.61c sidecar 0.49cwrk hottest 0.26c mean 0.04c------------------------------mean request 3.01 ms before 0.97 envoy 0.68 sidecar 0.78 worker 0.58 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1169 /sachieved 1169 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 0.69c sidecar 0.49cwrk hottest 0.25c mean 0.05c------------------------------mean request 3.00 ms before 0.97 envoy 0.71 sidecar 0.78 worker 0.53 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 10.0 pool 28.0------------------------------envoy 1.17c sidecar 0.00cwrk hottest 0.34c mean 0.07c------------------------------mean request 2.94 ms before 0.91 envoy 0.77 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 1.24c sidecar 0.90cwrk hottest 0.32c mean 0.08c------------------------------mean request 2.96 ms before 0.92 envoy 0.75 sidecar 0.80 worker 0.49 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 10.0 pool 28.0------------------------------envoy 1.22c sidecar 0.92cwrk hottest 0.34c mean 0.08c------------------------------mean request 2.99 ms before 0.94 envoy 0.78 sidecar 0.82 worker 0.44 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2560 /sachieved 2560 /s (100.0% ok)p50 3.0 ms p95 8.9 msin-flight 939 pool 939------------------------------envoy 1.66c sidecar 0.97cwrk hottest 0.40c mean 0.10c------------------------------mean request 9.50 ms before 4.14 envoy 2.17 sidecar 1.10 worker 2.08 resume 1.06 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 22.0 pool 939------------------------------envoy 1.79c sidecar 1.29cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.20 ms before 1.07 envoy 0.64 sidecar 0.81 worker 0.68 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 4.4 msin-flight 19.0 pool 939------------------------------envoy 1.67c sidecar 1.24cwrk hottest 0.67c mean 0.10c------------------------------mean request 3.28 ms before 1.13 envoy 0.58 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3756 /sachieved 3756 /s (100.0% ok)p50 3.3 ms p95 5.6 msin-flight 32.0 pool 939------------------------------envoy 2.22c sidecar 1.30cwrk hottest 0.79c mean 0.14c------------------------------mean request 3.57 ms before 1.21 envoy 0.70 sidecar 0.82 worker 0.83 resume 0.78 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.7 msin-flight 378 pool 939------------------------------envoy 2.47c sidecar 1.59cwrk hottest 0.78c mean 0.15c------------------------------mean request 4.20 ms before 1.11 envoy 0.85 sidecar 1.33 worker 0.91 resume 1.29 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 6.5 msin-flight 39.0 pool 939------------------------------envoy 2.47c sidecar 1.69cwrk hottest 0.82c mean 0.15c------------------------------mean request 3.69 ms before 1.25 envoy 0.79 sidecar 0.81 worker 0.83 resume 0.77 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4848 /sachieved 4848 /s (100.0% ok)p50 3.4 ms p95 8.4 msin-flight 44.0 pool 939------------------------------envoy 3.04c sidecar 1.73cwrk hottest 0.82c mean 0.19c------------------------------mean request 4.10 ms before 1.42 envoy 0.87 sidecar 0.81 worker 1.01 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.3 ms p95 6.0 msin-flight 67.0 pool 939------------------------------envoy 3.25c sidecar 1.92cwrk hottest 0.76c mean 0.20c------------------------------mean request 3.70 ms before 1.23 envoy 0.77 sidecar 0.82 worker 0.87 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.4 ms p95 7.5 msin-flight 1383 pool 1383------------------------------envoy 3.25c sidecar 1.95cwrk hottest 0.65c mean 0.21c------------------------------mean request 9.94 ms before 3.76 envoy 1.18 sidecar 2.66 worker 2.34 resume 2.62 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5836 /sachieved 5836 /s (100.0% ok)p50 4.2 ms p95 7.8 msin-flight 79.0 pool 1383------------------------------envoy 3.91c sidecar 2.07cwrk hottest 0.64c mean 0.25c------------------------------mean request 4.60 ms before 1.53 envoy 1.00 sidecar 0.84 worker 1.24 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 4.3 ms p95 8.8 msin-flight 321 pool 1383------------------------------envoy 4.12c sidecar 2.21cwrk hottest 0.68c mean 0.26c------------------------------mean request 5.01 ms before 1.68 envoy 1.10 sidecar 0.88 worker 1.36 resume 0.84 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6309 /sachieved 6306 /s (100.0% ok)p50 4.3 ms p95 24.1 msin-flight 750 pool 1383------------------------------envoy 4.49c sidecar 2.29cwrk hottest 0.69c mean 0.29c------------------------------mean request 8.32 ms before 3.67 envoy 1.49 sidecar 1.08 worker 2.08 resume 1.04 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7005 /s (100.0% ok)p50 5.6 ms p95 51.7 msin-flight 1273 pool 1383------------------------------envoy 5.07c sidecar 2.43cwrk hottest 0.65c mean 0.32c------------------------------mean request 13.36 ms before 4.83 envoy 3.15 sidecar 0.91 worker 4.48 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.9 ms p95 13.6 msin-flight 128 pool 1383------------------------------envoy 5.05c sidecar 2.52cwrk hottest 0.65c mean 0.32c------------------------------mean request 6.13 ms before 2.07 envoy 1.41 sidecar 0.88 worker 1.77 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 5.3 ms p95 12.2 msin-flight 925 pool 1383------------------------------envoy 4.99c sidecar 2.59cwrk hottest 0.63c mean 0.31c------------------------------mean request 6.16 ms before 2.11 envoy 1.38 sidecar 0.90 worker 1.77 resume 0.85 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7964 /sachieved 7909 /s (100.0% ok)p50 144.9 ms p95 260.9 msin-flight 2058 pool 2058------------------------------envoy 6.09c sidecar 2.41cwrk hottest 0.48c mean 0.38c------------------------------mean request 145.66 ms before 60.25 envoy 32.22 sidecar 1.27 worker 51.92 resume 1.21 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7825 /s (100.0% ok)p50 146.2 ms p95 484.7 msin-flight 4583 pool 4582------------------------------envoy 5.93c sidecar 2.42cwrk hottest 0.43c mean 0.37c------------------------------mean request 179.36 ms before 107.05 envoy 27.38 sidecar 1.36 worker 43.56 resume 1.30 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8648 /sachieved 7279 /s (99.4% ok)p50 1088.4 ms p95 2827.0 msin-flight 34k pool 34k------------------------------envoy 5.91c sidecar 2.27cwrk hottest 0.40c mean 0.37c------------------------------mean request 1331.75 ms before 573.44 envoy 449.57 sidecar 3.64 worker 305.09 resume 3.43 (control plane) +rung -1 0 qps setpoint------------------------------offered 449 /sachieved 449 /s (100.0% ok)p50 3.1 ms p95 5.6 msin-flight 28.0 pool 28.0------------------------------envoy 0.32c sidecar 0.00c------------------------------mean request 3.66 ms before 0.94 envoy 0.62 sidecar 0.83 worker 1.27 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 7.0 pool 28.0------------------------------envoy 0.59c sidecar 0.43cwrk hottest 0.24c mean 0.04c------------------------------mean request 3.03 ms before 0.98 envoy 0.68 sidecar 0.80 worker 0.57 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 5.0 pool 28.0------------------------------envoy 0.61c sidecar 0.49cwrk hottest 0.26c mean 0.04c------------------------------mean request 3.01 ms before 0.97 envoy 0.68 sidecar 0.78 worker 0.58 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1169 /sachieved 1169 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 0.69c sidecar 0.49cwrk hottest 0.25c mean 0.05c------------------------------mean request 3.00 ms before 0.97 envoy 0.71 sidecar 0.78 worker 0.53 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 10.0 pool 28.0------------------------------envoy 1.17c sidecar 0.00cwrk hottest 0.34c mean 0.07c------------------------------mean request 2.94 ms before 0.91 envoy 0.77 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 1.24c sidecar 0.90cwrk hottest 0.32c mean 0.08c------------------------------mean request 2.96 ms before 0.92 envoy 0.75 sidecar 0.80 worker 0.49 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 10.0 pool 28.0------------------------------envoy 1.22c sidecar 0.92cwrk hottest 0.34c mean 0.08c------------------------------mean request 2.99 ms before 0.94 envoy 0.78 sidecar 0.82 worker 0.44 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2560 /sachieved 2560 /s (100.0% ok)p50 3.0 ms p95 8.9 msin-flight 939 pool 939------------------------------envoy 1.66c sidecar 0.97cwrk hottest 0.40c mean 0.10c------------------------------mean request 9.50 ms before 4.14 envoy 2.17 sidecar 1.10 worker 2.08 resume 1.06 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 22.0 pool 939------------------------------envoy 1.79c sidecar 1.29cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.20 ms before 1.07 envoy 0.64 sidecar 0.81 worker 0.68 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 4.4 msin-flight 19.0 pool 939------------------------------envoy 1.67c sidecar 1.24cwrk hottest 0.67c mean 0.10c------------------------------mean request 3.28 ms before 1.13 envoy 0.58 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3756 /sachieved 3756 /s (100.0% ok)p50 3.3 ms p95 5.6 msin-flight 32.0 pool 939------------------------------envoy 2.22c sidecar 1.30cwrk hottest 0.79c mean 0.14c------------------------------mean request 3.57 ms before 1.21 envoy 0.70 sidecar 0.82 worker 0.83 resume 0.78 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.7 msin-flight 378 pool 939------------------------------envoy 2.47c sidecar 1.59cwrk hottest 0.78c mean 0.15c------------------------------mean request 4.20 ms before 1.11 envoy 0.85 sidecar 1.33 worker 0.91 resume 1.29 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 6.5 msin-flight 39.0 pool 939------------------------------envoy 2.47c sidecar 1.69cwrk hottest 0.82c mean 0.15c------------------------------mean request 3.69 ms before 1.25 envoy 0.79 sidecar 0.81 worker 0.83 resume 0.77 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4848 /sachieved 4848 /s (100.0% ok)p50 3.4 ms p95 8.4 msin-flight 44.0 pool 939------------------------------envoy 3.04c sidecar 1.73cwrk hottest 0.82c mean 0.19c------------------------------mean request 4.10 ms before 1.42 envoy 0.87 sidecar 0.81 worker 1.01 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.3 ms p95 6.0 msin-flight 67.0 pool 939------------------------------envoy 3.25c sidecar 1.92cwrk hottest 0.76c mean 0.20c------------------------------mean request 3.70 ms before 1.23 envoy 0.77 sidecar 0.82 worker 0.87 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.4 ms p95 7.5 msin-flight 1383 pool 1383------------------------------envoy 3.25c sidecar 1.95cwrk hottest 0.65c mean 0.21c------------------------------mean request 9.94 ms before 3.76 envoy 1.18 sidecar 2.66 worker 2.34 resume 2.62 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5836 /sachieved 5836 /s (100.0% ok)p50 4.2 ms p95 7.8 msin-flight 79.0 pool 1383------------------------------envoy 3.91c sidecar 2.07cwrk hottest 0.64c mean 0.25c------------------------------mean request 4.60 ms before 1.53 envoy 1.00 sidecar 0.84 worker 1.24 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 4.3 ms p95 8.8 msin-flight 321 pool 1383------------------------------envoy 4.12c sidecar 2.21cwrk hottest 0.68c mean 0.26c------------------------------mean request 5.01 ms before 1.68 envoy 1.10 sidecar 0.88 worker 1.36 resume 0.84 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6309 /sachieved 6306 /s (100.0% ok)p50 4.3 ms p95 24.1 msin-flight 750 pool 1383------------------------------envoy 4.49c sidecar 2.29cwrk hottest 0.69c mean 0.29c------------------------------mean request 8.32 ms before 3.67 envoy 1.49 sidecar 1.08 worker 2.08 resume 1.04 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7005 /s (100.0% ok)p50 5.6 ms p95 51.7 msin-flight 1273 pool 1383------------------------------envoy 5.07c sidecar 2.43cwrk hottest 0.65c mean 0.32c------------------------------mean request 13.36 ms before 4.83 envoy 3.15 sidecar 0.91 worker 4.48 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.9 ms p95 13.6 msin-flight 128 pool 1383------------------------------envoy 5.05c sidecar 2.52cwrk hottest 0.65c mean 0.32c------------------------------mean request 6.13 ms before 2.07 envoy 1.41 sidecar 0.88 worker 1.77 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 5.3 ms p95 12.2 msin-flight 925 pool 1383------------------------------envoy 4.99c sidecar 2.59cwrk hottest 0.63c mean 0.31c------------------------------mean request 6.16 ms before 2.11 envoy 1.38 sidecar 0.90 worker 1.77 resume 0.85 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7964 /sachieved 7909 /s (100.0% ok)p50 144.9 ms p95 260.9 msin-flight 2058 pool 2058------------------------------envoy 6.09c sidecar 2.41cwrk hottest 0.48c mean 0.38c------------------------------mean request 145.66 ms before 60.25 envoy 32.22 sidecar 1.27 worker 51.92 resume 1.21 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7825 /s (100.0% ok)p50 146.2 ms p95 484.7 msin-flight 4583 pool 4582------------------------------envoy 5.93c sidecar 2.42cwrk hottest 0.43c mean 0.37c------------------------------mean request 179.36 ms before 107.05 envoy 27.38 sidecar 1.36 worker 43.56 resume 1.30 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8648 /sachieved 7279 /s (99.4% ok)p50 1088.4 ms p95 2827.0 msin-flight 34k pool 34k------------------------------envoy 5.91c sidecar 2.27cwrk hottest 0.40c mean 0.37c------------------------------mean request 1331.75 ms before 573.44 envoy 449.57 sidecar 3.64 worker 305.09 resume 3.43 (control plane) +rung -1 0 qps setpoint------------------------------offered 449 /sachieved 449 /s (100.0% ok)p50 3.1 ms p95 5.6 msin-flight 28.0 pool 28.0------------------------------envoy 0.32c sidecar 0.00c------------------------------mean request 3.66 ms before 0.94 envoy 0.62 sidecar 0.83 worker 1.27 resume 0.79 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 7.0 pool 28.0------------------------------envoy 0.59c sidecar 0.43cwrk hottest 0.24c mean 0.04c------------------------------mean request 3.03 ms before 0.98 envoy 0.68 sidecar 0.80 worker 0.57 resume 0.75 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1000 /sachieved 1000 /s (100.0% ok)p50 3.0 ms p95 3.7 msin-flight 5.0 pool 28.0------------------------------envoy 0.61c sidecar 0.49cwrk hottest 0.26c mean 0.04c------------------------------mean request 3.01 ms before 0.97 envoy 0.68 sidecar 0.78 worker 0.58 resume 0.74 (control plane) +rung 0 1000 qps setpoint------------------------------offered 1169 /sachieved 1169 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 0.69c sidecar 0.49cwrk hottest 0.25c mean 0.05c------------------------------mean request 3.00 ms before 0.97 envoy 0.71 sidecar 0.78 worker 0.53 resume 0.74 (control plane) +rung 1 2000 qps setpoint [warmup]------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 10.0 pool 28.0------------------------------envoy 1.17c sidecar 0.00cwrk hottest 0.34c mean 0.07c------------------------------mean request 2.94 ms before 0.91 envoy 0.77 sidecar 0.80 worker 0.46 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.7 msin-flight 11.0 pool 28.0------------------------------envoy 1.24c sidecar 0.90cwrk hottest 0.32c mean 0.08c------------------------------mean request 2.96 ms before 0.92 envoy 0.75 sidecar 0.80 worker 0.49 resume 0.76 (control plane) +rung 1 2000 qps setpoint------------------------------offered 2000 /sachieved 2000 /s (100.0% ok)p50 2.9 ms p95 3.8 msin-flight 10.0 pool 28.0------------------------------envoy 1.22c sidecar 0.92cwrk hottest 0.34c mean 0.08c------------------------------mean request 2.99 ms before 0.94 envoy 0.78 sidecar 0.82 worker 0.44 resume 0.78 (control plane) +rung 2 3000 qps setpoint [warmup]------------------------------offered 2560 /sachieved 2560 /s (100.0% ok)p50 3.0 ms p95 8.9 msin-flight 939 pool 939------------------------------envoy 1.66c sidecar 0.97cwrk hottest 0.40c mean 0.10c------------------------------mean request 9.50 ms before 4.14 envoy 2.17 sidecar 1.10 worker 2.08 resume 1.06 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.1 ms p95 4.3 msin-flight 22.0 pool 939------------------------------envoy 1.79c sidecar 1.29cwrk hottest 0.63c mean 0.11c------------------------------mean request 3.20 ms before 1.07 envoy 0.64 sidecar 0.81 worker 0.68 resume 0.77 (control plane) +rung 2 3000 qps setpoint------------------------------offered 3000 /sachieved 3000 /s (100.0% ok)p50 3.2 ms p95 4.4 msin-flight 19.0 pool 939------------------------------envoy 1.67c sidecar 1.24cwrk hottest 0.67c mean 0.10c------------------------------mean request 3.28 ms before 1.13 envoy 0.58 sidecar 0.80 worker 0.76 resume 0.76 (control plane) +rung 3 4000 qps setpoint [warmup]------------------------------offered 3756 /sachieved 3756 /s (100.0% ok)p50 3.3 ms p95 5.6 msin-flight 32.0 pool 939------------------------------envoy 2.22c sidecar 1.30cwrk hottest 0.79c mean 0.14c------------------------------mean request 3.57 ms before 1.21 envoy 0.70 sidecar 0.82 worker 0.83 resume 0.78 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.3 ms p95 5.7 msin-flight 378 pool 939------------------------------envoy 2.47c sidecar 1.59cwrk hottest 0.78c mean 0.15c------------------------------mean request 4.20 ms before 1.11 envoy 0.85 sidecar 1.33 worker 0.91 resume 1.29 (control plane) +rung 3 4000 qps setpoint------------------------------offered 4000 /sachieved 4000 /s (100.0% ok)p50 3.2 ms p95 6.5 msin-flight 39.0 pool 939------------------------------envoy 2.47c sidecar 1.69cwrk hottest 0.82c mean 0.15c------------------------------mean request 3.69 ms before 1.25 envoy 0.79 sidecar 0.81 worker 0.83 resume 0.77 (control plane) +rung 4 5000 qps setpoint [warmup]------------------------------offered 4848 /sachieved 4848 /s (100.0% ok)p50 3.4 ms p95 8.4 msin-flight 44.0 pool 939------------------------------envoy 3.04c sidecar 1.73cwrk hottest 0.82c mean 0.19c------------------------------mean request 4.10 ms before 1.42 envoy 0.87 sidecar 0.81 worker 1.01 resume 0.77 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.3 ms p95 6.0 msin-flight 67.0 pool 939------------------------------envoy 3.25c sidecar 1.92cwrk hottest 0.76c mean 0.20c------------------------------mean request 3.70 ms before 1.23 envoy 0.77 sidecar 0.82 worker 0.87 resume 0.78 (control plane) +rung 4 5000 qps setpoint------------------------------offered 5000 /sachieved 5000 /s (100.0% ok)p50 3.4 ms p95 7.5 msin-flight 1383 pool 1383------------------------------envoy 3.25c sidecar 1.95cwrk hottest 0.65c mean 0.21c------------------------------mean request 9.94 ms before 3.76 envoy 1.18 sidecar 2.66 worker 2.34 resume 2.62 (control plane) +rung 5 6000 qps setpoint [warmup]------------------------------offered 5836 /sachieved 5836 /s (100.0% ok)p50 4.2 ms p95 7.8 msin-flight 79.0 pool 1383------------------------------envoy 3.91c sidecar 2.07cwrk hottest 0.64c mean 0.25c------------------------------mean request 4.60 ms before 1.53 envoy 1.00 sidecar 0.84 worker 1.24 resume 0.80 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6000 /sachieved 6000 /s (100.0% ok)p50 4.3 ms p95 8.8 msin-flight 321 pool 1383------------------------------envoy 4.12c sidecar 2.21cwrk hottest 0.68c mean 0.26c------------------------------mean request 5.01 ms before 1.68 envoy 1.10 sidecar 0.88 worker 1.36 resume 0.84 (control plane) +rung 5 6000 qps setpoint------------------------------offered 6309 /sachieved 6306 /s (100.0% ok)p50 4.3 ms p95 24.1 msin-flight 750 pool 1383------------------------------envoy 4.49c sidecar 2.29cwrk hottest 0.69c mean 0.29c------------------------------mean request 8.32 ms before 3.67 envoy 1.49 sidecar 1.08 worker 2.08 resume 1.04 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7005 /s (100.0% ok)p50 5.6 ms p95 51.7 msin-flight 1273 pool 1383------------------------------envoy 5.07c sidecar 2.43cwrk hottest 0.65c mean 0.32c------------------------------mean request 13.36 ms before 4.83 envoy 3.15 sidecar 0.91 worker 4.48 resume 0.86 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 6999 /s (100.0% ok)p50 4.9 ms p95 13.6 msin-flight 128 pool 1383------------------------------envoy 5.05c sidecar 2.52cwrk hottest 0.65c mean 0.32c------------------------------mean request 6.13 ms before 2.07 envoy 1.41 sidecar 0.88 worker 1.77 resume 0.84 (control plane) +rung 6 7000 qps setpoint------------------------------offered 7000 /sachieved 7000 /s (100.0% ok)p50 5.3 ms p95 12.2 msin-flight 925 pool 1383------------------------------envoy 4.99c sidecar 2.59cwrk hottest 0.63c mean 0.31c------------------------------mean request 6.16 ms before 2.11 envoy 1.38 sidecar 0.90 worker 1.77 resume 0.85 (control plane) +rung 7 8000 qps setpoint [warmup]------------------------------offered 7964 /sachieved 7909 /s (100.0% ok)p50 144.9 ms p95 260.9 msin-flight 2058 pool 2058------------------------------envoy 6.09c sidecar 2.41cwrk hottest 0.48c mean 0.38c------------------------------mean request 145.66 ms before 60.25 envoy 32.22 sidecar 1.27 worker 51.92 resume 1.21 (control plane) +rung 7 8000 qps setpoint------------------------------offered 8000 /sachieved 7825 /s (100.0% ok)p50 146.2 ms p95 484.7 msin-flight 4583 pool 4582------------------------------envoy 5.93c sidecar 2.42cwrk hottest 0.43c mean 0.37c------------------------------mean request 179.36 ms before 107.05 envoy 27.38 sidecar 1.36 worker 43.56 resume 1.30 (control plane) +rung 8 9000 qps setpoint [warmup]------------------------------offered 8648 /sachieved 7279 /s (99.4% ok)p50 1088.4 ms p95 2827.0 msin-flight 34k pool 34k------------------------------envoy 5.91c sidecar 2.27cwrk hottest 0.40c mean 0.37c------------------------------mean request 1331.75 ms before 573.44 envoy 449.57 sidecar 3.64 worker 305.09 resume 3.43 (control plane) +
+ +

Every measured rung

+
Warmup windows excluded. Latency, in-flight and memory are the worst window in the rung; +throughput and CPU are the mean across its windows. The highlighted row is each arm's last sustained rung.
+ + +
armrungofferedachievedsuccesssuccess %p50 msp95 msin-flightenvoy c meansidecar c meanenvoy mem
2c0100010001000100.00%3.13.910.00.560.5033.8M
2c1200020002000100.00%3.24.63491.040.8751.9M
2c2300030003000100.00%3.54.942.01.481.1596.9M
2c3400040194019100.00%6.324.42561.831.22168.6M
4c-1100100.0100.0100.00%3.313.028.00.080.0034.3M
4c0100010001000100.00%3.03.915.00.560.5148.8M
4c1201920192019100.00%3.04.015.01.090.8849.2M
4c2300030003000100.00%3.14.525.01.651.2449.6M
4c3405240524052100.00%3.65.841.02.211.5564.3M
4c4503650305030100.00%7.6789.046252.621.62376.5M
4c5600059945994100.00%33.1153.25752.841.65377.6M
4c670005467546699.98%2864.35747.139k3.401.172.3G
8c/2t-1375375375100.00%3.14.539.00.210.1533.9M
8c/2t0100010001000100.00%3.03.86.00.530.4835.0M
8c/2t1206720672067100.00%3.24.319.01.060.8835.1M
8c/2t2313731363136100.00%4.721.15781.541.1868.7M
8c/2t3409740944094100.00%7.953.93181.831.3167.9M
8c/2t4500037323732100.00%536.64274.226k1.901.071.5G
8c/4t-1497497497100.00%3.03.810.00.280.0074.3M
8c/4t0100010001000100.00%2.93.613.00.560.4473.6M
8c/4t1200020002000100.00%2.93.966.01.120.8974.9M
8c/4t2318031793179100.00%3.24.931.01.691.2174.6M
8c/4t3414141414141100.00%3.45.61992.261.4975.2M
8c/4t4500050005000100.00%3.99.111442.731.73110.4M
8c/4t560005996598099.73%8.4161.119353.151.88328.2M
8c/4t6700054845484100.00%3060.65954.741k3.401.172.5G
8c/8t-1296296296100.00%3.16.445.00.220.0362.9M
8c/8t0105710571057100.00%3.03.821.00.610.5171.8M
8c/8t1206920692069100.00%2.93.821.01.200.9372.9M
8c/8t2300030003000100.00%3.04.032.01.811.3080.5M
8c/8t3404440444044100.00%3.25.62762.391.63106.9M
8c/8t4512651255125100.00%3.621.285.02.971.92107.1M
8c/8t5600060006000100.00%3.919.91163.652.21106.6M
8c/8t670757061701699.37%7.081.718654.392.45201.4M
8c/8t780006543654299.98%2758.76534.934k4.721.572.2G
16c0114511451145100.00%3.14.33930.700.51125.3M
16c1208020802080100.00%3.03.979.01.260.87143.7M
16c2300030003000100.00%3.14.910411.841.27191.5M
16c3410741054105100.00%4.0213.915942.381.55230.2M
16c4500050025002100.00%6.326.01942.821.77229.2M
16c5612961186118100.00%15.5320.131123.862.08330.8M
16c670847084707199.82%5.751.614734.592.38331.0M
16c780008000798099.75%8.570.918625.782.74333.1M
16c8900090899089100.00%363.0611.051757.282.44549.0M
16c pinned-1449449449100.00%3.15.628.00.320.0061.1M
16c pinned0105610561056100.00%3.03.711.00.630.4766.6M
16c pinned1200020002000100.00%2.93.811.01.230.9167.1M
16c pinned2300030003000100.00%3.24.422.01.731.27168.5M
16c pinned3400040004000100.00%3.36.53782.471.64176.7M
16c pinned4500050005000100.00%3.47.513833.251.94212.4M
16c pinned5615561536153100.00%4.324.17504.302.25213.7M
16c pinned6700070017001100.00%5.651.712735.032.51217.6M
16c pinned7800078257825100.00%146.2484.745835.932.42494.7M
+ +