diff --git a/performance/comparison.md b/performance/comparison.md new file mode 100644 index 00000000..94752900 --- /dev/null +++ b/performance/comparison.md @@ -0,0 +1,207 @@ +[toc] + + + +# Introduction + +This file shows comparisons between Distrinet and Distrinet after docker replacement, which we call as "Docker-based Distrinet". We compare these two tools from 3 aspects, performance, scale and time. + +# Performance Comparison + +We tested the performances of these examples: a linear topology with 6 hosts and 6 switches; a binary tree whose depth is 3; customized topology. + +We care about these metrics: the connection of the topology and the bandwidth of iperf. + +"Pingall and links" are metrics to test the connection of the topology. + +"Worker containers" part shows lxc/docker containers created in worker host. + +"Iperf" is used to test the bandwidth. + +## Linear topology (6 hosts,6 switches) + +There are 6 hosts and 6 switches in the topology. + +### LXC-based Distrinet + +- pingall + +![](./img/lxd/6pingall.png) + +- links + +![](./img/lxd/6links.png) + +- worker containers + +![](./img/lxd/6worker_containers.png) + +- iperf + +![](./img/lxd/6iperf.png) + +### Docker-based Distrinet + +- pingall + +![](./img/docker/6pingall.png) + +- links + +![](./img/docker/6links.png) + +- worker containers + +![](./img/docker/6worker_containers.png) + +- iperf + +![](./img/docker/6iperf.png) + +## Binary tree (depth = 3) + +There are 8 hosts and 7 switches in the tree topology. + +### LXC-based Distrinet + +- pingall + +![](./img/lxd/tree_32_pingall.png) + +- links & net + +![](./img/lxd/tree_32_links_and_net.png) + +- worker containers + +![](./img/lxd/tree_32_worker_containers.png) + +- iperf + +![](./img/lxd/tree_32_iperf.png) + +### Docker-based Distrinet + +- pingall + +![](./img/docker/tree_32_pingall.png) + +- links & net + +![](./img/docker/tree_32_links_and_net.png) + +- worker containers + +![](./img/docker/tree_32_worker_containers.png) + +- iperf + +![](./img/docker/tree_32_iperf.png) + +## Customized topology + +### Illustrstion + +There are 8 hosts and 10 switches in the customized topology. + +![](./img/dc_topo.png) + +### LXC-based Distrinet + +- pingall + +![](./img/lxd/custom_pingall.png) + +- links & net + +![](./img/lxd/custom_links.png) + +- worker containers + +![](./img/lxd/custom_container.png) + +- iperf + +![](./img/lxd/custom_iperf.png) + +### Docker-based Distrinet + +- pingall + +![](./img/docker/custom_pingall.png) + +- links & net + +![](./img/docker/custom_links.png) + +- worker containers + +![](./img/docker/custom_container.png) + +- iperf + +![](./img/docker/custom_iperf.png) + +## Conclusion + +These tests show that Docker-based Distrinet can establish all kinds of connected topology. Besides, we illustrate iperf performance as follows. However, the iperf performance of Docker-based Distrinet is worse for some reason. + +- Linear topology: + +![](./img/iperf_linear.png) + +- Binary tree topology: + +![](./img/iperf_tree.png) + +- Customized topology: + +![](./img/iperf_custom.png) + +# Scale Comparison + +We test **the maximum scale of linear topology** that two tools can support. It shows that after the docker replacement, we can establish much larger scale than before. + +## LXC-based Distrinet: between 90 and 99 + +90 hosts and 90 switches can be established. + +![](./img/lxd/90.png) + +However, when establishing 99 hosts and 99 switches, this error occurred. + +![](./img/lxd/99.png) + +## docker container: at least 400 + +300 hosts and 300 switches can be established. + +![](./img/docker/300start.png) + +400 hosts and 400 switches can be established. However, some packages are lost. + +![](./img/docker/400ping.png) + +# Time Comparison + +Otherwise, we added a new module to test the time to establish and stop the topology. And we noticed that after the docker replacement, time performance is better. + +We show average start/stop time in the chart and figure below. + +| topology type | LXC-based Distrinet start time(ms) | Docker-based Distrinet start time(ms) | LXC-based Distrinet stop time(ms) | Docker-based Distrinet stop time(ms) | +| -------------------------- | ---------------------------------- | ------------------------------------- | --------------------------------- | ------------------------------------ | +| linear_10 | 43283.724 | 39618.796 | 17412.770 | 14586.549 | +| linear_20 | 85042.979 | 74643.517 | 29419.694 | 29851.002 | +| linear_30 | 145570.518 | 116308.483 | 51053.180 | 49878.966 | +| linear_40 | 189553.188 | 150765.474 | 73760.429 | 71019.690 | +| full_tree_depth=2_degree=2 | 21257.863 | 17055.762 | 9104.821 | 6914.141 | +| full_tree_depth=2_degree=3 | 30518.782 | 29184.385 | 13705.951 | 13014.671 | +| full_tree_depth=3_degree=2 | 33082.372 | 31833.954 | 14239.144 | 11628.452 | +| full_tree_depth=3_degree=3 | 81417.790 | 75521.407 | 28824.049 | 29416.155 | +| custom | 46788.506 | 45623.731 | 22290.356 | 18247.342 | + +![](./img/time.png) + +For example, we notice that with the expansion of linear topology's scale, LXC-based Distrinet need much more time to establish the topology than Docker-based Distrinet. + +![](./img/linear_time.png) \ No newline at end of file diff --git a/performance/img/dc_topo.png b/performance/img/dc_topo.png new file mode 100644 index 00000000..724f4e13 Binary files /dev/null and b/performance/img/dc_topo.png differ diff --git a/performance/img/docker/300start.png b/performance/img/docker/300start.png new file mode 100644 index 00000000..3ccec77c Binary files /dev/null and b/performance/img/docker/300start.png differ diff --git a/performance/img/docker/400ping.png b/performance/img/docker/400ping.png new file mode 100644 index 00000000..b202d5d1 Binary files /dev/null and b/performance/img/docker/400ping.png differ diff --git a/performance/img/docker/6iperf.png b/performance/img/docker/6iperf.png new file mode 100644 index 00000000..d9ee1eb8 Binary files /dev/null and b/performance/img/docker/6iperf.png differ diff --git a/performance/img/docker/6links.png b/performance/img/docker/6links.png new file mode 100644 index 00000000..f9dd77be Binary files /dev/null and b/performance/img/docker/6links.png differ diff --git a/performance/img/docker/6pingall.png b/performance/img/docker/6pingall.png new file mode 100644 index 00000000..20d88d76 Binary files /dev/null and b/performance/img/docker/6pingall.png differ diff --git a/performance/img/docker/6worker_containers.png b/performance/img/docker/6worker_containers.png new file mode 100644 index 00000000..e88109c2 Binary files /dev/null and b/performance/img/docker/6worker_containers.png differ diff --git a/performance/img/docker/custom_container.png b/performance/img/docker/custom_container.png new file mode 100644 index 00000000..36cff32b Binary files /dev/null and b/performance/img/docker/custom_container.png differ diff --git a/performance/img/docker/custom_iperf.png b/performance/img/docker/custom_iperf.png new file mode 100644 index 00000000..dbfebde4 Binary files /dev/null and b/performance/img/docker/custom_iperf.png differ diff --git a/performance/img/docker/custom_links.png b/performance/img/docker/custom_links.png new file mode 100644 index 00000000..edc3436f Binary files /dev/null and b/performance/img/docker/custom_links.png differ diff --git a/performance/img/docker/custom_pingall.png b/performance/img/docker/custom_pingall.png new file mode 100644 index 00000000..66f8d8df Binary files /dev/null and b/performance/img/docker/custom_pingall.png differ diff --git a/performance/img/docker/tree_32_iperf.png b/performance/img/docker/tree_32_iperf.png new file mode 100644 index 00000000..d91412ce Binary files /dev/null and b/performance/img/docker/tree_32_iperf.png differ diff --git a/performance/img/docker/tree_32_links_and_net.png b/performance/img/docker/tree_32_links_and_net.png new file mode 100644 index 00000000..cac184fb Binary files /dev/null and b/performance/img/docker/tree_32_links_and_net.png differ diff --git a/performance/img/docker/tree_32_pingall.png b/performance/img/docker/tree_32_pingall.png new file mode 100644 index 00000000..052dca44 Binary files /dev/null and b/performance/img/docker/tree_32_pingall.png differ diff --git a/performance/img/docker/tree_32_worker_containers.png b/performance/img/docker/tree_32_worker_containers.png new file mode 100644 index 00000000..defb1b35 Binary files /dev/null and b/performance/img/docker/tree_32_worker_containers.png differ diff --git a/performance/img/iperf_custom.png b/performance/img/iperf_custom.png new file mode 100644 index 00000000..f33ff4cc Binary files /dev/null and b/performance/img/iperf_custom.png differ diff --git a/performance/img/iperf_linear.png b/performance/img/iperf_linear.png new file mode 100644 index 00000000..62b82b5b Binary files /dev/null and b/performance/img/iperf_linear.png differ diff --git a/performance/img/iperf_tree.png b/performance/img/iperf_tree.png new file mode 100644 index 00000000..043ca6de Binary files /dev/null and b/performance/img/iperf_tree.png differ diff --git a/performance/img/linear_time.png b/performance/img/linear_time.png new file mode 100644 index 00000000..1a0630d9 Binary files /dev/null and b/performance/img/linear_time.png differ diff --git a/performance/img/lxd/6iperf.png b/performance/img/lxd/6iperf.png new file mode 100644 index 00000000..d4f78c8a Binary files /dev/null and b/performance/img/lxd/6iperf.png differ diff --git a/performance/img/lxd/6links.png b/performance/img/lxd/6links.png new file mode 100644 index 00000000..98cbf4a8 Binary files /dev/null and b/performance/img/lxd/6links.png differ diff --git a/performance/img/lxd/6pingall.png b/performance/img/lxd/6pingall.png new file mode 100644 index 00000000..681e054e Binary files /dev/null and b/performance/img/lxd/6pingall.png differ diff --git a/performance/img/lxd/6worker_containers.png b/performance/img/lxd/6worker_containers.png new file mode 100644 index 00000000..999c6a23 Binary files /dev/null and b/performance/img/lxd/6worker_containers.png differ diff --git a/performance/img/lxd/90.png b/performance/img/lxd/90.png new file mode 100644 index 00000000..a921eacb Binary files /dev/null and b/performance/img/lxd/90.png differ diff --git a/performance/img/lxd/99.png b/performance/img/lxd/99.png new file mode 100644 index 00000000..843859b8 Binary files /dev/null and b/performance/img/lxd/99.png differ diff --git a/performance/img/lxd/custom_container.png b/performance/img/lxd/custom_container.png new file mode 100644 index 00000000..cae778a4 Binary files /dev/null and b/performance/img/lxd/custom_container.png differ diff --git a/performance/img/lxd/custom_iperf.png b/performance/img/lxd/custom_iperf.png new file mode 100644 index 00000000..4e078351 Binary files /dev/null and b/performance/img/lxd/custom_iperf.png differ diff --git a/performance/img/lxd/custom_links.png b/performance/img/lxd/custom_links.png new file mode 100644 index 00000000..e7e8cfe4 Binary files /dev/null and b/performance/img/lxd/custom_links.png differ diff --git a/performance/img/lxd/custom_pingall.png b/performance/img/lxd/custom_pingall.png new file mode 100644 index 00000000..4895a63f Binary files /dev/null and b/performance/img/lxd/custom_pingall.png differ diff --git a/performance/img/lxd/tree_32_iperf.png b/performance/img/lxd/tree_32_iperf.png new file mode 100644 index 00000000..62709db1 Binary files /dev/null and b/performance/img/lxd/tree_32_iperf.png differ diff --git a/performance/img/lxd/tree_32_links_and_net.png b/performance/img/lxd/tree_32_links_and_net.png new file mode 100644 index 00000000..16ea8a42 Binary files /dev/null and b/performance/img/lxd/tree_32_links_and_net.png differ diff --git a/performance/img/lxd/tree_32_pingall.png b/performance/img/lxd/tree_32_pingall.png new file mode 100644 index 00000000..db959989 Binary files /dev/null and b/performance/img/lxd/tree_32_pingall.png differ diff --git a/performance/img/lxd/tree_32_worker_containers.png b/performance/img/lxd/tree_32_worker_containers.png new file mode 100644 index 00000000..2327f2a5 Binary files /dev/null and b/performance/img/lxd/tree_32_worker_containers.png differ diff --git a/performance/img/time.png b/performance/img/time.png new file mode 100644 index 00000000..16f2950b Binary files /dev/null and b/performance/img/time.png differ