Features
Release 1.4 ("Eunomia") features
-
Fully automated, end-to-end testing: NSTAT is responsible for automating
and sequencing every single step of the testing procedure, including:
- component lifecycle management and orchestration: build, start, stop, etc., and synchronize the controller, emulator, NB apps.
- test dimensioning: iterate the test over a large space of experimental parameters.
- online monitoring and sampling: additional threads are being launched in the background to monitor component progress and collect performance results and runtime statistics.
- reporting: report generation and result plotting.
-
Exhaustive stress testing: within a single run, NSTAT repeats the test over an extensive experimental space defined by the cartesian product of all test dimensions, e.g. the number of switches simulated by the SB emulator, the number of simulated hosts, etc.. This allows the user to express and execute multiple test iterations into a single run, and to easily discover controller behavior at extreme cases.
-
Scalable traffic generation: NSTAT provides two different tools, to generate traffic on the SouthBound controller's interface, the MT-Cbench, based on Cbench and a traffic generator based on Multinet.
- MT-Cbench, a multi-threaded implementation of Cbench, aiming at emulating large-scale SDN topologies with thousands of switches and bring controller performance to its limits.
- multinet is a tool used to create large scale topologies, based on mininet API. Apart from this core functionality, it has the capability of triggering traffic to the controller as PacketIN messages with ARP payload.
-
Unification of stress tests: NSTAT unifies existing and new test cases and tools under a common environment. A summary of the supported tests is presented here.
-
Easy and rich configuration system: NSTAT allows the user to easily specify the test configuration parameters using a single JSON file. Nearly all test components and procedures from component orchestration to plotting are largely described by these configuration parameters.
-
Comprehensive reporting and flexible plotting system: NSTAT provides comprehensive output in a single HTML report, including all the information required to fully describe a test. It is also supported by a flexible plot generation methodology (within the configuration parameters JSON file) which allows the user to easily specify plot types and parameters.
-
Modular and extensible architecture: for most cases, NSTAT is not tightly coupled to a specific controller or emulator implementation or version. Rather, it is pluggable to any controller/emulator provided it adheres to certain conventions. Furthermore, extending the existing test dimensions and result statistics is rather easy and doesn't affect existing logic.
-
Multi-node test execution: NSTAT allows the execution of its basic components (controller, SouthBound emulator, NorthBound generator) on different nodes. Every component can be built, started, executed and monitored on a node (e.g. physical or virtual machine) specified in the configuration file, which is assumed to be already up and running. NSTAT itself may run on a separate node from which it orchestrates and monitors all other components. To make this possible, nodes are assumed to support SSH connectivity and have already checked out an identical instance of NSTAT source tree.
-
Fast provisioning with Docker containers: Easy provisioning of the environment is provided through the use of Docker containers, that can be deployed either with the use of Dockerfiles or can be downloaded directly from DockerHub.
See the future releases page