Skip to content

advanced-tools.md

Latest commit

 

History

History
176 lines (127 loc) · 5.31 KB

advanced-tools.md

File metadata and controls

176 lines (127 loc) · 5.31 KB
Raw

Advanced Testing Tools

The iperf3-based approach in this repo is practical for commodity hardware. For higher-precision testing, line-rate saturation at 10G+ with small packets, or application-layer benchmarking, the tools below are the industry standard.

DPDK-based traffic generators

Standard Linux networking uses kernel interrupt processing. At small packet sizes, this saturates CPU before the NIC is saturated — this is why iperf3 shows reduced throughput at 128-byte. DPDK bypasses the kernel entirely, polling the NIC in userspace to achieve true line-rate at any packet size.

flowchart LR
    subgraph kernel["Standard Linux (iperf3)"]
        nic1["NIC"] -->|"interrupt"| kernel_net["Kernel\nnetwork stack"] --> app["iperf3\nuserspace"]
    end

    subgraph dpdk_stack["DPDK-based (TRex / Pktgen)"]
        nic2["NIC"] -->|"poll mode\n(no kernel)"| dpdk["DPDK PMD\nuserspace"] --> gen["TRex /\nPktgen"]
    end
Loading

Cisco TRex

Open-source, DPDK-based. The most capable and widely used software traffic generator. Used by FD.io CSIT for all VPP benchmarks.

Capabilities:

  • Stateless mode: raw L2/L3 packet generation at line rate (10G, 40G, 100G)
  • ASTF (Advanced Stateful) mode: full TCP/HTTP connection simulation
  • Built-in RFC 2544 test suite
  • Lua scripting for custom traffic profiles
  • Real-time statistics dashboard
# Install TRex
wget https://trex-tgn.cisco.com/trex/release/latest_release
tar -xzf v*.tar.gz

# Basic stateless run — 10Gbps bidirectional IMIX
./t-rex-64 -f cap2/imix_64.yaml -d 60 -m 1

# RFC 2544 automated test
./t-rex-64 --cfg /etc/trex_cfg.yaml --rfc2544 -f cap2/64b.yaml

Requirements: Two dedicated 10G Intel NICs (82599/X710/XL710) bound to DPDK via dpdk-devbind. Cannot share NICs with the OS.

Pktgen-DPDK

Simpler than TRex — focused on maximum raw PPS (packets per second) with fine-grained control over packet fields. Ideal for measuring exact PPS limits at specific packet sizes.

# Launch Pktgen with two ports
./pktgen -l 0-7 -n 4 -- -P -m "1.0, 2.1" -f txrx.lua

Ostinato

GUI-based packet crafter and traffic generator. Build custom packets byte-by-byte and send at controlled rates. Useful for:

  • Protocol fuzzing
  • Custom header testing
  • RFC 2544 automation via Python API

Application-layer and bufferbloat testing

Flent (Flexible Network Tester)

Tests bufferbloat — how much latency and jitter spike when the network is fully saturated. This measures a different dimension from raw throughput: whether the network remains usable under load.

pip install flent
# RRUL test — simultaneous TCP up/down + latency measurement
flent -H 192.168.100.11 rrul -l 60 -o results.png

RRUL (Realtime Response Under Load): Runs 4 TCP upload + 4 TCP download streams simultaneously while measuring ICMP RTT every 100ms. A well-tuned router with proper QoS should maintain <10ms added latency; an untuned router may show hundreds of milliseconds.

Apache JMeter / wrk

For Layer-7 testing — load balancer throughput, WAF performance, HTTP/HTTPS request rates.

# wrk: 12 threads, 400 connections, 30 seconds
wrk -t12 -c400 -d30s http://192.168.100.1/

MTR (Matt's Traceroute)

Combines ping and traceroute into continuous per-hop latency and loss measurement. Useful for identifying which hop in a path is introducing latency or loss under load.

mtr --report --report-cycles 100 192.168.100.1

Network impairment simulation

Testing maximum throughput is only half the picture. Real networks have latency, jitter, loss, and reordering. Test how VyOS handles these with Linux's built-in traffic control tools.

netem (Network Emulator)

tc with the netem discipline injects impairments on a Linux interface:

# Add 50ms RTT delay + 10ms jitter to eth1
sudo tc qdisc add dev eth1 root netem delay 50ms 10ms

# Add 1% random packet loss
sudo tc qdisc add dev eth1 root netem loss 1%

# Add delay + loss + 0.1% duplication
sudo tc qdisc add dev eth1 root netem delay 50ms loss 1% duplicate 0.1%

# Remove all impairments
sudo tc qdisc del dev eth1 root

Impairment test playbook pattern:

Place a Linux node between the sender and DUT. Use netem on that node to simulate WAN conditions. Benchmark VyOS behaviour under each scenario:

Scenario netem parameters
LAN (baseline) no impairment
Good WAN delay 20ms
Poor WAN delay 100ms loss 0.5%
Terrible WAN delay 200ms 50ms loss 2% corrupt 0.1%

FD.io CSIT — the pre-packaged E2E suite

FD.io CSIT (Continuous System Integration and Testing) is the open-source benchmark suite used to test VPP (Vector Packet Processing) and DPDK-based data planes. It is the closest thing to a download-and-run RFC 2544 suite.

  • Test automation: Robot Framework
  • Traffic generation: Cisco TRex (DPDK)
  • Output: NDR (Non-Drop Rate) and PDR (Partial Drop Rate) per RFC 2544
  • Reports: HTML/PDF with throughput, latency, and loss curves
# Clone CSIT
git clone https://github.com/FDio/csit.git

# Run a VPP benchmarking scenario (requires TRex + DUT node)
cd csit
./resources/tools/testbed-setup/ansible/deploy_local.sh

CSIT is complex to set up but produces publication-quality results. See the FD.io CSIT docs for setup guides.