On the Unreliability of Network Simulation Results FROM Mininet and iPerf

The networking research community and industry both use the Mininet network simulator to assess new network architectures and predict how new designs will perform. Mininet employs an emulation approach that uses concurrent processes running in a single computer instead of physical hardware. A user specifies a network consisting of network switches, routers, and host computers that have network interface connections and links connecting all the pieces. Mininet creates software artifacts to represent each of the network devices and allows application programs to send packets across the resulting network. Researchers often use the iPerf application to measure network performance. Many research papers report results from Mininet and iPerf and use the results to validate new designs for Software-Defined Networks (SDNs). However, there has been little investigation into the scenarios where these emulations can perform different than intended. The goal of this paper is to understand the edge cases of these emulation methods and understand the severity of these scenarios. This paper reports surprising anomalies in the results of Mininet and iPerf. We show that the choice of apparently valid configuration options can make the reported throughput completely invalid. Our initial discoveries focused on a complex simulation of a data center network. However, we were able to show that Mininet produces completely invalid results for a basic case: network traffic traveling across a single emulated link between two switches with no other network traffic. The paper makes recommendations for ways to configure Mininet to avoid some of the anomalies.