Testing the Fidelity of an Emulab Testbed

Abstract

Studying the resilience of complex systems and Critical Infrastructures (CI), e.g., the Internet, in order to improve protection and response mechanisms is an important research activity due to their vital role in modern economy and society. Such studies are frequently based on experimentation using a) real systems, b) software simulation or c) hardware emulation. In this paper we present how our emulation testbed, based on Emulab, is able to realistically reproduce real system configurations (fidelity or system representativeness). We compare experimental results between two different emulation configurations against a reference configuration without use of emulation (real). Our results lead to two main contributions. First, we confirm that the current trend of using emulation testbeds is justified as both realistic and efficient. We highlight the fact that Emulab-based configurations are representative of real systems in terms of emerging behavior (qualitative) and that the interpretation of experimental results should not be based on absolute numbers, e.g., performance metrics, because exact values are highly hardware dependent. Secondly, we indicate that users of Emulab-based testbeds should favor the ”delay-nodeshaping” rather than the ”end-node-shaping” strategy because it frequently leads to more consistent results.