Pushing the envelope in distributed ns-3 simulations: one billion nodes

In this paper, we describe the results of simulation of very large (up to 109 nodes), planetary-scale networks using ns-3 simulator. The modeled networks consist of the small-world core graph of network routers and an equal number of the leaf nodes (one leaf node per router). Each bidirectional link in the simulation carries on-off traffic. Using LLNL's high-performance computing (HPC) clusters, we conducted strong and weak scaling studies, and investigated on-node scalability for MPI nodes. The scaling relations for both runtime and memory are derived. In addition we examine the packet transmission rate in the simulation and its scalability. Performance of the default ns-3 parallel scheduler is compared to the custom-designed NULL-message scheduler.