CloudJoin: Experimenting at scale with Hybrid Cloud Computing

Abstract

To continue innovating in an age of at-scale computer systems research, the academic computing and networking systems research community must explore new approaches to addressing growing researcher demands to support larger size experiments. CloudJoin explores a transformational approach to scaling out successful Computing Research Infrastructures (CRI) into larger testbeds by creating hybrid cloud computing systems. We describe how to create a seamless, scalable, single experiment testbed that spans CloudLab and the Google Cloud Platform (GCP), while requiring no infrastructure changes. In addition to added elastic computing capacity, CloudJoin experiments benefit from easy access specialized hardware and cloud services and APIs to leverage world class data analytics and experimental infrastructure monitoring. In this work-in-progress, we show how to integrate the infrastructures by creating a Virtual Private Network between a CloudLab experiment and a GCP Virtual Private Cloud (VPC). To simplify understanding of large-scale experiment behavior, problem diagnosing and debugging, we also demonstrate how to use scalable, single dashboard cloud monitoring and logging tools across the hybrid testbed infrastructure.11This material is based upon work supported by the National Science Foundation under Grant No. CNS-1923692