Optics in Data Centers: Adapting to Diverse Modern Workloads

Over the recent years we witness a massive growth of cloud usage, accelerated by new types of 'born-to-the-cloud' workloads. These new types of workloads are increasingly multi-component, dynamic and often present highly intensive communication patterns. Massive innovation of Data Center Network (DCN) technologies is required to support the demand, giving raise to new network topologies, new network control paradigms, and management models. One particularly promising technology candidate for improving the DCN efficiency is Optical Circuit Switching (OCS). Several hybrid solutions combining OCS with the traditional Electronic Packet Switching (EPS) have been proposed [1, 2], aiming to take advantage of the benefits of the OCS technology (e.g., high bandwidth, low latency and power consumption) while leveling out its shortcomings (e.g., slow reconfiguration time, integration with IP fabric). The first comprehensive work advocating OCS for DCN [1] considered HPC workloads with semi-static communication patterns. Follow up works, such as Helios [2], proposed new ways for identifying heavy flows, heuristics for computing the circuits configuration, and control hooks for dispatching the traffic over EPS and OCS paths. In yet newer works, e.g. [3], further advances were made -- supporting richer sets of communication patterns, employing Software Defined Networking (SDN) to steer the traffic and to achieve more reactive control planes in anticipation for faster OCS capabilities, and more. We observe that in hybrid solutions, the basic approach remains the same -- the network is partitioned between the two separate fabrics, one based on OCS and one based on EPS, so that each network flow is handled by one of the fabrics, depending on its properties. In this work, we present a new architecture where optical circuitry does not merely augment the EPS but is properly integrated with it into a coherently managed unified fabric. Our approach is based on understanding that modern workloads impose diverse traffic demands. Specifically, we identify the abundance of few-to-many and many-to-few communication patterns with multiple dynamic hot spots and observe that such traffic is better served by tighter integration of OCS and EPS achieved through introducing composite paths across the OCS-EPS boundaries. As a preliminary proof of concept, we have evaluated our architecture and compared it to the previously proposed hybrid solutions, considering the known uniform and skewed, as well as few-to-many and many-to-few demand models. For each traffic pattern, we evaluate both whether it can be met by each of the solutions and, if yes, the resulting link utilization. Our preliminary results show a significant improvement in both these metrics -- the feasibility and the link utilization. Looking forward, we plan to expand this research and explore a new thread of opportunities for leveraging the reconfiguration capabilities of contemporary OCS, posing it as a viable DCN technology. This research is partially supported by the European Communitys Seventh Framework Programme (FP7/2001-2013) under grant agreement no. 619572 (COSIGN Project).