The cloud to things continuum: infrastructure and management

The advent of IoT has seen an explosive growth in the number of connected devices generating a large variety of data in high volumes at high velocities. The unique set of requirements posed by IoT demands innovation in the communication and information infrastructure. Such infrastructure should be massively distributed for achieving scalability; highly interoperable for seamless interaction between different enabling technologies; highly flexible for collecting, fusing, mining, and processing IoT data; and easily programmable for service orchestration and application-enablement. This talk will discuss our experience developing and operating such an infrastructure across Canada as part of the Smart Applications on Virtualized Infrastructure project, highlighting some of the underlying management challenges. In particular, traditional human-in-the-loop management operations are costly, error-prone, and slow to adapt to changes. Automation can address these challenges and has been the holy grail of network management research for decades, however with limited practical deployment. Several factors can be attributed to this, including the existence of many stakeholders with conflicting goals, reliance on proprietary solutions, the inability to process network monitoring data at scale, and the lack of global visibility restricting network-wide optimizations. We believe that the stars are now aligned to realize this long-term vision thanks to advances in network softwarization, recent breakthroughs in machine learning, and the availability of large-scale data processing platforms. This talk will discuss some of the research directions in this perspective with particular focus on programmable network monitoring leveraging network softwarization, predictive machine learning for automated management decision making, and on-demand orchestration of network services.