Hi-Clust: Unsupervised Analysis of Cloud Latency Measurements Through Hierarchical Clustering

Abstract

Latency is nowadays one of the most relevant network and service performance metrics reflecting end-user experience. With the wide adoption and deployment of delay-sensitive applications in the Cloud (e.g., gaming, interactive video conferencing, corporate services, etc.), monitoring and analysis of Cloud service latency is becoming increasingly relevant for Cloud service providers, tenants and even users. Traditional network monitoring approaches based on time-series analysis and thresholding are capable of raising alarms when anomalous events arise, but are not applicable to detect correlations among multiple monitored dimensions, necessary to provide an adequate interpretation of an anomaly. In this paper we present Hi-Clust, an unsupervised-based approach for analyzing and interpreting anomalies in multi-dimensional network data, through the application of hierarchical clustering techniques. While Hi-Clust is applicable to the analysis of different types of nested or hierarchically structured data, we particularly focus on the analysis of Cloud service latency, using active measurements collected from geographically distributed vantage points. We implement and benchmark multiple density-based clustering approaches for Hi-Clust over four weeks of real multidimensional Cloud service latency measurements. Using the most robust underlying clustering algorithm from the benchmark, we show how to automatically extract and interpret anomalous Cloud service behavior with Hi-Clust. In addition, we show the advantages of Hi-Clust over traditional threshold-based approaches for detecting and interpreting anomalous behavior, through practical examples over the collected measurements.