Anomaly Detection for Black Box Services in Edge Clouds Using Packet Size Distribution

Future services in fields like autonomous driving and virtual reality rely on cloud computing resources located at the edge of Internet Service Provider(ISP) networks. Instead of deploying many service-specific monitoring and reliability platforms, a centralized monitoring solution can reduce the usage of the already sparse edge cloud resources. The ISP can offer such a service using the black box monitoring approach presented in this paper. Current cloud providers already collect data about customer services for cloud performance and cloud reliability. We propose to extend current monitoring solutions for virtual machines by real-time analysis of network packet headers. In particular, we use the packet size distribution and the TCP connection time to infer the operational state of the service. We conduct an evaluation of the presented approach using a content delivery system which is set into different load and anomaly states. The random forest algorithm trained to differentiate normal from abnormal service states based on the collected data resulted in an accuracy of 94%. The overhead of collecting the data on a commodity hardware hypervisor using eBPF is about 3% CPU at 10GB/s.