Towards Fine-Grained, High-Coverage Internet Monitoring at Scale

Abstract

The massiveness of the Internet makes it rather difficult to achieve high-coverage monitoring at scale with reasonable overhead. The traditional wisdom for scalable and high-coverage Internet monitoring is to consider clients in each /24 as a whole and only monitor the representatives, either by active probing or by passive traffic sniffing, such that performance of the rest can be predicted for high coverage. There are two basic assumptions behind this traditional wisdom: 1) clients in the same /24 have similar performance, and 2) tracking all targeted /24s equates to full-coverage monitoring. With the increasing prevalence of load balancing, both assumptions are now questionable. Through large-scale measurements, we evaluate the coverage and predictability issues of current practices, motivate the necessity of link-level fine-grained, high-coverage monitoring, and present new insights on how to achieve it. Our key findings are: 1) the current practices using the representatives of /24s may fail to capture the changes of up to 85% of links in the Internet; 2) the path difference between client flows to the same /24 is both significant and prevalent; 3) it is possible to cover most of the visible links from DCs to both small and large prefixes by carefully choosing client flows; 4) high-coverage monitoring can be achieved with at least three times less overhead than direct link monitoring.