IP mining: Extracting knowledge from the dynamics of the Internet addressing space

Abstract

Going back to the Internet of one decade ago, HTTP-based content and web services were provided by centralized or barely distributed servers. Single hosts providing exclusive services at fixed IP addresses was the standard approach. Current situation has drastically changed, and the mapping of IPs to different content and services is nowadays extremely dynamic. The adoption of large CDNs by major Internet players, the extended usage of transparent content caching, the explosion of Cloud-based services, and the decoupling between content providers and the hosting infrastructure have created a difficult to manage Internet landscape. Understanding such a complex scenario is paramount for network operators, both to control the traffic on their networks and to improve the quality experienced by their customers, specially when something goes wrong. Using a full week of HTTP traffic traces collected at the mobile broadband network of a major European ISP, this paper studies the associations between web services, the hosting organizations/ASes, and the content servers' IPs. By mining correlations among these, we extract useful insights about the dynamics of the IP addressing space used by the top web services, and the way content providers and hosting organizations deliver their services to the mobile end-users. The extracted knowledge is applied on two specific use-cases, the former on hosting and service delivery characterization, the latter on automatic IP-based HTTP services classification.