Measuring web similarity from dual-stacked hosts

We compare the similarity of webpages delivered over IPv4 and IPv6. Using the SamKnows web performance (webget) test, we implemented an extension (simweb) that allows us to measure the similarity of webpages. The simweb test measures against ALEXA top 100 dual-stacked websites from 80 SamKnows probes connected to dual-stacked networks representing 58 different ASes. Using a two months-long dataset we show that 14% of these dual-stacked websites exhibit a dissimilarity in the number of fetched webpage elements, with 94% of them exhibiting a dissimilarity in their size. We show that 6% of these websites announce AAAA entries in the DNS but no content is delivered over IPv6 when an HTTP request is made. We also noticed several cases where not all webpage elements (such as images, javascript and CSS) of a dual-stacked website are available over IPv6. We show that 27% of the dual-stacked websites have some fraction of webpage elements that fail over IPv6, with 9% of the websites having more than 50% webpage elements that fail over IPv6. We perform a causality analysis and also identify sources for these failing elements. We show that 12% of these websites have more than 50% webpage elements that belong to the same origin source and fail over IPv6. Failure rates are largely affected by DNS resolution error on images, javascript and CSS content delivered from both same-origin and cross-origin sources. These failures tend to cripple experience for users behind an IPv6-only network and a quantification of failure cases may help improve IPv6 adoption on the Internet.