A method for identifying and confirming the use of URL filtering products for censorship

Products used for managing network traffic and restricting access to Web content represent a dual-use technology. While they were designed to improve performance and protect users from inappropriate content, these products are also used to censor the Web by authoritarian regimes around the globe. This dual use has not gone unnoticed, with Western governments placing restrictions on their export. Our contribution is to present methods for identifying installations of URL filtering products and confirming their use for censorship. We first present a methodology for identifying externally visible installations of URL filtering products in ISPs around the globe. Further, we leverage the fact that many of these products accept user-submitted sites for blocking to confirm that a specific URL filtering product is being used for censorship. Using this method, we are able to confirm the use of McAfee SmartFilter in Saudi Arabia and the United Arab Emirates (UAE) and Netsweeper in Qatar, the UAE, and Yemen. Our results show that these products are being used to block a range of content, including oppositional political speech, religious discussion and gay and lesbian material, speech generally protected by international human rights norms.