A Review of Dark Web: Trends and Future Directions

Abstract

The dark web is often discussed in taboo by many who are unfamiliar with the subject. However, this paper takes a dive into the skeleton of what constructs the dark web by compiling the research of published essays. The Onion Router (TOR) and other discussed browsers are specialized web browsers that provide anonymity by going through multiple servers and encrypted networks between the host and client, hiding the IP address of both ends. This provides difficulty in terms of controlling or monitoring the dark web, leading to its popularity in criminal underworlds. In this work, we provide an overview of data mining and penetration testing tools that are being widely used to crawl and collect data. We compare the tools to provide strengths and weaknesses of the tools while providing challenges of harnessing massive data from dark web using crawlers and penetration testing tools including machine learning (ML) techniques. Despite the effort to crawl dark web has progressed, there are still rooms to advance existing approaches to combat the ever-changing landscape of the dark web.