PostCog: A tool for interdisciplinary research into underground forums at scale

Abstract

Underground forums provide useful insights into cybercrime, where researchers analyse underlying economies, key actors, their discussions and interactions, as well as different types of cybercrime. This interdisciplinary topic of study incorporates expertise from diverse areas, including computer science, criminology, economics, psychol-ogy, and other social sciences. Historically, there were sig-nificant challenges around access to data, but there are now research datasets of millions of messages scraped from underground forums. The problems now stem from the size of these datasets and the technical nature of methods and tools available for data sampling and analysis at scale, which make data exploration difficult for non-technical users. Postcoghas been developed to solve this problem. We first provide a survey of prior work into underground forums; this was used to understand the requirements and functionalities valued by researchers, and to inform the design of a data exploration tool. We then describe Postcog,a web application developed to support users from both technical and non-technical backgrounds in forum analyses, such as search, information extraction and cross-forum comparison. The prototype's usability is then evaluated through two user studies with expert users of the Crimebbdataset. Postcogis made available for academic research upon signing an agreement with the Cambridge Cybercrime Centre.