A multimedia analytics framework for browsing image collections in digital forensics

Abstract

Searching through large collections of images to find patterns of use or to find sets of relevant items is difficult, especially when the information to consider is not only the content of the images itself, but also the associated metadata. Multimedia analytics is a new approach to such problems. We consider the case of forensic experts facing image collections of growing size during digital forensic investigations. We answer the forensic challenge by developing specialised novel interactive visualisations which employ content-based image clusters in both the analysis as well as in all visualizations. Their synergy makes the task of manually browsing these collections more effective and efficient. Evaluation of such multimedia analytics is a notoriously hard problem as there are so many factors influencing the result. As a controlled evaluation, we developed a user simulation framework to create image collections with time and directory information as metadata. We apply it in a number of scenarios to illustrate its use. The simulation tool is available to other researchers via our website.