Autocrime - open multimodal platform for combating organized crime

Abstract

A criminal investigation is a labor-intensive work requiring expert knowledge from several disciplines. Due to a large amount of heterogeneous data available from several modalities (i.e., audio/speech, text, video, non-content data), its processing raises many challenges. It may become impossible for law enforcement agents to deal with large amounts of highly-diverse data, especially for cross-border investigations focused on organized crime. ROXANNE EC H2020 project developed an all-in-one investigation platform for processing such diverse data. The platform mainly focuses on analyzing lawfully intercepted telephone conversations extended by non-content data (e.g., metadata related to the calls, time/spatial positions, and data collected from social media). Several state-of-the-art components are integrated into the pipeline, including speaker identification, automatic speech recognition, and named entity detection. With information extracted from this pipeline, the platform builds multiple knowledge graphs that capture phone and speaker criminal network interactions, including the central network and their clans. After hands-on sessions, law enforcement agents found the Autocrime platform easy to understand and highlighted its innovative, multi-technology functionalities that streamline forensic investigations, reducing manual effort. The AI-powered platform marks a significant first step toward creating an open investigative tool that combines advanced speech, text, and video processing algorithms with criminal network analysis, aimed at mitigating organized crime.