Machine learning model for predicting cyber-criminal characteristics

Abstract

This study aims to predict investigation outcomes of individual cybercrime cases using the most relevant information provided by complainants. We curated a dataset on solved hacking cases from 2019 to 2022 from the cyber-crime combating department (3CD) in the State of Kuwait. Each case has a set of information provided by the complainants (input features), and a corresponding set of investigation results (outputs). For each output, several machine learning models, such as decision trees and feed-forward neural networks, were evaluated, via nested 5-fold cross-validation to measure how well they could predict the output, given the input features. Input feature sets were either selected from all possible input feature combinations (brute force) or from a limited set of officer-provided combinations (officer-guided). Finally, a post-hoc analysis of the results was performed to identify a single set of features that can be used to build reasonably predictive models for all collected outputs. Depending on the output, the brute force and officer-guided approaches have a median relative advantage of 92% and 53% over the baseline models and worst-officer score respectively. On almost all outputs, the brute-force approach is just as good, if not better, than the officer-guided approach. No relationship was observed between officer rank and the predictive power of the combination of features they selected. Different outputs require different sets of features, and there is a significant overlap between brute force and officer-guided features in five out of the 10 outputs. Most selected features have a reliable negative impact on prediction performance when perturbed, with some outputs relying on a few critical features and others on a spectrum of features. Finally, a single set of features can predict most outputs almost as well as output-specific features.