LaAeb: A comprehensive log-text analysis based approach for insider threat detection

Abstract

Insider threats have increasingly become a critical issue that modern enterprises and organizations faced. They are mainly initiated by insider attackers, which may cause disastrous impacts. Numerous research studies have been conducted for insider threat detection. However, most of them are limited due to a small number of malicious samples. Moreover, as existing methods often concentrate on feature information or statistical characteristics for anomaly detection, they still lack effective use of comprehensive textual content information contained in logs and thus will affect detection efficiency.

We propose LaAeb, a novel unsupervised insider threat detection framework that leverages rich linguistic information in log contents to enable conventional methods, such as an Isolation Forest-based anomaly detection, to better detect insider threats besides using various features and statistical information. To find malicious acts under different scenarios, we consider three patterns of insider threats, including attention, emotion, and behavior anomaly. The attention anomaly detection analyzes textual contents of operation objects (e.g., emails and web pages) in logs to detect threats, where the textual information reflects the areas that employees focus on. When the attention seriously deviates from daily work, an employee may involve malicious acts. The emotion anomaly detection analyzes all dialogs between every two employees’ daily communicated texts and uses the degree of negative to find potential psychological problems. The behavior anomaly detection analyzes the operations of logs to detect threats. It utilizes information acquired from attention and emotion anomalies as ancillary features, integrating them with features and statistics extracted from log operations to create log embeddings. With these log embeddings, LaAeb employs anomaly detection algorithm like Isolation Forest to analyze an employee’s malicious operations, and further detects the employee’s behavior anomaly by considering all employees’ acts in the same department. Finally, LaAeb consolidates detection results of three patterns indicative of insider threats in a comprehensive manner.

We implement the prototype of LaAeb and test it on CERT and LANL datasets. Our evaluations demonstrate that compared with state-of-the-art unsupervised methods, LaAeb reduces FPR by 50% to reach 0.05 on CERT dataset under the same AUC $(0.93)$, and gets the best AUC $(0.97)$ with 0.06 higher value on LANL dataset.