Sovereignty-Aware Intrusion Detection on Streaming Data: Automatic Machine Learning Pipeline and Semantic Reasoning

Abstract

Intrusion Detection Systems (IDS) are critical in safeguarding network infrastructures against malicious attacks. Traditional IDSs often struggle with knowledge representation, real-time detection, and accuracy, especially when dealing with high-throughput data. This paper proposes a novel IDS framework that leverages machine learning models, streaming data, and semantic knowledge representation to enhance intrusion detection accuracy and scalability. Additionally, the study incorporates the concept of Digital Sovereignty, ensuring that data control, security, and privacy are maintained according to national and regional regulations. The proposed system integrates Apache Kafka for real-time data processing, an automatic machine learning pipeline (e.g., Tree-based Pipeline Optimization Tool (TPOT)) for classifying network traffic, and OWL-based semantic reasoning for advanced threat detection. The proposed system, evaluated on NSL-KDD and CIC-IDS-2017 datasets, demonstrated qualitative outcomes such as local compliance, reduced data storage needs due to real-time processing, and improved adaptability to local data laws. Experimental results reveal significant improvements in detection accuracy, processing efficiency, and Sovereignty alignment.