NTLFlowLyzer: Towards generating an intrusion detection dataset and intruders behavior profiling through network and transport layers traffic analysis and pattern extraction

Network security remains a critical concern in modern computing systems due to the constant emergence of threats and attacks. This paper introduces a comprehensive behavioral profiling solution to address the limitations of current intrusion detection methods in identifying zero-day attacks and novel malicious behaviors. Beginning with raw network data, the proposed framework progresses through multiple stages, ultimately culminating in the creation of activity-specific profiles. Central to this approach is NTLFlowLyzer, a novel network traffic analyzer, which generates an updated dataset, BCCC-CIC-IDS2017, for enhanced profile generation. The core of the profiling system leverages the distinct behaviors exhibited by individual features and the diverse correlations observed across various activities. The profiling procedure attains accuracy and robustness by integrating a novel feature selection algorithm and a pattern extraction process. Furthermore, behavior similarity is introduced to quantify the resemblance between activities based on their features and behaviors. We rigorously evaluate the effectiveness of our model by subjecting it to comprehensive testing, followed by meticulous comparison with previous works. Our proposed framework proficiently characterizes eight malicious activities with an accuracy rate surpassing 99.8%, while displaying promising performance in profiling various other activities. These findings, derived from our comprehensive experiments, provide valuable guidance for accurately implementing behavioral profiling.