A double-compression method for searchable network packets in network forensics and analysis

Abstract

Efficiently storing, searching, and extracting structured data such as network packets can significantly enhance cybersecurity analysis and artificial intelligence model training. This paper presents an efficient searchable double-compression method, PKTDC, which involves two processes: double compression and searchable decompression of specific packets. In double compression, PKTDC dynamically constructs an index to compress the searched data and then performs a second round of compression on this data and other payloads via a conventional algorithm. In searchable decompression, PKTDC reconstructs searchable packet information from the compressed data, partially decompresses the matched payloads, and stitches them together to restore the original packets. The experimental results show that PKTDC achieves up to 7.55% greater compression efficiency than LZMA2, reduces the search and decompression time by up to 21.6 times, reduces CPU usage by up to 5.51 times, and reduces memory usage by up to 2.9 times.