VuldiffFinder: Discovering inconsistencies in unstructured vulnerability information

Abstract

The information conveyed by vulnerability reports is crucial for enhancing the security of information systems. Nonetheless, there are widespread information inconsistencies across reports, including, numerical discrepancies, misreported version ranges, semantic conflict, and so on. Identifying these inconsistencies is essential for improving information quality. Current research primarily focuses on standardized, non-free-form information’s inconsistency at the character or numerical level, while research for unstructured ones at the semantic level is limited. Given this, we introduce Vul_diffFinder to determine the inconsistency of unstructured vulnerability information at the semantic level. Firstly, it utilizes NLP tools to break down unstructured information into constituent sets, and design a determination strategy based on the constituent’s syntactic hierarchies and semantic similarity. The designed strategy can determine information pairs in arbitrary structure. Secondly, it creates a span similarity-based fine-tuning task to enhance the embedding capabilities of the SpanBERT model, ensuring accurately capturing semantic information in the vulnerability domain. Finally, a dataset containing eight categories of vulnerability information and 1,612 samples is utilized to validate the proposed method. The results demonstrate that Vul_diffFinder outperforms the state-of-the-art schemes, showing a 4.31% improvement in the F1-score. Additionally, we discover that consistency is higher in information that has simpler writing structures (up to 56.46%). Heterogeneous and Contained are often found in information with fixed or complex writing structures (up to 23.33% and 38.30%, respectively). Divergent and Repugnant mainly occur in information with a high missing rate.