POSVIA: Inconsistency analyzer for open-source Proof-of-Concept reports

IF 4.3 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information and Software Technology Pub Date : 2025-08-16 DOI:10.1016/j.infsof.2025.107868

Lingyan Ding , Xingya Wang , Zhenyu Chen , Song Huang

{"title":"POSVIA: Inconsistency analyzer for open-source Proof-of-Concept reports","authors":"Lingyan Ding , Xingya Wang , Zhenyu Chen , Song Huang","doi":"10.1016/j.infsof.2025.107868","DOIUrl":null,"url":null,"abstract":"<div><h3>Context:</h3><div>Proof-of-Concept (PoC) reports are indispensable for evaluating the exploitability of vulnerabilities. Various PoC data sources are responsible for collecting and sharing these reports. We have identified inconsistencies in the information pertaining to affected software versions across these data sources. These inconsistencies serve as red flags, alerting security experts to exercise caution during exploitability assessments and ensuring the effective allocation of resources.</div></div><div><h3>Objective:</h3><div>This paper analyzes software version inconsistencies in PoC reports and proposes “POSVIA” (<strong><u>P</u></strong>oC <strong><u>O</u></strong>riented <strong><u>S</u></strong>oftware <strong><u>V</u></strong>ersion <strong><u>I</u></strong>nconsistency <strong><u>A</u></strong>nalyzer), a deep learning tool designed to automatically detect and evaluate these inconsistencies across multiple PoC data sources, overcoming the impracticality of manual detection.</div></div><div><h3>Methods:</h3><div>A Named Entity Recognition (NER) model was developed with high performance: precision (93.76%) and recall (93.48%) for extracting CVE IDs, affected software names, and version data from PoC reports. Additionally, a Relation Extraction (RE) model was designed with metrics of 95.04% precision and 96.40% recall, to identify relationships between software and versions. These models analyzed 173,239 PoC reports from four data sources and assessed version inconsistencies using “POSVIA”.</div></div><div><h3>Results:</h3><div>Analysis revealed that Openwall had the lowest strict match rate (32.75%) for affected software versions, compared to other sources. The strict match rate for verified software versions ranged from 60.00% to 78.16%, indicating substantial inconsistencies. Over time, the match rate fluctuated, improving when using ExploitDB, Packet Storm Security, and CXSecurity as benchmarks. Openwall’s rate remained low, suggesting it should be considered alongside other sources for vulnerability exploitability assessments.</div></div><div><h3>Conclusion:</h3><div>This study introduces an automated tool named “POSVIA”, which is designed to address the challenge of detecting inconsistencies in software versions within PoC reports. By automating inconsistency detection across multiple data sources, POSVIA overcomes the limitations of manual methods and enhances the accuracy of exploitability assessments. This approach provides critical support for improving software security and resource allocation.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"188 ","pages":"Article 107868"},"PeriodicalIF":4.3000,"publicationDate":"2025-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information and Software Technology","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0950584925002071","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Context:

Proof-of-Concept (PoC) reports are indispensable for evaluating the exploitability of vulnerabilities. Various PoC data sources are responsible for collecting and sharing these reports. We have identified inconsistencies in the information pertaining to affected software versions across these data sources. These inconsistencies serve as red flags, alerting security experts to exercise caution during exploitability assessments and ensuring the effective allocation of resources.

Objective:

This paper analyzes software version inconsistencies in PoC reports and proposes “POSVIA” (PoC Oriented Software Version Inconsistency Analyzer), a deep learning tool designed to automatically detect and evaluate these inconsistencies across multiple PoC data sources, overcoming the impracticality of manual detection.

Methods:

A Named Entity Recognition (NER) model was developed with high performance: precision (93.76%) and recall (93.48%) for extracting CVE IDs, affected software names, and version data from PoC reports. Additionally, a Relation Extraction (RE) model was designed with metrics of 95.04% precision and 96.40% recall, to identify relationships between software and versions. These models analyzed 173,239 PoC reports from four data sources and assessed version inconsistencies using “POSVIA”.

Results:

Analysis revealed that Openwall had the lowest strict match rate (32.75%) for affected software versions, compared to other sources. The strict match rate for verified software versions ranged from 60.00% to 78.16%, indicating substantial inconsistencies. Over time, the match rate fluctuated, improving when using ExploitDB, Packet Storm Security, and CXSecurity as benchmarks. Openwall’s rate remained low, suggesting it should be considered alongside other sources for vulnerability exploitability assessments.

Conclusion:

This study introduces an automated tool named “POSVIA”, which is designed to address the challenge of detecting inconsistencies in software versions within PoC reports. By automating inconsistency detection across multiple data sources, POSVIA overcomes the limitations of manual methods and enhances the accuracy of exploitability assessments. This approach provides critical support for improving software security and resource allocation.

查看原文本刊更多论文

POSVIA：用于开源概念验证报告的不一致性分析器

上下文：概念验证（PoC）报告对于评估漏洞的可利用性是必不可少的。各种PoC数据源负责收集和共享这些报告。我们已经确定了这些数据源中与受影响的软件版本有关的信息中的不一致之处。这些不一致是危险信号，提醒安全专家在进行可利用性评估时要谨慎行事，并确保资源的有效分配。目的：分析PoC报告中的软件版本不一致性，提出“POSVIA”（面向PoC的软件版本不一致性分析器），这是一种深度学习工具，旨在自动检测和评估多个PoC数据源中的这些不一致性，克服人工检测的不实用性。方法：建立命名实体识别（NER）模型，用于从PoC报告中提取CVE id、受影响软件名称和版本数据，准确率为93.76%，召回率为93.48%。此外，设计了一个关系提取（RE）模型，该模型具有95.04%的精度和96.40%的召回率，用于识别软件和版本之间的关系。这些模型分析了来自四个数据源的173239份PoC报告，并使用“POSVIA”评估版本不一致性。结果：分析显示，与其他来源相比，Openwall对受影响软件版本的严格匹配率最低（32.75%）。经过验证的软件版本的严格匹配率从60.00%到78.16%不等，表明存在大量的不一致。随着时间的推移，匹配率有所波动，使用ExploitDB、Packet Storm Security和CXSecurity作为基准时，匹配率有所提高。Openwall的比率仍然很低，这表明它应该与其他漏洞利用评估来源一起考虑。结论：本研究引入了一个名为“POSVIA”的自动化工具，该工具旨在解决在PoC报告中检测软件版本不一致的挑战。通过自动化跨多个数据源的不一致检测，POSVIA克服了手工方法的局限性，提高了可利用性评估的准确性。这种方法为改进软件安全性和资源分配提供了关键的支持。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Information and Software Technology 工程技术-计算机：软件工程

CiteScore

9.10

自引率

7.70%

发文量

164

审稿时长

9.6 weeks

期刊介绍： Information and Software Technology is the international archival journal focusing on research and experience that contributes to the improvement of software development practices. The journal''s scope includes methods and techniques to better engineer software and manage its development. Articles submitted for review should have a clear component of software engineering or address ways to improve the engineering and management of software development. Areas covered by the journal include: • Software management, quality and metrics, • Software processes, • Software architecture, modelling, specification, design and programming • Functional and non-functional software requirements • Software testing and verification & validation • Empirical studies of all aspects of engineering and managing software development Short Communications is a new section dedicated to short papers addressing new ideas, controversial opinions, "Negative" results and much more. Read the Guide for authors for more information. The journal encourages and welcomes submissions of systematic literature studies (reviews and maps) within the scope of the journal. Information and Software Technology is the premiere outlet for systematic literature studies in software engineering.