What Causes Bugs in Numerical Simulation Software? An Empirical Study

IF 5.7 2区计算机科学 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

IEEE Transactions on Reliability Pub Date : 2024-11-27 DOI:10.1109/TR.2024.3492380

Xiaochen Li;Youcheng Zhu;Shikai Guo;He Jiang

{"title":"What Causes Bugs in Numerical Simulation Software? An Empirical Study","authors":"Xiaochen Li;Youcheng Zhu;Shikai Guo;He Jiang","doi":"10.1109/TR.2024.3492380","DOIUrl":null,"url":null,"abstract":"Numerical simulation (NS) software is widely used in safety-critical domains (e.g., aerospace design) to simulate actual physical processes of real-world entities on computers. However, NS software is error-prone, whose bugs lead to incorrect simulation, and may even cause disastrous flaws in safety-critical applications. Although many studies investigate the bug characteristics of computation-centered software, such as machine learning systems, the characteristics of NS software bugs have not been fully studied: what are the root causes and symptoms; how are they different from other computation-centered software; and why the difference occurs. To bridge this gap, we present a systematic study of NS software bugs by analyzing 352 bugs in three popular NS projects (i.e., FDS, SU2, and Kratos) for different domains. We summarize seven root causes (with 18 subcategories) and five symptoms. We find that the correctness, completeness, compatibility, and parallelization to implement NS algorithms (i.e., models) are error-prone. Many root causes (e.g., incorrect model and incorrect initialization) require physical and chemical knowledge to avoid bugs, which may not be mastered by typical software developers. These findings motivate new challenges and opportunities for future NS software development, such as designing domain specific language systems for NS model.","PeriodicalId":56305,"journal":{"name":"IEEE Transactions on Reliability","volume":"74 3","pages":"3383-3397"},"PeriodicalIF":5.7000,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Reliability","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10767865/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}

引用次数: 0

Abstract

Numerical simulation (NS) software is widely used in safety-critical domains (e.g., aerospace design) to simulate actual physical processes of real-world entities on computers. However, NS software is error-prone, whose bugs lead to incorrect simulation, and may even cause disastrous flaws in safety-critical applications. Although many studies investigate the bug characteristics of computation-centered software, such as machine learning systems, the characteristics of NS software bugs have not been fully studied: what are the root causes and symptoms; how are they different from other computation-centered software; and why the difference occurs. To bridge this gap, we present a systematic study of NS software bugs by analyzing 352 bugs in three popular NS projects (i.e., FDS, SU2, and Kratos) for different domains. We summarize seven root causes (with 18 subcategories) and five symptoms. We find that the correctness, completeness, compatibility, and parallelization to implement NS algorithms (i.e., models) are error-prone. Many root causes (e.g., incorrect model and incorrect initialization) require physical and chemical knowledge to avoid bugs, which may not be mastered by typical software developers. These findings motivate new challenges and opportunities for future NS software development, such as designing domain specific language systems for NS model.

查看原文本刊更多论文

是什么导致了数值模拟软件中的bug ？实证研究

数值模拟（NS）软件广泛应用于安全关键领域（如航空航天设计），在计算机上模拟现实世界实体的实际物理过程。然而，NS软件是容易出错的，它的错误会导致不正确的模拟，甚至可能在安全关键的应用程序中造成灾难性的缺陷。虽然很多研究都在研究以计算为中心的软件（如机器学习系统）的bug特征，但NS软件bug的特征并没有得到充分的研究：根本原因和症状是什么；它们与其他以计算为中心的软件有何不同？以及为什么会出现这种差异。为了弥补这一差距，我们通过分析三个流行的NS项目（即FDS， SU2和Kratos）中不同领域的352个漏洞，对NS软件漏洞进行了系统研究。我们总结了7个根本原因（18个子类别）和5个症状。我们发现，实现NS算法（即模型）的正确性、完整性、兼容性和并行性容易出错。许多根本原因（例如，不正确的模型和不正确的初始化）需要物理和化学知识来避免错误，这可能不是典型的软件开发人员所能掌握的。这些发现为未来的神经网络软件开发带来了新的挑战和机遇，例如为神经网络模型设计特定领域的语言系统。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Reliability 工程技术-工程：电子与电气

CiteScore

12.20

自引率

8.50%

发文量

153

审稿时长

7.5 months

期刊介绍： IEEE Transactions on Reliability is a refereed journal for the reliability and allied disciplines including, but not limited to, maintainability, physics of failure, life testing, prognostics, design and manufacture for reliability, reliability for systems of systems, network availability, mission success, warranty, safety, and various measures of effectiveness. Topics eligible for publication range from hardware to software, from materials to systems, from consumer and industrial devices to manufacturing plants, from individual items to networks, from techniques for making things better to ways of predicting and measuring behavior in the field. As an engineering subject that supports new and existing technologies, we constantly expand into new areas of the assurance sciences.