Characterizing Tests in IoT Software: Practices, Challenges and Opportunities

IF 6 1区计算机科学 Q1 COMPUTER SCIENCE, SOFTWARE ENGINEERING

IEEE Transactions on Software Engineering Pub Date : 2026-09-01 Epub Date: 2026-06-05 DOI:10.1109/TSE.2026.3700841

Rufeng Chen;Hengcheng Zhu;Wuqi Zhang;Zixu Zhou;Lili Wei

{"title":"Characterizing Tests in IoT Software: Practices, Challenges and Opportunities","authors":"Rufeng Chen;Hengcheng Zhu;Wuqi Zhang;Zixu Zhou;Lili Wei","doi":"10.1109/TSE.2026.3700841","DOIUrl":null,"url":null,"abstract":"The Internet of Things (IoT) is experiencing rapid growth. Smart devices are emerging in smart homes and industrial applications, performing mission-critical tasks. Bugs in IoT software can lead to severe consequences. For example, a buggy smart lock can allow unauthorized access to a private property. Testing is a primary practice to expose software bugs and ensure software quality. However, little is known about how IoT software is tested. To bridge this gap, we conducted the first empirical study on test cases in open-source IoT software. Specifically, we evaluated the effectiveness of test cases in IoT software, explored the challenges inherent in testing IoT software, and analyzed the usage of mock objects. Our results indicate that while IoT software often contains a considerable number of tests, their effectiveness remains limited. We identified the primary challenges in testing IoT software as managing complex interactions with various external dependencies, such as other network-reliant IoT components, file systems, operating systems, and databases. We also observed that the use of mock objects in IoT software closely aligns with our identified testing challenges. This alignment demonstrates the potential of mocking as a solution to enhance test coverage and address the complexities of IoT software testing.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 9","pages":"2587-2602"},"PeriodicalIF":6.0000,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Software Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/11552357/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/6/5 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

The Internet of Things (IoT) is experiencing rapid growth. Smart devices are emerging in smart homes and industrial applications, performing mission-critical tasks. Bugs in IoT software can lead to severe consequences. For example, a buggy smart lock can allow unauthorized access to a private property. Testing is a primary practice to expose software bugs and ensure software quality. However, little is known about how IoT software is tested. To bridge this gap, we conducted the first empirical study on test cases in open-source IoT software. Specifically, we evaluated the effectiveness of test cases in IoT software, explored the challenges inherent in testing IoT software, and analyzed the usage of mock objects. Our results indicate that while IoT software often contains a considerable number of tests, their effectiveness remains limited. We identified the primary challenges in testing IoT software as managing complex interactions with various external dependencies, such as other network-reliant IoT components, file systems, operating systems, and databases. We also observed that the use of mock objects in IoT software closely aligns with our identified testing challenges. This alignment demonstrates the potential of mocking as a solution to enhance test coverage and address the complexities of IoT software testing.

查看原文本刊更多论文

表征物联网软件中的测试：实践、挑战和机遇

物联网（IoT）正在快速发展。智能设备正在智能家居和工业应用中兴起，执行关键任务。物联网软件中的漏洞可能导致严重后果。例如，有缺陷的智能锁可以允许未经授权的访问私有财产。测试是暴露软件缺陷和确保软件质量的主要实践。然而，人们对物联网软件的测试方式知之甚少。为了弥补这一差距，我们对开源物联网软件的测试用例进行了首次实证研究。具体来说，我们评估了物联网软件中测试用例的有效性，探讨了物联网软件测试中固有的挑战，并分析了模拟对象的使用情况。我们的研究结果表明，虽然物联网软件通常包含相当数量的测试，但其有效性仍然有限。我们确定了测试物联网软件的主要挑战是管理与各种外部依赖关系的复杂交互，例如其他依赖网络的物联网组件、文件系统、操作系统和数据库。我们还观察到，在物联网软件中使用模拟对象与我们确定的测试挑战密切相关。这种一致性证明了mock作为提高测试覆盖率和解决物联网软件测试复杂性的解决方案的潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Software Engineering 工程技术-工程：电子与电气

CiteScore

9.70

自引率

10.80%

发文量

724

审稿时长

6 months

期刊介绍： IEEE Transactions on Software Engineering seeks contributions comprising well-defined theoretical results and empirical studies with potential impacts on software construction, analysis, or management. The scope of this Transactions extends from fundamental mechanisms to the development of principles and their application in specific environments. Specific topic areas include: a) Development and maintenance methods and models: Techniques and principles for specifying, designing, and implementing software systems, encompassing notations and process models. b) Assessment methods: Software tests, validation, reliability models, test and diagnosis procedures, software redundancy, design for error control, and measurements and evaluation of process and product aspects. c) Software project management: Productivity factors, cost models, schedule and organizational issues, and standards. d) Tools and environments: Specific tools, integrated tool environments, associated architectures, databases, and parallel and distributed processing issues. e) System issues: Hardware-software trade-offs. f) State-of-the-art surveys: Syntheses and comprehensive reviews of the historical development within specific areas of interest.