How Do Developers Structure Unit Test Cases? An Empirical Analysis of the AAA Pattern in Open Source Projects

IF 6.5 1区计算机科学 Q1 COMPUTER SCIENCE, SOFTWARE ENGINEERING

IEEE Transactions on Software Engineering Pub Date : 2025-01-31 DOI:10.1109/TSE.2025.3537337

Chenhao Wei;Lu Xiao;Tingting Yu;Sunny Wong;Abigail Clune

{"title":"How Do Developers Structure Unit Test Cases? An Empirical Analysis of the AAA Pattern in Open Source Projects","authors":"Chenhao Wei;Lu Xiao;Tingting Yu;Sunny Wong;Abigail Clune","doi":"10.1109/TSE.2025.3537337","DOIUrl":null,"url":null,"abstract":"The AAA (Arrange, Act, Assert) pattern provides a unified structure for unit test cases, potentially benefiting comprehension and maintenance. However, its adoption and implementation in practice remain insufficiently understood. This study investigates the prevalence of AAA pattern usage, identifies recurring deviations and design issues within AAA structures, and assesses developers’ receptiveness to AAA-based improvements. We conducted an empirical study on 735 real-life unit test cases randomly selected from seven open-source projects. We manually analyzed these test cases, identified AAA-related issues, and proposed fixes to developers. Our analysis found that 77% of test cases follow the AAA structure. We identified three recurring patterns deviating from AAA and four design issues within A blocks. Comparison with classic test smells revealed unique insights provided by AAA analysis. Of 27 improvement proposals sent to developers, 78% received positive feedback. These findings show that the AAA pattern is widely adopted in practice, but deviations from and design issues within AAA patterns are common. Our analysis provides a novel perspective on test case quality, complementing traditional test smell analysis. The high acceptance rate of our improvement proposals suggests that developers value AAA-based enhancements. These findings can guide the development of tools for improving AAA practice in unit tests.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 4","pages":"1007-1038"},"PeriodicalIF":6.5000,"publicationDate":"2025-01-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Software Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10859187/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

The AAA (Arrange, Act, Assert) pattern provides a unified structure for unit test cases, potentially benefiting comprehension and maintenance. However, its adoption and implementation in practice remain insufficiently understood. This study investigates the prevalence of AAA pattern usage, identifies recurring deviations and design issues within AAA structures, and assesses developers’ receptiveness to AAA-based improvements. We conducted an empirical study on 735 real-life unit test cases randomly selected from seven open-source projects. We manually analyzed these test cases, identified AAA-related issues, and proposed fixes to developers. Our analysis found that 77% of test cases follow the AAA structure. We identified three recurring patterns deviating from AAA and four design issues within A blocks. Comparison with classic test smells revealed unique insights provided by AAA analysis. Of 27 improvement proposals sent to developers, 78% received positive feedback. These findings show that the AAA pattern is widely adopted in practice, but deviations from and design issues within AAA patterns are common. Our analysis provides a novel perspective on test case quality, complementing traditional test smell analysis. The high acceptance rate of our improvement proposals suggests that developers value AAA-based enhancements. These findings can guide the development of tools for improving AAA practice in unit tests.

查看原文本刊更多论文

开发人员如何构建单元测试用例？开源项目AAA模式的实证分析

AAA （Arrange, Act, Assert）模式为单元测试用例提供了一个统一的结构，潜在地有利于理解和维护。然而，在实践中对其采用和执行仍然了解不足。本研究调查了AAA模式使用的普遍性，确定了AAA结构中反复出现的偏差和设计问题，并评估了开发人员对基于AAA的改进的接受程度。我们对从7个开源项目中随机选择的735个实际单元测试用例进行了实证研究。我们手动分析了这些测试用例，确定了aaa相关的问题，并向开发人员提出了修复建议。我们的分析发现77%的测试用例遵循AAA结构。我们确定了偏离AAA的三个重复模式和A块中的四个设计问题。与经典测试气味的比较揭示了AAA分析提供的独特见解。在发给开发者的27个改进建议中，78%得到了积极的反馈。这些发现表明，AAA模式在实践中被广泛采用，但偏离AAA模式和AAA模式中的设计问题是常见的。我们的分析提供了一个关于测试用例质量的新视角，补充了传统的测试气味分析。我们的改进建议的高接受率表明开发人员重视基于aaa的增强。这些发现可以指导在单元测试中改进AAA实践的工具的开发。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Software Engineering 工程技术-工程：电子与电气

CiteScore

9.70

自引率

10.80%

发文量

724

审稿时长

6 months

期刊介绍： IEEE Transactions on Software Engineering seeks contributions comprising well-defined theoretical results and empirical studies with potential impacts on software construction, analysis, or management. The scope of this Transactions extends from fundamental mechanisms to the development of principles and their application in specific environments. Specific topic areas include: a) Development and maintenance methods and models: Techniques and principles for specifying, designing, and implementing software systems, encompassing notations and process models. b) Assessment methods: Software tests, validation, reliability models, test and diagnosis procedures, software redundancy, design for error control, and measurements and evaluation of process and product aspects. c) Software project management: Productivity factors, cost models, schedule and organizational issues, and standards. d) Tools and environments: Specific tools, integrated tool environments, associated architectures, databases, and parallel and distributed processing issues. e) System issues: Hardware-software trade-offs. f) State-of-the-art surveys: Syntheses and comprehensive reviews of the historical development within specific areas of interest.