{"title":"基于机器学习的代码气味检测的数据预处理:系统的文献综述","authors":"Fábio do Rosario Santos, Ricardo Choren","doi":"10.1016/j.infsof.2025.107752","DOIUrl":null,"url":null,"abstract":"<div><h3>Context:</h3><div>Detecting code smells using Machine Learning presents inherent challenges due to the unbalanced nature of the problem and susceptibility to interpretation biases. It is a data-driven process for code quality assurance that aims to detect if a given piece of code presents a fundamental design principles violation that negatively impacts design quality. Researchers in the field have been advised to carefully analyze the internal mechanisms of forecasting models before interpreting the results generated by them.</div></div><div><h3>Objective:</h3><div>The review aims to summarize and synthesize studies that utilized Data Preprocessing techniques for Machine Learning-based code smell detection. And also, to investigate the relationship between Data Preprocessing and more advanced Machine Learning techniques, i.e., Ensemble Methods, Deep Learning, and Transfer Learning.</div></div><div><h3>Method:</h3><div>To obtain insights into Data Preprocessing for Machine Learning-based code smell detection solutions, we employed a systematic approach, identifying and analyzing 69 studies published up to November 2023.</div></div><div><h3>Results:</h3><div>In Data Preprocessing, Data Balancing techniques, Feature Selection techniques, and Filtering emerged as prominent strategies. SMOTE was the most frequently used Data Balancing technique, while Autoencoder, Chi-square, Gain Ratio, Information Gain, PCA, and CFS were notable choices for Feature Selection. Tokenization and Syntax Trees were commonly paired with Deep Learning or Transfer Learning methods. Normalization and Standardization were implemented for Data Scaling. Regarding Machine Learning techniques used with Data Preprocessing, 46% of the combinations occurred with at least one Ensemble Method. Deep Learning was employed in 37% of cases. Data Balancing techniques combined with Deep Learning (32%) or Ensemble Methods (19%) were used most.</div></div><div><h3>Conclusion:</h3><div>The findings of this SLR are an integrated and comprehensive source of information regarding data preparation practices, challenges, and solutions for Machine Learning-based code smell detection, emphasizing the continuous endeavor towards more resilient, contextually sensitive, and developer-informed strategies within this dynamic field.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"184 ","pages":"Article 107752"},"PeriodicalIF":4.3000,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Data preprocessing for machine learning based code smell detection: A systematic literature review\",\"authors\":\"Fábio do Rosario Santos, Ricardo Choren\",\"doi\":\"10.1016/j.infsof.2025.107752\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Context:</h3><div>Detecting code smells using Machine Learning presents inherent challenges due to the unbalanced nature of the problem and susceptibility to interpretation biases. It is a data-driven process for code quality assurance that aims to detect if a given piece of code presents a fundamental design principles violation that negatively impacts design quality. Researchers in the field have been advised to carefully analyze the internal mechanisms of forecasting models before interpreting the results generated by them.</div></div><div><h3>Objective:</h3><div>The review aims to summarize and synthesize studies that utilized Data Preprocessing techniques for Machine Learning-based code smell detection. And also, to investigate the relationship between Data Preprocessing and more advanced Machine Learning techniques, i.e., Ensemble Methods, Deep Learning, and Transfer Learning.</div></div><div><h3>Method:</h3><div>To obtain insights into Data Preprocessing for Machine Learning-based code smell detection solutions, we employed a systematic approach, identifying and analyzing 69 studies published up to November 2023.</div></div><div><h3>Results:</h3><div>In Data Preprocessing, Data Balancing techniques, Feature Selection techniques, and Filtering emerged as prominent strategies. SMOTE was the most frequently used Data Balancing technique, while Autoencoder, Chi-square, Gain Ratio, Information Gain, PCA, and CFS were notable choices for Feature Selection. Tokenization and Syntax Trees were commonly paired with Deep Learning or Transfer Learning methods. Normalization and Standardization were implemented for Data Scaling. Regarding Machine Learning techniques used with Data Preprocessing, 46% of the combinations occurred with at least one Ensemble Method. Deep Learning was employed in 37% of cases. Data Balancing techniques combined with Deep Learning (32%) or Ensemble Methods (19%) were used most.</div></div><div><h3>Conclusion:</h3><div>The findings of this SLR are an integrated and comprehensive source of information regarding data preparation practices, challenges, and solutions for Machine Learning-based code smell detection, emphasizing the continuous endeavor towards more resilient, contextually sensitive, and developer-informed strategies within this dynamic field.</div></div>\",\"PeriodicalId\":54983,\"journal\":{\"name\":\"Information and Software Technology\",\"volume\":\"184 \",\"pages\":\"Article 107752\"},\"PeriodicalIF\":4.3000,\"publicationDate\":\"2025-04-25\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Information and Software Technology\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0950584925000916\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information and Software Technology","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0950584925000916","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
Data preprocessing for machine learning based code smell detection: A systematic literature review
Context:
Detecting code smells using Machine Learning presents inherent challenges due to the unbalanced nature of the problem and susceptibility to interpretation biases. It is a data-driven process for code quality assurance that aims to detect if a given piece of code presents a fundamental design principles violation that negatively impacts design quality. Researchers in the field have been advised to carefully analyze the internal mechanisms of forecasting models before interpreting the results generated by them.
Objective:
The review aims to summarize and synthesize studies that utilized Data Preprocessing techniques for Machine Learning-based code smell detection. And also, to investigate the relationship between Data Preprocessing and more advanced Machine Learning techniques, i.e., Ensemble Methods, Deep Learning, and Transfer Learning.
Method:
To obtain insights into Data Preprocessing for Machine Learning-based code smell detection solutions, we employed a systematic approach, identifying and analyzing 69 studies published up to November 2023.
Results:
In Data Preprocessing, Data Balancing techniques, Feature Selection techniques, and Filtering emerged as prominent strategies. SMOTE was the most frequently used Data Balancing technique, while Autoencoder, Chi-square, Gain Ratio, Information Gain, PCA, and CFS were notable choices for Feature Selection. Tokenization and Syntax Trees were commonly paired with Deep Learning or Transfer Learning methods. Normalization and Standardization were implemented for Data Scaling. Regarding Machine Learning techniques used with Data Preprocessing, 46% of the combinations occurred with at least one Ensemble Method. Deep Learning was employed in 37% of cases. Data Balancing techniques combined with Deep Learning (32%) or Ensemble Methods (19%) were used most.
Conclusion:
The findings of this SLR are an integrated and comprehensive source of information regarding data preparation practices, challenges, and solutions for Machine Learning-based code smell detection, emphasizing the continuous endeavor towards more resilient, contextually sensitive, and developer-informed strategies within this dynamic field.
期刊介绍:
Information and Software Technology is the international archival journal focusing on research and experience that contributes to the improvement of software development practices. The journal''s scope includes methods and techniques to better engineer software and manage its development. Articles submitted for review should have a clear component of software engineering or address ways to improve the engineering and management of software development. Areas covered by the journal include:
• Software management, quality and metrics,
• Software processes,
• Software architecture, modelling, specification, design and programming
• Functional and non-functional software requirements
• Software testing and verification & validation
• Empirical studies of all aspects of engineering and managing software development
Short Communications is a new section dedicated to short papers addressing new ideas, controversial opinions, "Negative" results and much more. Read the Guide for authors for more information.
The journal encourages and welcomes submissions of systematic literature studies (reviews and maps) within the scope of the journal. Information and Software Technology is the premiere outlet for systematic literature studies in software engineering.