Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing最新文献

Investigating Test Smells in JavaScript Test Code 调查JavaScript测试代码中的测试气味

Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing Pub Date : 2021-09-27 DOI: 10.1145/3482909.3482915

Dalton N. Jorge, Patricia D. L. Machado, W. Andrade

{"title":"Investigating Test Smells in JavaScript Test Code","authors":"Dalton N. Jorge, Patricia D. L. Machado, W. Andrade","doi":"10.1145/3482909.3482915","DOIUrl":"https://doi.org/10.1145/3482909.3482915","url":null,"abstract":"Writing automated test cases is a challenging and demanding activity. The test case itself is software that requires proper design to ensure it can be implemented and maintained as long as the production code evolves. Like code smells, test smells may indicate violations of principles that negatively affect the quality of test code design, making it difficult to comprehend and, consequently, impairing its proper use and evolution. This work aims to investigate the occurrence of test smells in JavaScript test code and whether their presence can be correlated with test code quality. We perform an empirical study using the STEEL tool where the test suites of 11 open-source JavaScript projects from the Github repository are analyzed to detect a set of previously cataloged test smells. We then investigate: i) which ones occur more frequently; ii) whether given test smells are likely to occur together, and iii) if the presence of certain test smells is related to classical bad design indicators on the test code. We found that the Duplicate Assert, Magic Number Test, Unknown Test and Conditional Test Logic smells are the most common in JavaScript test code, whereas the Mystery Guest, Ignored Test and Resource Optimism smells are the least common. Moreover, the Conditional Test Logic, Magic Number Test, Duplicate Assert and the Exception Handling smells may often appear together. Furthermore, there is a moderate to a strong positive correlation between some smells count and quality measures in the test code. We can conclude that test smells are frequently found in JavaScript test code, and their presence may be an indicator of low design quality.","PeriodicalId":355243,"journal":{"name":"Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125542931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

An Analysis of Automated Code Inspection Tools for PHP Available on Github Marketplace Github市场上PHP自动代码检查工具分析

Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing Pub Date : 2021-09-27 DOI: 10.1145/3482909.3482912

Iderli Pereira de Souza Filho, Lucas Campello, Elder Rodrigues, Gilleanes Thorwald Araujo Guedes, Maicon Bernardino

{"title":"An Analysis of Automated Code Inspection Tools for PHP Available on Github Marketplace","authors":"Iderli Pereira de Souza Filho, Lucas Campello, Elder Rodrigues, Gilleanes Thorwald Araujo Guedes, Maicon Bernardino","doi":"10.1145/3482909.3482912","DOIUrl":"https://doi.org/10.1145/3482909.3482912","url":null,"abstract":"Code Inspection is a validation process widely used to improve the quality of software. To streamline this process and decrease the possibility of human error, improving the reliability of inspection results, it is possible to use specialized automated code inspection tools. Thus, this article proposes to analyze code inspection tools for PHP programming language, freely available on Github Marketplace. To achieve this goal, the GLPI system was chosen to be inspected, in addition, four code inspection tools were selected, out of twenty-eight available. Criteria were used for the tools selection, consistent with the sytem profile to be inspected and that do not have limitations on the inspection result. To classify the results obtained, the Common Weakness Enumeration (CWE) was used, a list of software and hardware weaknesses developed by numerous renowned companies, such as Microsoft, Apple and IBM. As a result of the inspection work, we found more than ten thousand failures divided into thirty-four different CWEs and from these we analyzed the individual feedback of each tool, as each one of them had unique advantages and disadvantages.","PeriodicalId":355243,"journal":{"name":"Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126226325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

On Using Decision Tree Coverage Criteria forTesting Machine Learning Models 用决策树覆盖标准测试机器学习模型

Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing Pub Date : 2021-09-27 DOI: 10.1145/3482909.3482911

Sebastião Santos, B. Silveira, Vinicius H. S. Durelli, R. Durelli, S. Souza, M. Delamaro

{"title":"On Using Decision Tree Coverage Criteria forTesting Machine Learning Models","authors":"Sebastião Santos, B. Silveira, Vinicius H. S. Durelli, R. Durelli, S. Souza, M. Delamaro","doi":"10.1145/3482909.3482911","DOIUrl":"https://doi.org/10.1145/3482909.3482911","url":null,"abstract":"Over the past decade, there has been a growing interest in applying machine learning (ML) to address a myriad of tasks. Owing to this interest, the adoption of ML-based systems has gone mainstream. However, this widespread adoption of ML-based systems poses new challenges for software testers that must improve the quality and reliability of these ML-based solutions. To cope with the challenges of testing ML-based systems, we propose novel test adequacy criteria based on decision tree models. Differently from the traditional approach to testing ML models, which relies on manual collection and labelling of data, our criteria leverage the internal structure of decision tree models to guide the selection of test inputs. Thus, we introduce decision tree coverage (DTC) and boundary value analysis (BVA) as approaches to systematically guide the creation of effective test data that exercises key structural elements of a given decision tree model. To evaluate these criteria, we carried out an experiment using 12 datasets. We measured the effectiveness of test inputs in terms of the difference in model’s behavior between the test input and the training data. The experiment results indicate that our testing criteria can be used to guide the generation of effective test data.","PeriodicalId":355243,"journal":{"name":"Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114643685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Impact on the Test Coverage Caused by the Introduction of Adaptive Behavior in a Legacy Web Application: A Case Study 在遗留Web应用程序中引入自适应行为对测试覆盖率的影响:一个案例研究

Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing Pub Date : 2021-09-27 DOI: 10.1145/3482909.3482914

Rogerio Jeronimo Gentil, D. Porto, F. Ferrari

{"title":"The Impact on the Test Coverage Caused by the Introduction of Adaptive Behavior in a Legacy Web Application: A Case Study","authors":"Rogerio Jeronimo Gentil, D. Porto, F. Ferrari","doi":"10.1145/3482909.3482914","DOIUrl":"https://doi.org/10.1145/3482909.3482914","url":null,"abstract":"Self-Adaptive systems (ASs) are an alternative to get IT professionals free from details of operation and maintenance. Web applications can incorporate characteristics of an AS in order to make them autonomous. However, one of the challenges of software engineering is to transform a legacy system into an AS. Furthermore, transforming a legacy system into an AS can lead to the insertion of faults. This way, the code coverage information produced by software tests assist in activities related to the evolutionary maintenance of the system. In this work, we analyzed, through the comparison of coverage metrics, how the introduction of adaptive behaviors in a real world legacy web application impacted on code coverage. For each implemented scenario, a set of test cases was created in the legacy version of the web application. Then, evolutionary versions were developed with the introduction of adaptive behavior. The versions (legacy and adaptive) were organized into branches of the version control system so that adaptive implementations have evolved in two technological ways. The coverage metrics for each version of the web application were collected to allow data analysis. Beyond the legacy version, we analyzed the coverage of six versions of the web application with adaptive behavior. The results indicate that the impact on the total code coverage is relevant, where even relatively small changes can considerably affect, either positively or negatively, the percentage of coverage of the changed unit.","PeriodicalId":355243,"journal":{"name":"Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing","volume":"44 6","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120848748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Automating Transaction Testing Involving Payment at PoS (Point of Sale) Terminals: An Experience Report at a Technology Company 涉及PoS(销售点)终端支付的自动化交易测试:一家技术公司的经验报告

Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing Pub Date : 2021-09-27 DOI: 10.1145/3482909.3482913

Chiara Rolim, A. Rebouças

引用次数: 0

How do Testers Feel it? An Experience Report on Evaluating TX in an Industrial Context 测试员有何感受?工业环境下TX评估经验报告

Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing Pub Date : 2021-09-27 DOI: 10.1145/3482909.3482917

B. Bonifácio, O. Ekwoge, Camila Bernardon, Thiago L. Souto

{"title":"How do Testers Feel it? An Experience Report on Evaluating TX in an Industrial Context","authors":"B. Bonifácio, O. Ekwoge, Camila Bernardon, Thiago L. Souto","doi":"10.1145/3482909.3482917","DOIUrl":"https://doi.org/10.1145/3482909.3482917","url":null,"abstract":"Background: Software development is almost always performed by teams. Consequently, several factors such as environment, human aspects, and technologies adopted are likely to affect individual and team performance. Method: Motivated by this context, we conducted a survey with 27 testers of the SIDIA Institute of Science and Technology. The goal was to understand how human and technological aspects can affect test effectiveness, by considering the practitioners’ perspective on their homologation testing review practices with respect three aspects: the testing infrastructure, how they perceive the value of their contribution, and how they feel about their work. Qualitative and quantitative analyses of testers’ perception using the Tester Experience (TX) approach was conducted. Results: Our results show that by using scenarios, the testers’ perceptions with respect to the value of their contribution and how they feel about their work were clearly the highest rated factors. Conclusion: This work can serve as a basis for other companies in their testing teams to evaluate how the use of testing artifacts and how testing activities can improve TX.","PeriodicalId":355243,"journal":{"name":"Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127077619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

On the use of test smells for prediction of flaky tests 用试验气味预测片状试验

Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing Pub Date : 2021-08-26 DOI: 10.1145/3482909.3482916

Bruno Henrique Pachulski Camara, Marco Aurélio Graciotto Silva, A. T. Endo, S. Vergilio

{"title":"On the use of test smells for prediction of flaky tests","authors":"Bruno Henrique Pachulski Camara, Marco Aurélio Graciotto Silva, A. T. Endo, S. Vergilio","doi":"10.1145/3482909.3482916","DOIUrl":"https://doi.org/10.1145/3482909.3482916","url":null,"abstract":"Regression testing is an important phase to deliver software with quality. However, flaky tests hamper the evaluation of test results and can increase costs. This is because a flaky test may pass or fail non-deterministically and to identify properly the flakiness of a test requires rerunning the test suite multiple times. To cope with this challenge, approaches have been proposed based on prediction models and machine learning. Existing approaches based on the use of the test case vocabulary may be context-sensitive and prone to overfitting, presenting low performance when executed in a cross-project scenario. To overcome these limitations, we investigate the use of test smells as predictors of flaky tests. We conducted an empirical study to understand if test smells have good performance as a classifier to predict the flakiness in the cross-project context, and analysed the information gain of each test smell. We also compared the test smell-based approach with the vocabulary-based one. As a result, we obtained a classifier that had a reasonable performance (Random Forest, 0.83%) to predict the flakiness in the testing phase. This classifier presented better performance than vocabulary-based model for cross-project prediction. The Assertion Roulette and Sleepy Test test smell types are the ones associated with the best information gain values.","PeriodicalId":355243,"journal":{"name":"Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127564364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12