2021 IEEE/ACM International Conference on Automation of Software Test (AST)最新文献

Message from the AST 2021 Program Chairs 来自AST 2021项目主席的信息

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-11-01 DOI: 10.1109/ast52587.2021.00005

引用次数: 0

Model-based Automation of Test Script Generation Across Product Variants: a Railway Perspective 跨产品变体测试脚本生成的基于模型的自动化:铁路的视角

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00011

Alessio Bucaioni, F. D. Silvestro, I. Singh, Mehrdad Saadatmand, H. Muccini, Thorvaldur Jochumsson

引用次数: 5

A framework for the automation of testing computer vision systems 测试计算机视觉系统的自动化框架

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00023

F. Wotawa, Lorenz Klampfl, Ledio Jahaj

引用次数: 1

Towards exhaustive branch coverage with PathCrawler 通过PathCrawler实现详尽的分支覆盖

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00022

Nicky Williams

引用次数: 2

Multimodal Surprise Adequacy Analysis of Inputs for Natural Language Processing DNN Models 自然语言处理DNN模型输入的多模态惊喜充分性分析

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00017

Seah Kim, Shin Yoo

{"title":"Multimodal Surprise Adequacy Analysis of Inputs for Natural Language Processing DNN Models","authors":"Seah Kim, Shin Yoo","doi":"10.1109/AST52587.2021.00017","DOIUrl":"https://doi.org/10.1109/AST52587.2021.00017","url":null,"abstract":"As Deep Neural Networks (DNNs) are rapidly adopted in various domains, many test adequacy metrics for DNN inputs have been introduced to help evaluating, and validating, trained DNN models. Surprise Adequacy (SA) is one such metric that aims to quantitatively measure how surprising a new input is with respect to the data used to train the given model. While SA has been shown to be effective for computer vision tasks such as image classification or object segmentation, its efficacy for DNN based Natural Language Processing has not been thoroughly studied. This paper evaluates whether it is feasible to apply SA analysis to DNN models trained for NLP tasks. We also show that the input distribution captured in the latent embedding space can be multimodal1 for some NLP tasks, unlike those observed in computer vision tasks, and investigate if catering for the multimodal property of NLP models can improve SA analysis. An empirical evaluation of extended SA metrics with three NLP tasks and nine DNN models shows that, while unimodal SAs perform sufficiently well for text classification, multimodal SA can outperform unimodal metrics.","PeriodicalId":315603,"journal":{"name":"2021 IEEE/ACM International Conference on Automation of Software Test (AST)","volume":"118 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115680248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12

Test Suites as a Source of Training Data for Static Analysis Alert Classifiers 测试套件作为静态分析警报分类器的训练数据来源

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00019

Lori Flynn, William Snavely, Z. Kurtz

{"title":"Test Suites as a Source of Training Data for Static Analysis Alert Classifiers","authors":"Lori Flynn, William Snavely, Z. Kurtz","doi":"10.1109/AST52587.2021.00019","DOIUrl":"https://doi.org/10.1109/AST52587.2021.00019","url":null,"abstract":"Flaw-finding static analysis tools typically generate large volumes of code flaw alerts including many false positives. To save on human effort to triage these alerts, a significant body of work attempts to use machine learning to classify and prioritize alerts. Identifying a useful set of training data, however, remains a fundamental challenge in developing such classifiers in many contexts. We propose using static analysis test suites (i.e., repositories of \"benchmark\" programs that are purpose-built to test coverage and precision of static analysis tools) as a novel source of training data. In a case study, we generated a large quantity of alerts by executing various static analyzers on the Juliet C/C++ test suite, and we automatically derived ground truth labels for these alerts by referencing the Juliet test suite metadata. Finally, we used this data to train classifiers to predict whether an alert is a false positive. Our classifiers obtained high precision (90.2%) and recall (88.2%) for a large number of code flaw types on a hold-out test set. This preliminary result suggests that pre-training classifiers on test suite data could help to jumpstart static analysis alert classification in data-limited contexts.","PeriodicalId":315603,"journal":{"name":"2021 IEEE/ACM International Conference on Automation of Software Test (AST)","volume":"11 31","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114044345","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Automatically Assessing and Extending Code Coverage for NPM Packages 自动评估和扩展NPM包的代码覆盖率

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00013

Haiyang Sun, Andrea Rosà, Daniele Bonetta, Walter Binder

{"title":"Automatically Assessing and Extending Code Coverage for NPM Packages","authors":"Haiyang Sun, Andrea Rosà, Daniele Bonetta, Walter Binder","doi":"10.1109/AST52587.2021.00013","DOIUrl":"https://doi.org/10.1109/AST52587.2021.00013","url":null,"abstract":"Typical Node.js applications extensively rely on packages hosted in the npm registry. As such packages may be used by thousands of other packages or applications, it is important to assess their code coverage. Moreover, increasing code coverage may help detect previously unknown issues. In this paper, we introduce TESA, a new tool that automatically assembles a test suite for any package in the npm registry. The test suite includes 1) tests written for the target package and usually hosted in its development repository, and 2) tests selected from dependent packages. The former tests allow assessing the code coverage of the target package, while the latter ones can increase code coverage by exploiting third-party tests that also exercise code in the target package. We use TESA to assess the code coverage of 500 popular npm packages. Then, we demonstrate that TESA can significantly increase code coverage by including tests from dependent packages. Finally, we show that the test suites assembled by TESA increase the effectiveness of existing dynamic program analyses to identify performance issues that are not detectable when only executing the developer’s tests.","PeriodicalId":315603,"journal":{"name":"2021 IEEE/ACM International Conference on Automation of Software Test (AST)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123610436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Continuous Testing Improvement Model 持续测试改进模型

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00020

M. A. Mascheroni, E. Irrazábal, G. Rossi

{"title":"Continuous Testing Improvement Model","authors":"M. A. Mascheroni, E. Irrazábal, G. Rossi","doi":"10.1109/AST52587.2021.00020","DOIUrl":"https://doi.org/10.1109/AST52587.2021.00020","url":null,"abstract":"Continuous Delivery is a practice where high-quality software is built in a way that it can be released into production at any time. However, systematic literature reviews and surveys performed as part of this Doctoral Research report that both the literature and the industry are still facing problems related to testing using practices like Continuous Delivery or Continuous Deployment. Thus, we propose Continuous Testing Improvement Model (CTIM) as a solution to the testing problems in continuous software development environments. It brings together proposals and approaches from different authors which are presented as good practices grouped by type of tests and divided into four levels. These levels indicate an improvement hierarchy and an evolutionary path in the implementation of Continuous Testing. Also, an application called EvalCTIM was developed to support the appraisal of a testing process using the proposed model. Finally, to validate the model, an action-research methodology was employed through an interpretive theoretical evaluation followed by case studies conducted in real software development projects. After several improvements made as part of the validation outcomes, the results demonstrate that the model can be used as a solution for implementing Continuous Testing gradually at companies using Continuous Deployment or Continuous Delivery and measuring its progress.","PeriodicalId":315603,"journal":{"name":"2021 IEEE/ACM International Conference on Automation of Software Test (AST)","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126581775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

SuMo: A Mutation Testing Strategy for Solidity Smart Contracts SuMo:稳定性智能合约的突变测试策略

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00014

Morena Barboni, A. Morichetta, A. Polini

{"title":"SuMo: A Mutation Testing Strategy for Solidity Smart Contracts","authors":"Morena Barboni, A. Morichetta, A. Polini","doi":"10.1109/AST52587.2021.00014","DOIUrl":"https://doi.org/10.1109/AST52587.2021.00014","url":null,"abstract":"Smart Contracts are software programs that are deployed and executed within a blockchain infrastructure. Due to their immutable nature, directly resulting from the specific characteristics of the deploying infrastructure, smart contracts must be thoroughly tested before their release. Testing is one of the main activities that can help to improve the reliability of a smart contract, so as to possibly prevent considerable loss of valuable assets. It is therefore important to provide the testers with tools that permit them to assess the activity they performed.Mutation testing is a powerful approach for assessing the fault-detection capability of a test suite. In this paper, we propose SuMo, a novel mutation testing tool for Ethereum Smart Contracts. SuMo implements a set of 44 mutation operators that were designed starting from the latest Solidity documentation, and from well-known mutation testing tools. These allow to simulate a wide variety of faults that can be made by smart contract developers. The set of operators was designed to limit the generation of stillborn mutants, which slow down the mutation testing process and limit the usability of the tool. We report a first evaluation of SuMo on open-source projects for which test suites were available. The results we got are encouraging, and they suggest that SuMo can effectively help developers to deliver more reliable smart contracts.","PeriodicalId":315603,"journal":{"name":"2021 IEEE/ACM International Conference on Automation of Software Test (AST)","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132825183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Using Machine Learning to Build Test Oracles: an Industrial Case Study on Elevators Dispatching Algorithms 使用机器学习构建测试oracle:电梯调度算法的工业案例研究

2021 IEEE/ACM International Conference on Automation of Software Test (AST) Pub Date : 2021-05-01 DOI: 10.1109/AST52587.2021.00012

Aitor Arrieta, J. Ayerdi, M. Illarramendi, Aitor Agirre, Goiuria Sagardui Mendieta, Maite Arratibel

{"title":"Using Machine Learning to Build Test Oracles: an Industrial Case Study on Elevators Dispatching Algorithms","authors":"Aitor Arrieta, J. Ayerdi, M. Illarramendi, Aitor Agirre, Goiuria Sagardui Mendieta, Maite Arratibel","doi":"10.1109/AST52587.2021.00012","DOIUrl":"https://doi.org/10.1109/AST52587.2021.00012","url":null,"abstract":"The software of elevators requires maintenance over several years to deal with new functionality, correction of bugs or legislation changes. To automatically validate this software, test oracles are necessary. A typical approach in industry is to use regression oracles. These oracles have to execute the test input both, in the software version under test and in a previous software version. This practice has several issues when using simulation to test elevators dispatching algorithms at system level. These issues include a long test execution time and the impossibility of re-using test oracles both at different test levels and in operation. To deal with these issues, we propose DARIO, a test oracle that relies on regression learning algorithms to predict the Qualify of Service of the system. The regression learning algorithms of this oracle are trained by using data from previously tested versions. An empirical evaluation with an industrial case study demonstrates the feasibility of using our approach in practice. A total of five regression learning algorithms were validated, showing that the regression tree algorithm performed best. For the regression tree algorithm, the accuracy when predicting verdicts by DARIO ranged between 79 to 87%.","PeriodicalId":315603,"journal":{"name":"2021 IEEE/ACM International Conference on Automation of Software Test (AST)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128988622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12