2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)最新文献

Automated Function Assessment in Driving Scenarios 驾驶场景中的自动功能评估

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00050

Christian King, Lennart Ries, Christopher Kober, C. Wohlfahrt, E. Sax

{"title":"Automated Function Assessment in Driving Scenarios","authors":"Christian King, Lennart Ries, Christopher Kober, C. Wohlfahrt, E. Sax","doi":"10.1109/ICST.2019.00050","DOIUrl":"https://doi.org/10.1109/ICST.2019.00050","url":null,"abstract":"In recent years, numerous innovations in the automotive industry have addressed the field of driver assistance systems and automated driving. Therefore additional required sensors, as well as the need for digital maps and online services, lead to an ever-increasing system space, which must be covered. Established test approaches in the area of Hardware-in-the-Loop (HiL) use predefined and structured test cases to test the systems on the basis of requirements. In the approach of systematic testing, an evaluation is only carried out for a specific test case respectively the duration of a test step. This paper presents a concept for an automated quality assessment of driving scenarios or digital test drives. The aim is the analysis and subsequent evaluation of continuous function behavior during a realistic test drive within a simulated environment. Compared to conventional systematic test approaches, the presented concept allows a continuous evaluation of the test drive, whereby multiple evaluations of systems in similar scenarios with deviating boundary conditions is possible. For the first time, this enables a functional evaluation of a complete test drive comprising numerous scenarios and situations. The presented approach was prototypically implemented and demonstrated on a Hardware-in-the-Loop (HiL) test bench evaluating an adaptive cruise control (ACC) system.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"413 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124410800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

Fixing of Security Vulnerabilities in Open Source Projects: A Case Study of Apache HTTP Server and Apache Tomcat 修复开源项目中的安全漏洞:以Apache HTTP服务器和Apache Tomcat为例

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00017

Valentina Piantadosi, Simone Scalabrino, R. Oliveto

{"title":"Fixing of Security Vulnerabilities in Open Source Projects: A Case Study of Apache HTTP Server and Apache Tomcat","authors":"Valentina Piantadosi, Simone Scalabrino, R. Oliveto","doi":"10.1109/ICST.2019.00017","DOIUrl":"https://doi.org/10.1109/ICST.2019.00017","url":null,"abstract":"Software vulnerabilities are particularly dangerous bugs that may allow an attacker to violate the confidentiality, integrity or availability constraints of a software system. Fixing vulnerabilities soon is of primary importance; besides, it is crucial to release complete patches that do not leave any corner case not covered. In this paper we study the process of vulnerability fixing in Open Source Software. We focus on three dimensions: personal, i.e., who fixes software vulnerabilities; temporal, i.e., how long does it take to release a patch; procedural, i.e., what is the process followed to fix the vulnerability. In the context of our study we analyzed 337 CVE Entries regarding Apache HTTP Server and Apache Tomcat and we manually linked them to the patches written to fix such vulnerabilities and their related commits. The results show that developers who fix software vulnerabilities are much more experienced than the average. Furthermore, we observed that the vulnerabilities are fixed through more than a commit and, surprisingly, that in about 3% of the cases such vulnerabilities show up again in future releases (i.e., they are not actually fixed). In the light of such results, we derived some lessons learned that represent a starting point for future research directions aiming at better supporting developers during the documentation and fixing of vulnerabilities.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130745884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 25

Techniques for Evolution-Aware Runtime Verification 进化感知运行时验证技术

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00037

Owolabi Legunsen, Yi Zhang, Milica Hadzi-Tanovic, Grigore Roşu, D. Marinov

{"title":"Techniques for Evolution-Aware Runtime Verification","authors":"Owolabi Legunsen, Yi Zhang, Milica Hadzi-Tanovic, Grigore Roşu, D. Marinov","doi":"10.1109/ICST.2019.00037","DOIUrl":"https://doi.org/10.1109/ICST.2019.00037","url":null,"abstract":"Runtime Verification (RV) can help find bugs by monitoring program executions against formal properties. Developers should ideally use RV whenever they run tests, to find more bugs earlier. Despite tremendous research progress, RV still incurs high overhead in (1) machine time to monitor properties and (2) developer time to wait for and inspect violations from test executions that do not satisfy the properties. Moreover, all prior RV techniques consider only one program version and wastefully re-monitor unaffected properties and code as software evolves. We present the first evolution-aware RV techniques that reduce RV overhead across multiple program versions. Regression Property Selection (RPS) re-monitors only properties that can be violated in parts of code affected by changes, reducing machine time and developer time. Violation Message Suppression (VMS) simply shows only new violations to reduce developer time; it does not reduce machine time. Regression Property Prioritization (RPP) splits RV in two phases: properties more likely to find bugs are monitored in a critical phase to provide faster feedback to the developers; the rest are monitored in a background phase. We compare our techniques with the evolution-unaware (base) RV when monitoring test executions in 200 versions of 10 open-source projects. RPS and the RPP critical phase reduce the average RV overhead from 9.4x (for base RV) to 1.8x, without missing any new violations. VMS reduces the average number of violations 540x, from 54 violations per version (for base RV) to one violation per 10 versions.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125135119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 10

Automatic Visual Verification of Layout Failures in Responsively Designed Web Pages 响应式设计网页中布局失败的自动视觉验证

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00027

Ibrahim Althomali, G. M. Kapfhammer, Phil McMinn

{"title":"Automatic Visual Verification of Layout Failures in Responsively Designed Web Pages","authors":"Ibrahim Althomali, G. M. Kapfhammer, Phil McMinn","doi":"10.1109/ICST.2019.00027","DOIUrl":"https://doi.org/10.1109/ICST.2019.00027","url":null,"abstract":"Responsively designed web pages adjust their layout according to the viewport width of the device in use. Although tools exist to help developers test the layout of a responsive web page, they often rely on humans to flag problems. Yet, the considerable number of web-enabled devices with unique viewport widths makes this manual process both time-consuming and error-prone. Capable of detecting some common responsive layout failures, the ReDeCheck tool partially automates this process. Since ReDeCheck focuses on a web page's document object model (DOM), some of the issues it finds are not observable by humans. This paper presents a tool, called Viser, that renders a ReDeCheck-reported layout issue in a browser, adjusting the opacity of certain elements and checking for a visible difference. Unless Viser classifies an issue as a human-observable layout failure, a web developer can ignore it. This paper's experiments reveal the benefit of using Viser to support automated visual verification of layout failures in responsively designed web pages. Viser automatically classified all of the 117 layout failures that ReDeCheck reported for 20 web pages, each of which had to be manually analyzed in a prior study. Viser's automated manipulation of element opacity also highlighted manual classification's subjectivity: it categorized 28 issues differently to manual analysis, including three correctly reclassified as false positives.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125199715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 10

Efficiently Repairing Internationalization Presentation Failures by Solving Layout Constraints 通过求解布局约束有效修复国际化表示失败

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00026

Abdulmajeed Alameer, Paul T. Chiou, William G. J. Halfond

{"title":"Efficiently Repairing Internationalization Presentation Failures by Solving Layout Constraints","authors":"Abdulmajeed Alameer, Paul T. Chiou, William G. J. Halfond","doi":"10.1109/ICST.2019.00026","DOIUrl":"https://doi.org/10.1109/ICST.2019.00026","url":null,"abstract":"Web developers employ internationalization frameworks to automate web page translations and enable their web apps to more easily communicate with a global audience. However, the change of text size in different languages can lead to distortions in the translated web page's layout. These distortions are known as Internationalization Presentation Failures (IPFs). Debugging these IPFs can be a tedious and error-prone process. Previous research efforts to develop an automatic IPF repair technique could compromise the attractiveness and readability of the repaired web page. In this paper, we present a novel approach that can rapidly repair IPFs and maintain the readability and the attractiveness of the web page. Our approach models the correct layout of a web page as a system of constraints. The solution to the system represents the new and correct layout of the web page that resolves its IPFs. In the evaluation, we found that our approach could more quickly produce repairs that were rated as more attractive and more readable than those produced by a prior state-of-the-art technique.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134292511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 13

Parallel Many-Objective Search for Unit Tests 并行多目标搜索单元测试

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00014

Verena Bader, José Campos, G. Fraser

引用次数: 1

Suspend-Less Debugging for Interactive and/or Realtime Programs 交互式和/或实时程序的无挂起调试

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00028

Haruto Tanno, H. Iwasaki

引用次数: 1

SeqFuzzer: An Industrial Protocol Fuzzing Framework from a Deep Learning Perspective SeqFuzzer:深度学习视角下的工业协议模糊框架

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00016

Hui Zhao, Zhihui Li, Hansheng Wei, Jianqi Shi, Yanhong Huang

{"title":"SeqFuzzer: An Industrial Protocol Fuzzing Framework from a Deep Learning Perspective","authors":"Hui Zhao, Zhihui Li, Hansheng Wei, Jianqi Shi, Yanhong Huang","doi":"10.1109/ICST.2019.00016","DOIUrl":"https://doi.org/10.1109/ICST.2019.00016","url":null,"abstract":"Industrial networks are the cornerstone of modern industrial control systems. Performing security checks of industrial communication processes helps detect unknown risks and vulnerabilities. Fuzz testing is a widely used method for performing security checks that takes advantage of automation. However, there is a big challenge to carry out security checks on industrial network due to the increasing variety and complexity of industrial communication protocols. In this case, existing approaches usually take a long time to model the protocol for generating test cases, which is labor-intensive and time-consuming. This becomes even worse when the target protocol is stateful. To help in addressing this problem, we employed a deep learning model to learn the structures of protocol frames and deal with the temporal features of stateful protocols. We propose a fuzzing framework named SeqFuzzer which automatically learns the protocol frame structures from communication traffic and generates fake but plausible messages as test cases. For proving the usability of our approach, we applied SeqFuzzer to widely-used Ethernet for Control Automation Technology (EtherCAT) devices and successfully detected several security vulnerabilities.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122479957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 32

iDFlakies: A Framework for Detecting and Partially Classifying Flaky Tests flakies:一个检测和部分分类flakies测试的框架

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00038

Wing Lam, Reed Oei, A. Shi, D. Marinov, Tao Xie

{"title":"iDFlakies: A Framework for Detecting and Partially Classifying Flaky Tests","authors":"Wing Lam, Reed Oei, A. Shi, D. Marinov, Tao Xie","doi":"10.1109/ICST.2019.00038","DOIUrl":"https://doi.org/10.1109/ICST.2019.00038","url":null,"abstract":"Regression testing is increasingly important with the wide use of continuous integration. A desirable requirement for regression testing is that a test failure reliably indicates a problem in the code under test and not a false alarm from the test code or the testing infrastructure. However, some test failures are unreliable, stemming from flaky tests that can nondeterministically pass or fail for the same code under test. There are many types of flaky tests, with order-dependent tests being a prominent type. To help advance research on flaky tests, we present (1) a framework, iDFlakies, to detect and partially classify flaky tests; (2) a dataset of flaky tests in open-source projects; and (3) a study with our dataset. iDFlakies automates experimentation with our tool for Maven-based Java projects. Using iDFlakies, we build a dataset of 422 flaky tests, with 50.5% order-dependent and 49.5% not. Our study of these flaky tests finds the prevalence of two types of flaky tests, probability of a test-suite run to have at least one failure due to flaky tests, and how different test reorderings affect the number of detected flaky tests. We envision that our work can spur research to alleviate the problem of flaky tests.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126248715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 100

Classifying False Positive Static Checker Alarms in Continuous Integration Using Convolutional Neural Networks 基于卷积神经网络的连续积分静态检查器报警误报分类

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-22 DOI: 10.1109/ICST.2019.00048

Seongmin Lee, Shin Hong, Jungbae Yi, Taeksu Kim, Chul-Joo Kim, S. Yoo

{"title":"Classifying False Positive Static Checker Alarms in Continuous Integration Using Convolutional Neural Networks","authors":"Seongmin Lee, Shin Hong, Jungbae Yi, Taeksu Kim, Chul-Joo Kim, S. Yoo","doi":"10.1109/ICST.2019.00048","DOIUrl":"https://doi.org/10.1109/ICST.2019.00048","url":null,"abstract":"Static code analysis in Continuous Integration (CI) environment can significantly improve the quality of a software system because it enables early detection of defects without any test executions or user interactions. However, being a conservative over-approximation of system behaviours, static analysis also produces a large number of false positive alarms, identification of which takes up valuable developer time. We present an automated classifier based on Convolutional Neural Networks (CNNs). We hypothesise that many false positive alarms can be classified by identifying specific lexical patterns in the parts of the code that raised the alarm: human engineers adopt a similar tactic. We train a CNN based classifier to learn and detect these lexical patterns, using a total of about 10K historical static analysis alarms generated by six static analysis checkers for over 27 million LOC, and their labels assigned by actual developers. The results of our empirical evaluation suggest that our classifier can be highly effective for identifying false positive alarms, with the average precision across all six checkers of 79.72%.","PeriodicalId":446827,"journal":{"name":"2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122586835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 21