2022 IEEE International Conference On Artificial Intelligence Testing (AITest)最新文献

A Passive Testing Approach using a Semi-Supervised Intrusion Detection Model for SCADA Network Traffic 基于半监督入侵检测模型的SCADA网络流量被动测试方法

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00015

Herbert Muehlburger, F. Wotawa

{"title":"A Passive Testing Approach using a Semi-Supervised Intrusion Detection Model for SCADA Network Traffic","authors":"Herbert Muehlburger, F. Wotawa","doi":"10.1109/AITest55621.2022.00015","DOIUrl":"https://doi.org/10.1109/AITest55621.2022.00015","url":null,"abstract":"Worldwide cyber-attacks constantly threaten the security of available infrastructure relying on cyber-physical systems. Infrastructure companies use passive testing approaches such as anomaly-based intrusion detection systems to observe such systems and prevent attacks. However, the effectiveness of intrusion detection systems depends on the underlying models used for detecting attacks and the observations that may suffer from scarce data availability. Hence, we need research on a) passive testing methods for obtaining appropriate detection models and b) for analysing the impact of the scarceness of data for improving intrusion detection systems. In this paper, we contribute to these challenges. We build on former work on supervised intrusion detection of power grid substation SCADA network traffic where a real-world data set (APG data set) is available. In contrast to previous work, we use a semi-supervised model with recurrent neural network architectures (i.e., LSTM Autoencoders and sequence models). This model only considers samples of ordinary data traffic without attacks to learn an adequate detection model. We outline the underlying foundations regarding the machine learning approach used. Furthermore, we present and discuss the obtained experimental results and compare them with prior results on supervised machine learning approaches. The source code of this work is available at:https: //github.com/muehlburger/semi-supervised-intrusion-detection-scada","PeriodicalId":427386,"journal":{"name":"2022 IEEE International Conference On Artificial Intelligence Testing (AITest)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124616734","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Metrics for Measuring Error Extents of Machine Learning Classifiers 测量机器学习分类器误差范围的度量方法

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00016

Hong Zhu, Ian Bayley, Mark Green

引用次数: 1

Construction of Semantic Model for GUI of Mobile Applications Using Deep Learning 基于深度学习的移动应用GUI语义模型构建

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00010

Qingying Liu, Tao Zhang, J. Gao, Shaoying Liu, Jing Cheng

引用次数: 0

An Approach to GUI Test Scenario Generation Using Machine Learning 一种使用机器学习生成GUI测试场景的方法

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00020

J. Gao, ShiTing Li, Chuanqi Tao, Yejun He, Amrutha Pavani Anumalasetty, Erica Wilson Joseph, Akshata Hatwar Kumbashi Sripathi, Himabindu Nayani

引用次数: 0

MBET: Resilience Improvement Method for DNNs MBET: dnn的弹性改进方法

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00019

Abdullah Murat Buldu, A. Sen, Karthik Swaminathan, B. Kahne

引用次数: 2

An Approach For Verifying And Validating Clustering Based Anomaly Detection Systems Using Metamorphic Testing 一种基于变形测试的聚类异常检测系统验证方法

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00011

Faqeer ur Rehman, C. Izurieta

{"title":"An Approach For Verifying And Validating Clustering Based Anomaly Detection Systems Using Metamorphic Testing","authors":"Faqeer ur Rehman, C. Izurieta","doi":"10.1109/AITest55621.2022.00011","DOIUrl":"https://doi.org/10.1109/AITest55621.2022.00011","url":null,"abstract":"An oracle or test oracle is a mechanism that a software tester uses to verify the program output. In software testing, the oracle problem arises when either the oracle is not available or it may be available but is so expensive that it is infeasible to apply. To help address this problem in testing machine learning-based applications, we propose an approach for testing clustering algorithms. We exemplify this in the implementation of the award-winning density-based clustering algorithm i.e., Density-based Spatial Clustering of Applications with Noise (DBSCAN). Our proposed approach is based on the ‘Metamorphic Testing’ technique which is considered an effective approach in alleviating the oracle problem. Our contributions in this paper include, i) proposing and showing the applicability of a broader set of 21 Metamorphic Relations (MRs), among which 8 target the verification aspect, whereas, 14 of them target the validation aspect of testing the algorithm under test, and ii) identifying and segregating the MRs (by providing a detailed analysis) to help both naive and expert users understand how the proposed MRs target both the verification and validation aspects of testing the DBSCAN algorithm. To show the effectiveness of the proposed approach, we further conduct a case study on an anomaly detection system. The results obtained show that, i) different MRs have the ability to reveal different violation rates (for the given data instances); thus, showing their effectiveness, and ii) although we have not found any implementation issues (through verification) in the algorithm under test (that further enhances our trust in the implementation), the results suggest that the DBSCAN algorithm may not be suitable for scenarios (meeting the user expectations a.k.a validation) captured by almost 79% of violated MRs; which show high susceptibility to small changes in the dataset.","PeriodicalId":427386,"journal":{"name":"2022 IEEE International Conference On Artificial Intelligence Testing (AITest)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132723071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

DeepAbstraction: 2-Level Prioritization for Unlabeled Test Inputs in Deep Neural Networks 深度抽象:深度神经网络中未标记测试输入的2级优先级

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00018

Hamzah Al-Qadasi, Changshun Wu, Yliès Falcone, S. Bensalem

{"title":"DeepAbstraction: 2-Level Prioritization for Unlabeled Test Inputs in Deep Neural Networks","authors":"Hamzah Al-Qadasi, Changshun Wu, Yliès Falcone, S. Bensalem","doi":"10.1109/AITest55621.2022.00018","DOIUrl":"https://doi.org/10.1109/AITest55621.2022.00018","url":null,"abstract":"Deep learning systems recently achieved unprecedented success in various industries. However, DNNs still exhibit some erroneous behaviors, which lead to catastrophic results. As a result, more data should be collected to cover more corner cases. On the other hand, a massive amount of data consumes more human annotators (oracle), which increases the labeling budget and time. We propose an effective test prioritization technique, called DeepAbstraction to prioritize the more likely error-exposing instances among the entire unlabeled test dataset. The ultimate goal of our framework is to reduce the labeling cost and select the potential corner cases earlier before production. Different from existing work, DeepAbstraction leverages runtime monitors. In the literature, runtime monitors are primarily used to supervise the prediction of the neural network. Then, monitors trigger a verdict for each prediction: acceptance, rejection, or uncertainty. Monitors quantity the acquired knowledge into box abstraction during the training. Each box abstraction contains instances that share similar high-level features. In the test part, the verdict of monitor depends in which box abstraction a test instance resides. Moreover, we study intensively where corner cases can reside in the feature space, either near-boundary regions or nearcentroid regions. The existing test prioritization techniques can only prioritize many near-boundary instances and a few nearcentroid instances. Nevertheless, DeepAbstraction can effectively prioritize numerous instances from both regions. Therefore, our evaluation shows that DeepAbstraction outperforms the state-of the-art test prioritization techniques.","PeriodicalId":427386,"journal":{"name":"2022 IEEE International Conference On Artificial Intelligence Testing (AITest)","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132385717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Anomalous Anomaly Detection 异常检测

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00009

Muyeed Ahmed, Iulian Neamtiu

{"title":"Anomalous Anomaly Detection","authors":"Muyeed Ahmed, Iulian Neamtiu","doi":"10.1109/AITest55621.2022.00009","DOIUrl":"https://doi.org/10.1109/AITest55621.2022.00009","url":null,"abstract":"Anomaly Detection (AD) is an integral part of AI, with applications ranging widely from health to finance, manufacturing, and computer security. Though AD is popular and various AD algorithm implementations are found in popular toolkits, no attempt has been made to test the reliability of these implementations. More generally, AD verification and validation are lacking. To address this need, we introduce an approach and study on 4 popular AD algorithms as implemented in 3 popular tools, as follows. First, we checked whether implementations can perform their basic task of finding anomalies in datasets with known anomalies. Next, we checked two basic properties, determinism and consistency. Finally, we quantified differences in algorithms’ outcome so users can get a idea of variations that can be expected when using different algorithms on the same dataset. We ran our suite of analyses on 73 datasets that contain anomalies. We found that, for certain implementations, validation can fail on 10–73% of datasets. Our analysis has revealed that five implementations suffer from nondeterminism (19–98% of runs are nondeterministic), and 10 out of 12 implementation pairs are inconsistent.","PeriodicalId":427386,"journal":{"name":"2022 IEEE International Conference On Artificial Intelligence Testing (AITest)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115710303","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

A Review of Quality Assurance Research of Dialogue Systems 对话系统质量保证研究综述

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00021

Xiaomin Li, Chuanqi Tao, Jerry Gao, Hongjing Guo

引用次数: 0

DeltaExplainer: A Software Debugging Approach to Generating Counterfactual Explanations DeltaExplainer:生成反事实解释的软件调试方法

2022 IEEE International Conference On Artificial Intelligence Testing (AITest) Pub Date : 2022-08-01 DOI: 10.1109/AITest55621.2022.00023

Sunny Shree, Jaganmohan Chandrasekaran, Yu Lei, R. Kacker, D. R. Kuhn

引用次数: 0