2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)最新文献_第2页

Algorithm Debt: Challenges and Future Paths 算法债务:挑战和未来路径

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00020

Emmanuel Simon, M. Vidoni, F. H. Fard

引用次数: 0

An Initial Analysis of Repair and Side-effect Prediction for Neural Networks 神经网络修复与副作用预测的初步分析

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00017

Yuta Ishimoto, Ken Matsui, Masanari Kondo, Naoyasu Ubayashi, Yasutaka Kamei

{"title":"An Initial Analysis of Repair and Side-effect Prediction for Neural Networks","authors":"Yuta Ishimoto, Ken Matsui, Masanari Kondo, Naoyasu Ubayashi, Yasutaka Kamei","doi":"10.1109/CAIN58948.2023.00017","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00017","url":null,"abstract":"With the prevalence of software systems adopting neural network models, the quality assurance of these systems has become crucial. Hence, various studies have proposed repairing methods for neural network models so far to improve the quality of the models. While these methods are evaluated by researchers, it is difficult to tell whether they succeed in all models and datasets (i.e., all developers’ environments). Because these methods require many resources, such as execution times, failing to repair neural networks would cost developers their resources. Hence, if developers can know whether repairing methods succeed before adopting them, they could avoid wasting their resources. This paper proposes prediction models that predict whether repairing methods succeed in repairing neural networks using a small resource. Our prediction models predict repairs and side-effects of repairing methods, respectively. We evaluated our prediction models on a state-of-the-art repairing method Arachne on three datasets, Fashion-MNIST, CIFAR-10, and GTSRB, and found our prediction models achieved high performance, an average ROC-AUC of 0.931 and an average f1score of 0.880 for the side-effects and an average ROC-AUC of 0.768 and an average f1-score of 0.725 for the repairs.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125610966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Engineering Challenges for AI-Supported Computer Vision in Small Uncrewed Aerial Systems 小型无人机系统中人工智能支持计算机视觉的工程挑战

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00033

Muhammed Tawfiq Chowdhury, J. Cleland-Huang

{"title":"Engineering Challenges for AI-Supported Computer Vision in Small Uncrewed Aerial Systems","authors":"Muhammed Tawfiq Chowdhury, J. Cleland-Huang","doi":"10.1109/CAIN58948.2023.00033","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00033","url":null,"abstract":"Computer Vision (CV) is used in a broad range of Cyber-Physical Systems such as surgical and factory floor robots and autonomous vehicles including small Unmanned Aerial Systems (sUAS). It enables machines to perceive the world by detecting and classifying objects of interest, reconstructing 3D scenes, estimating motion, and maneuvering around objects. CV algorithms are developed using diverse machine learning and deep learning frameworks, which are often deployed on limited resource edge devices. As sUAS rely upon an accurate and timely perception of their environment to perform critical tasks, problems related to CV can create hazardous conditions leading to crashes or mission failure. In this paper, we perform a systematic literature review (SLR) of CV-related challenges associated with CV, hardware, and software engineering. We then group the reported challenges into five categories and fourteen sub-challenges and present existing solutions. As current literature focuses primarily on CV and hardware challenges, we close by discussing implications for Software Engineering, drawing examples from a CV-enhanced multi-sUAS system.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134450318","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

How Federated Machine Learning Helps Increase the Mutual Benefit of Data-Sharing Ecosystems 联邦机器学习如何帮助增加数据共享生态系统的互利

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00023

Iva Krasteva, Boris Kraychev, Ensiye Kiyamousavi

引用次数: 0

Towards Understanding Model Quantization for Reliable Deep Neural Network Deployment 对可靠的深度神经网络部署模型量化的理解

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00015

Qiang Hu, Yuejun Guo, Maxime Cordy, Xiaofei Xie, Wei Ma, Mike Papadakis, Yves Le Traon

{"title":"Towards Understanding Model Quantization for Reliable Deep Neural Network Deployment","authors":"Qiang Hu, Yuejun Guo, Maxime Cordy, Xiaofei Xie, Wei Ma, Mike Papadakis, Yves Le Traon","doi":"10.1109/CAIN58948.2023.00015","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00015","url":null,"abstract":"Deep Neural Networks (DNNs) have gained considerable attention in the past decades due to their astounding performance in different applications, such as natural language modeling, self-driving assistance, and source code understanding. With rapid exploration, more and more complex DNN architectures have been proposed along with huge pre-trained model parameters. A common way to use such DNN models in user-friendly devices (e.g., mobile phones) is to perform model compression before deployment. However, recent research has demonstrated that model compression, e.g., model quantization, yields accuracy degradation as well as output disagreements when tested on unseen data. Since the unseen data always include distribution shifts and often appear in the wild, the quality and reliability of models after quantization are not ensured. In this paper, we conduct a comprehensive study to characterize and help users understand the behaviors of quantization models. Our study considers four datasets spanning from image to text, eight DNN architectures including both feed-forward neural networks and recurrent neural networks, and 42 shifted sets with both synthetic and natural distribution shifts. The results reveal that 1) data with distribution shifts lead to more disagreements than without. 2) Quantization-aware training can produce more stable models than standard, adversarial, and Mixup training. 3) Disagreements often have closer top-1 and top-2 output probabilities, and Margin is a better indicator than other uncertainty metrics to distinguish disagreements. 4) Retraining the model with disagreements has limited efficiency in removing disagreements. We release our code and models as a new benchmark for further study of model quantization.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130772890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Enabling Machine Learning in Software Architecture Frameworks 在软件架构框架中实现机器学习

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00021

Armin Moin, A. Badii, Stephan Günnemann, Moharram Challenger

引用次数: 0

Conceptualising Software Development Lifecycle for Engineering AI Planning Systems 工程人工智能规划系统的软件开发生命周期概念化

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00019

Ilche Georgievski

引用次数: 3

Exploring Hyperparameter Usage and Tuning in Machine Learning Research 探索机器学习研究中的超参数使用和调优

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00016

Sebastian Simon, Nikolay Kolyada, Christopher Akiki, Martin Potthast, Benno Stein, Norbert Siegmund

{"title":"Exploring Hyperparameter Usage and Tuning in Machine Learning Research","authors":"Sebastian Simon, Nikolay Kolyada, Christopher Akiki, Martin Potthast, Benno Stein, Norbert Siegmund","doi":"10.1109/CAIN58948.2023.00016","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00016","url":null,"abstract":"The success of machine learning (ML) models depends on careful experimentation and optimization of their hyperparameters. Tuning can affect the reliability and accuracy of a trained model and is the subject of ongoing research. However, little is known on whether and how hyperparameters are used and optimized in research practice. This lack of knowledge not only limits the adoption of best practices for tuning in research, but also affects the reproducibility of published results. Our research systematically analyzes the use and tuning of hyperparameters in ML publications. For this, we analyze 2000 code repositories and their associated research papers from Papers with Code. We compare the use and tuning of hyperparameters of three widely used ML libraries: scikit-learn, TensorFlow, and PyTorch. Our results show that the most of the available hyperparameters remain untouched, and those that have been changed use constant values. In particular, there is a significant difference between tuning hyperparameters and the reporting about it in the corresponding research papers. Our results suggest that there is a need for improved research and reporting practices when using ML methods to improve the reproducibility of published results.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128516069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Maintaining and Monitoring AIOps Models Against Concept Drift 维护和监控AIOps模型，防止概念漂移

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-05-01 DOI: 10.1109/CAIN58948.2023.00024

Lorena Poenaru-Olaru, Luís Cruz, Jan S. Rellermeyer, A. V. Deursen

引用次数: 2

A Meta-Summary of Challenges in Building Products with ML Components – Collecting Experiences from 4758+ Practitioners 用机器学习组件构建产品的挑战-收集4758+从业人员的经验

2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) Pub Date : 2023-03-31 DOI: 10.1109/CAIN58948.2023.00034

Nadia Nahar, Haoran Zhang, G. Lewis, Shurui Zhou, Christian Kästner

引用次数: 10