2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)最新文献

Data acquisition and the implications of machine learning in the development of a Clinical Decision Support system 数据采集和机器学习在临床决策支持系统开发中的意义

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-05-01 DOI: 10.1109/WAIN52551.2021.00022

Milan Unger

{"title":"Data acquisition and the implications of machine learning in the development of a Clinical Decision Support system","authors":"Milan Unger","doi":"10.1109/WAIN52551.2021.00022","DOIUrl":"https://doi.org/10.1109/WAIN52551.2021.00022","url":null,"abstract":"The abundance of healthcare data, with the collection of population-wide information in Electronical Medical Records, would be promising for the implementation of products using artificial intelligence and machine learning. This enables development of new advanced software applications for the clinical practice, especially for the large vendors with years long experience in developing medical software application. Nevertheless, the introduction of artificial intelligence and machine learning to the product development process makes the daily life of software engineers more challenging and brings new factors to consider during the development of a product that must meet the high standards of clinical world. This paper describes experience with the software development of a Clinical Decision Support system at Siemens Healthineers. The intention of the project is to build a software platform for the handling of patient longitudinal data and to provide supportive functionalities to the clinician, with application of Machine Learning and Artificial Intelligence methods to deliver relevant information to the user.","PeriodicalId":224912,"journal":{"name":"2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124321599","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems 基于人在环视觉机器人系统的自适应自治

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-28 DOI: 10.1109/WAIN52551.2021.00025

Sophia Abraham, Zachariah Carmichael, Sreya Banerjee, Rosaura G. VidalMata, Ankit Agrawal, M. N. A. Islam, W. Scheirer, J. Cleland-Huang

{"title":"Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems","authors":"Sophia Abraham, Zachariah Carmichael, Sreya Banerjee, Rosaura G. VidalMata, Ankit Agrawal, M. N. A. Islam, W. Scheirer, J. Cleland-Huang","doi":"10.1109/WAIN52551.2021.00025","DOIUrl":"https://doi.org/10.1109/WAIN52551.2021.00025","url":null,"abstract":"Computer vision approaches are widely used by autonomous robotic systems to sense the world around them and to guide their decision making as they perform diverse tasks such as collision avoidance, search and rescue, and object manipulation. High accuracy is critical, particularly for Human-on-the-loop (HoTL) systems where decisions are made autonomously by the system, and humans play only a supervisory role. Failures of the vision model can lead to erroneous decisions with potentially life or death consequences. In this paper, we propose a solution based upon adaptive autonomy levels, whereby the system detects loss of reliability of these models and responds by temporarily lowering its own autonomy levels and increasing engagement of the human in the decision-making process. Our solution is applicable for vision-based tasks in which humans have time to react and provide guidance. When implemented, our approach would estimate the reliability of the vision task by considering uncertainty in its model, and by performing covariate analysis to determine when the current operating environment is illmatched to the model’s training data. We provide examples from DroneResponse, in which small Unmanned Aerial Systems are deployed for Emergency Response missions, and show how the vision model’s reliability would be used in addition to confidence scores to drive and specify the behavior and adaptation of the system’s autonomy. This workshop paper outlines our proposed approach and describes open challenges at the intersection of Computer Vision and Software Engineering for the safe and reliable deployment of vision models in the decision making of autonomous systems.","PeriodicalId":224912,"journal":{"name":"2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133319425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 13

Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems 机器学习系统中不匹配的表征和检测

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-25 DOI: 10.1109/WAIN52551.2021.00028

G. Lewis, S. Bellomo, I. Ozkaya

{"title":"Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems","authors":"G. Lewis, S. Bellomo, I. Ozkaya","doi":"10.1109/WAIN52551.2021.00028","DOIUrl":"https://doi.org/10.1109/WAIN52551.2021.00028","url":null,"abstract":"Increasing availability of machine learning (ML) frameworks and tools, as well as their promise to improve solutions to data-driven decision problems, has resulted in popularity of using ML techniques in software systems. However, end-to-end development of ML-enabled systems, as well as their seamless deployment and operations, remain a challenge. One reason is that development and deployment of ML-enabled systems involves three distinct workflows, perspectives, and roles, which include data science, software engineering, and operations. These three distinct perspectives, when misaligned due to incorrect assumptions, cause ML mismatches which can result in failed systems. We conducted an interview and survey study where we collected and validated common types of mismatches that occur in end-to-end development of ML-enabled systems. Our analysis shows that how each role prioritizes the importance of relevant mismatches varies, potentially contributing to these mismatched assumptions. In addition, the mismatch categories we identified can be specified as machine readable descriptors contributing to improved ML-enabled system development. In this paper, we report our findings and their implications for improving end-to-end ML-enabled system development.","PeriodicalId":224912,"journal":{"name":"2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123418592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 24

Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report 工程智能作文评分和反馈系统:经验报告

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-25 DOI: 10.1109/WAIN52551.2021.00029

A. Chadda, Kelly Song, Raman Chandrasekar, I. Gorton

{"title":"Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report","authors":"A. Chadda, Kelly Song, Raman Chandrasekar, I. Gorton","doi":"10.1109/WAIN52551.2021.00029","DOIUrl":"https://doi.org/10.1109/WAIN52551.2021.00029","url":null,"abstract":"Artificial Intelligence (AI)/Machine Learning (ML)-based systems are widely sought-after commercial solutions that can automate and augment core business services. Intelligent systems can improve the quality of services offered and support scalability through automation. In this paper we describe our experience in engineering an exploratory system for assessing the quality of essays supplied by customers of a specialized recruitment support service. The problem domain is challenging because the open-ended customer-supplied source text has considerable scope for ambiguity and error, making models for analysis hard to build. There is also a need to incorporate specialized business domain knowledge into the intelligent processing systems. To address these challenges, we experimented with and exploited a number of cloud-based machine learning models and composed them into an application-specific processing pipeline. This design allows for modification of the underlying algorithms as more data and improved techniques become available. We describe our design, and the main challenges we faced, namely keeping a check on the quality control of the models, testing the software and deploying the computationally expensive ML models on the cloud.","PeriodicalId":224912,"journal":{"name":"2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)","volume":"106 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114252808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Lessons Learned from Educating AI Engineers 培养人工智能工程师的经验教训

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-19 DOI: 10.1109/WAIN52551.2021.00013

P. Heck, Gerard Schouten

引用次数: 5

Requirement Engineering Challenges for AI-intense Systems Development 人工智能密集系统开发的需求工程挑战

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-18 DOI: 10.1109/WAIN52551.2021.00020

Hans-Martin Heyn, E. Knauss, Amna Pir Muhammad, O. Eriksson, Jennifer Linder, P. Subbiah, Shameer Kumar Pradhan, Sagar Tungal

{"title":"Requirement Engineering Challenges for AI-intense Systems Development","authors":"Hans-Martin Heyn, E. Knauss, Amna Pir Muhammad, O. Eriksson, Jennifer Linder, P. Subbiah, Shameer Kumar Pradhan, Sagar Tungal","doi":"10.1109/WAIN52551.2021.00020","DOIUrl":"https://doi.org/10.1109/WAIN52551.2021.00020","url":null,"abstract":"Availability of powerful computation and communication technology as well as advances in artificial intelligence enable a new generation of complex, AI-intense systems and applications. Such systems and applications promise exciting improvements on a societal level, yet they also bring with them new challenges for their development. In this paper we argue that significant challenges relate to defining and ensuring behaviour and quality attributes of such systems and applications. We specifically derive four challenge areas from relevant use cases of complex, AI-intense systems and applications related to industry, transportation, and home automation: understanding, determining, and specifying (i) contextual definitions and requirements, (ii) data attributes and requirements, (iii) performance definition and monitoring, and (iv) the impact of human factors on system acceptance and success. Solving these challenges will imply process support that integrates new requirements engineering methods into development approaches for complex, AI-intense systems and applications. We present these challenges in detail and propose a research roadmap.","PeriodicalId":224912,"journal":{"name":"2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116446372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 19

Towards Productizing AI/ML Models: An Industry Perspective from Data Scientists 走向产品化AI/ML模型:来自数据科学家的行业视角

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-18 DOI: 10.1109/WAIN52551.2021.00027

F. Lanubile, Fabio Calefato, L. Quaranta, Maddalena Amoruso, Fabio Fumarola, Michele Filannino

引用次数: 4

Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help? 谁需要MLOps:数据科学家寻求完成什么以及MLOps如何提供帮助?

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-16 DOI: 10.1109/WAIN52551.2021.00024

Sasu Mäkinen, Henrik Skogström, Eero Laaksonen, T. Mikkonen

{"title":"Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?","authors":"Sasu Mäkinen, Henrik Skogström, Eero Laaksonen, T. Mikkonen","doi":"10.1109/WAIN52551.2021.00024","DOIUrl":"https://doi.org/10.1109/WAIN52551.2021.00024","url":null,"abstract":"Following continuous software engineering practices, there has been an increasing interest in rapid deployment of machine learning (ML) features, called MLOps. In this paper, we study the importance of MLOps in the context of data scientists’ daily activities, based on a survey where we collected responses from 331 professionals from 63 different countries in ML domain, indicating on what they were working on in the last three months. Based on the results, up to 40% respondents say that they work with both models and infrastructure; the majority of the work revolves around relational and time series data; and the largest categories of problems to be solved are predictive analysis, time series data, and computer vision. The biggest perceived problems revolve around data, although there is some awareness of problems related to deploying models to production and related procedures. To hypothesise, we believe that organisations represented in the survey can be divided to three categories – (i) figuring out how to best use data; (ii) focusing on building the first models and getting them to production; and (iii) managing several models, their versions and training datasets, as well as retraining and frequent deployment of retrained models. In the results, the majority of respondents are in category (i) or (ii), focusing on data and models; however the benefits of MLOps only emerge in category (iii) when there is a need for frequent retraining and redeployment. Hence, setting up an MLOps pipeline is a natural step to take, when an organization takes the step from ML as a proof-of-concept to ML as a part of nominal activities.","PeriodicalId":224912,"journal":{"name":"2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129699457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 86

MLOps Challenges in Multi-Organization Setup: Experiences from Two Real-World Cases MLOps在多组织设置中的挑战:来自两个现实案例的经验

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-16 DOI: 10.1109/WAIN52551.2021.00019

Tuomas Granlund, Aleksi Kopponen, Vlad Stirbu, Lalli Myllyaho, T. Mikkonen

引用次数: 24

Understanding and Modeling AI-Intensive System Development 理解和建模ai密集型系统开发

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) Pub Date : 2021-03-16 DOI: 10.1109/WAIN52551.2021.00015

L. Lavazza, S. Morasca

{"title":"Understanding and Modeling AI-Intensive System Development","authors":"L. Lavazza, S. Morasca","doi":"10.1109/WAIN52551.2021.00015","DOIUrl":"https://doi.org/10.1109/WAIN52551.2021.00015","url":null,"abstract":"Developers of AI-Intensive Systems—i.e., systems that involve both “traditional” software and Artificial Intelligence—are recognizing the need to organize development systematically and use engineered methods and tools. Since an AI-Intensive System (AIIS) relies heavily on software, it is expected that Software Engineering (SE) methods and tools can help. However, AIIS development differs from the development of “traditional” software systems in a few substantial aspects. Hence, traditional SE methods and tools are not suitable or sufficient by themselves and need to be adapted and extended.A quest for “SE for AI” methods and tools has started. We believe that, in this effort, we should learn from experience and avoid repeating some of the mistakes made in the quest for SE in past years. To this end, a fundamental instrument is a set of concepts and a notation to deal with AIIS and the problems that characterize their development processes.In this paper, we propose to describe AIIS via a notation that was proposed for SE and embeds a set of concepts that are suitable to represent AIIS as well. We demonstrate the usage of the notation by modeling some characteristics that are particularly relevant for AIIS.","PeriodicalId":224912,"journal":{"name":"2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN)","volume":"134 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123160647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0