2022 IEEE International Conference on Assured Autonomy (ICAA)最新文献_第2页

Selective Classification of Sequential Data Using Inductive Conformal Prediction 基于归纳共形预测的序列数据选择性分类

2022 IEEE International Conference on Assured Autonomy (ICAA) Pub Date : 2022-03-01 DOI: 10.1109/ICAA52185.2022.00015

Dimitrios Boursinos, X. Koutsoukos

{"title":"Selective Classification of Sequential Data Using Inductive Conformal Prediction","authors":"Dimitrios Boursinos, X. Koutsoukos","doi":"10.1109/ICAA52185.2022.00015","DOIUrl":"https://doi.org/10.1109/ICAA52185.2022.00015","url":null,"abstract":"Cyber-Physical Systems (CPS) operate in dynamic and uncertain environments where the use of deep neural networks (DNN) for perception can be advantageous. However, DNN integration in CPS is not straightforward. Perception outputs must be complemented with assurance metrics that represent if they can be trusted or not. Further, the inputs to DNNs are typically sequential capturing time-correlated data that can affect the accuracy of the predictions since machine learning models require inputs to be independent and identically distributed. In this paper, we propose a selective classification approach that rejects predictions that are not trustworthy. We quantify the credibility and confidence of each prediction by computing aggregate p-values from multiple subsequent inputs. We examine different multiple hypothesis testing approaches for combining p-values computed using Inductive Conformal Prediction (ICP) focusing on their ability to produce valid p-values for sequential data. Empirical evaluation results using the German Traffic Sign Recognition Benchmark demonstrate that ICP validity can be recovered when p-values from sequential inputs are combined and selective classification based on aggregate p-values produces predictions with less risk.","PeriodicalId":206047,"journal":{"name":"2022 IEEE International Conference on Assured Autonomy (ICAA)","volume":"35 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114013723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

[Copyright notice] (版权)

2022 IEEE International Conference on Assured Autonomy (ICAA) Pub Date : 2022-03-01 DOI: 10.1109/icaa52185.2022.00003

引用次数: 0

Resilient Multi-agent Reinforcement Learning Using Medoid and Soft-medoid Based Aggregation 基于介质和软介质聚合的弹性多智能体强化学习

2022 IEEE International Conference on Assured Autonomy (ICAA) Pub Date : 2022-03-01 DOI: 10.1109/ICAA52185.2022.00014

C. Bhowmick, Mudassir Shabbir, W. Abbas, X. Koutsoukos

引用次数: 0

Zero-Shot Policy Transfer in Autonomous Racing: Reinforcement Learning vs Imitation Learning 自动驾驶赛车中的零射击策略转移:强化学习与模仿学习

2022 IEEE International Conference on Assured Autonomy (ICAA) Pub Date : 2022-03-01 DOI: 10.1109/ICAA52185.2022.00011

Nathaniel P. Hamilton, Patrick Musau, Diego Manzanas Lopez, Taylor T. Johnson

{"title":"Zero-Shot Policy Transfer in Autonomous Racing: Reinforcement Learning vs Imitation Learning","authors":"Nathaniel P. Hamilton, Patrick Musau, Diego Manzanas Lopez, Taylor T. Johnson","doi":"10.1109/ICAA52185.2022.00011","DOIUrl":"https://doi.org/10.1109/ICAA52185.2022.00011","url":null,"abstract":"There are few technologies that hold as much promise in achieving safe, accessible, and convenient transportation as autonomous vehicles. However, as recent years have demonstrated, safety and reliability remain the most obstinate challenges, especially in complex domains. Autonomous racing has demonstrated unique benefits in that researchers can conduct research in controlled environments, allowing for experimentation with approaches that are too risky to evaluate on public roads. In this work, we compare two leading methods for training neural network controllers, Reinforcement Learning and Imitation Learning, for the autonomous racing task. We compare their viability by analyzing their performance and safety when deployed in novel scenarios outside their training via zero-shot policy transfer. Our evaluation is made up of a large number of experiments in simulation and on our real-world hardware platform that analyze whether these algorithms remain effective when transferred to the real-world. Our results show reinforcement learning outperforms imitation learning in most scenarios. However, the increased performance comes at the cost of reduced safety. Thus, both methods are effective under different criteria.","PeriodicalId":206047,"journal":{"name":"2022 IEEE International Conference on Assured Autonomy (ICAA)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134270130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Robustness Assurance Quotient: Demonstrating Context Matters for AI Performance and ML Security 鲁棒性保证商:展示人工智能性能和机器学习安全的上下文问题

2022 IEEE International Conference on Assured Autonomy (ICAA) Pub Date : 2022-03-01 DOI: 10.1109/ICAA52185.2022.00012

Samuel Lefcourt, Nathaniel G. Gordon, Hanting Wong, Gregory Falco

引用次数: 0

Measuring and Mitigating Bias in AI-Chatbots 人工智能聊天机器人的测量和减轻偏见

2022 IEEE International Conference on Assured Autonomy (ICAA) Pub Date : 2022-03-01 DOI: 10.1109/ICAA52185.2022.00023

Hedin Beattie, Lanier A Watkins, William H. Robinson, A. Rubin, Shari Watkins

引用次数: 5

Risk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems 自主系统动态保证的风险感知场景采样

2022 IEEE International Conference on Assured Autonomy (ICAA) Pub Date : 2022-02-28 DOI: 10.1109/ICAA52185.2022.00022

Shreyas Ramakrishna, Baiting Luo, Yogesh D. Barve, G. Karsai, A. Dubey

{"title":"Risk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems","authors":"Shreyas Ramakrishna, Baiting Luo, Yogesh D. Barve, G. Karsai, A. Dubey","doi":"10.1109/ICAA52185.2022.00022","DOIUrl":"https://doi.org/10.1109/ICAA52185.2022.00022","url":null,"abstract":"Autonomous Cyber-Physical Systems must often operate under uncertainties like sensor degradation and shifts in the operating conditions, which increases its operational risk. Dynamic Assurance of these systems requires designing runtime safety components like Out-of-Distribution detectors and risk estimators, which require labeled data from different operating modes of the system that belong to scenes with adverse operating conditions, sensors, and actuator faults. Collecting real-world data of these scenes can be expensive and sometimes not feasible. So, scenario description languages with samplers like random and grid search are available to generate synthetic data from simulators, replicating these real-world scenes. However, we point out three limitations in using these conventional samplers. First, they are passive samplers, which do not use the feedback of previous results in the sampling process. Second, the variables to be sampled may have constraints that are often not included. Third, they do not balance the tradeoff between exploration and exploitation, which we hypothesize is necessary for better search space coverage. We present a scene generation approach with two samplers called Random Neighborhood Search (RNS) and Guided Bayesian Optimization (GBO), which extend the conventional random search and Bayesian Optimization search to include the limitations. Also, to facilitate the samplers, we use a risk-based metric that evaluates how risky the scene was for the system. We demonstrate our approach using an Autonomous Vehicle example in CARLA simulation. To evaluate our samplers, we compared them against the baselines of random search, grid search, and Halton sequence search. Our samplers of RNS and GBO sampled a higher percentage of high-risk scenes of 83% and 92%, compared to 56% 66% and 71% of the grid, random and Halton samplers, respectively.","PeriodicalId":206047,"journal":{"name":"2022 IEEE International Conference on Assured Autonomy (ICAA)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127293251","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3