{"title":"Analysis of neural network takeover-time predictions for shared-control autonomous driving","authors":"C. Pasareanu","doi":"10.1145/3459086.3459630","DOIUrl":"https://doi.org/10.1145/3459086.3459630","url":null,"abstract":"Autonomous driving systems may encounter situations where it is necessary to transfer control to the human driver, for instance when encountering unpredictable dangerous road conditions. To be able to do so safely, the autonomous system needs an estimate of how long it will take for the human driver to take control of the vehicle. A neural network can be used for making such predictions. However ensuring that such a neural network can be used in safety-critical situations is very challenging. We discuss our recent efforts for building, analysing and formally verifying a neural network built for predicting takeover time in a shared-control autonomous driving system. The network was trained on data collected from a (semi-) autonomous driving simulator. We evaluated several techniques for the analysis of the neural network as follows. We performed robustness and sensitivity analysis for the neural network, using the Marabou formal verification tool. We evaluated off-the-shelf attribution tools to determine the important features upon which the neural network makes its predictions. We investigated trust and confidence analysis to better understand the neural network outputs. And finally, we performed adversarial training to improve the quality of the neural network. We discuss our results and outline directions for future work.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128438852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"RoboStar modelling stack: tackling the reality gap","authors":"Ana Cavalcanti","doi":"10.1145/3459086.3459628","DOIUrl":"https://doi.org/10.1145/3459086.3459628","url":null,"abstract":"RoboStar technology for model-based Software Engineering for Robotics enables the construction of artefacts that capture and relate assumptions that can play a role in the reality. In this paper, we give a brief overview of the RoboStar approach.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"20 4","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114023867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using control synthesis for falsification and corner case generation","authors":"N. Ozay","doi":"10.1145/3459086.3459632","DOIUrl":"https://doi.org/10.1145/3459086.3459632","url":null,"abstract":"This talk will describe algorithms that search for \"dynamical adversarial examples\" or \"corner cases\" for feedback control systems. This problem is related to the falsification problem, where the goal is to find initial conditions, disturbance profiles, and environment behaviors that force the system to violate its specifications. As opposed to the commonly adopted falsification approaches that treat the system under test as a black-box, we propose a synthesis-guided approach, which leverages the knowledge of a plant model if it exists and treats only the controller and perception mechanism as black-box. Our algorithm uses the plant model and backward reachable set computations to guide the search for falsifying trajectories. We will demonstrate the approach with examples from autonomous systems, including those using perception-based neural network controllers.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114821272","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards rigorous neural control","authors":"Sicun Gao","doi":"10.1145/3459086.3459633","DOIUrl":"https://doi.org/10.1145/3459086.3459633","url":null,"abstract":"Learning-based and data-driven approaches are becoming an indispensable part of robotic systems. Control and planning components based on neural networks challenge existing methods for ensuring reliability and safety of these systems. By taking a numerical and statistical perspective on synthesis and verification, we believe it is possible to still prove strong properties for highly nonlinear systems with highly nonlinear control laws. Interestingly, we often need to make use of inductive certificates that are themselves neural networks. I will survey some of our ongoing work in these directions towards rigorous neural control for nonlinear systems with \"provably approximate\" safety and reliability guarantees.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124591073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Safe control from signal temporal logic specifications using recurrent neural networks","authors":"C. Belta","doi":"10.1145/3459086.3459629","DOIUrl":"https://doi.org/10.1145/3459086.3459629","url":null,"abstract":"Temporal logics, such as Linear Temporal Logic (LTL) and Computation Tree Logic (CTL), have been traditionally used as specification languages for digital circuits and computer programs. Due to their expressivity, the availability of tools for verification and synthesis, and recent advances in abstraction techniques for dynamical systems, temporal logics have been increasing used as specification languages in robotics biology, and autonomous driving applications.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"72 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114493232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards combining deep learning, verification, and scenario-based programming","authors":"Guy Katz, Achiya Elyasaf","doi":"10.1145/3459086.3459631","DOIUrl":"https://doi.org/10.1145/3459086.3459631","url":null,"abstract":"Deep learning (DL) [4] is dramatically changing the world of software. The rapid improvement in deep neural network (DNN) technology now enables engineers to train models that achieve superhuman results, often surpassing algorithms that have been carefully hand-crafted by domain experts [19, 20]. There is even an intensifying trend of incorporating DNNs in safety-critical systems, e.g. as controllers for autonomous vehicles and drones [1, 12].","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134465846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Safety and reliability of deep learning: (brief overview)","authors":"Xiaowei Huang","doi":"10.1145/3459086.3459636","DOIUrl":"https://doi.org/10.1145/3459086.3459636","url":null,"abstract":"Robotics and Autonomous Systems (RAS) become ever more relying on deep learning components to support their perception and decision making. Given RAS will inevitably be applied to safety critical applications, efforts are needed to ensure that the deep learning is safe and reliable. In this lecture, I will give a brief overview on recent progress in the verification and validation techniques for deep learning, focusing on two major safety and reliability risks, i.e., robustness and generalisation. We consider formal verification, statistical evaluation, reliability assessment, and runtime monitoring techniques, all of which complement with each other in providing assurance to the reliability of deep learning in operation. The challenges and future directions will also be discussed.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"65 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128918384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards verifying neural autonomous systems","authors":"A. Lomuscio","doi":"10.1145/3459086.3459627","DOIUrl":"https://doi.org/10.1145/3459086.3459627","url":null,"abstract":"In this talk I will offer a personal perspective on the increasing challenges and the correspondingly more powerful solutions being developed in the area of verification of autonomous system. I will ground the presentation on the work carried out with several colleagues over the years in the Verification of Autonomous Systems at Imperial College London [32].","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115636047","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Specification development for autonomous system behavior","authors":"Signe A. Redfield","doi":"10.1145/3459086.3459634","DOIUrl":"https://doi.org/10.1145/3459086.3459634","url":null,"abstract":"Before we can verify a system, we need a specification to verify it against. But defining a specification for an autonomous behavior is a challenging problem. In addition to simply describing what the system needs to do, we need to ensure that when non-experts specify desired behavior, they provide enough information to the designer without imposing a significant unnecessary cost or complexity burden on the developer. Combining a capability representation with a capability analysis table enables the definition of a boundary between the specification and the design.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121603046","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Verifiable autonomy under perceptual limitations","authors":"U. Topcu","doi":"10.1145/3459086.3459635","DOIUrl":"https://doi.org/10.1145/3459086.3459635","url":null,"abstract":"A recent set of algorithms in the intersection of formal methods, convex optimization and machine learning offers orders-of-magnitude improvement in the scalability of verification and synthesis in partially observable Markov decision processes possibly with uncertain transition probabilities.","PeriodicalId":127610,"journal":{"name":"Proceedings of the 1st International Workshop on Verification of Autonomous & Robotic Systems","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116178813","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}