Myoungho Kim, Joohwan Seo, Mingoo Lee, Jongeun Choi
{"title":"基于视觉的深度强化学习不确定性车道保持策略","authors":"Myoungho Kim, Joohwan Seo, Mingoo Lee, Jongeun Choi","doi":"10.1115/1.4050396","DOIUrl":null,"url":null,"abstract":"\n Recent deep learning techniques promise high hopes for self-driving cars while there are still many issues to be addressed such as uncertainties (e.g., extreme weather conditions) in learned models. In this work, for the uncertainty-aware lane keeping, we first propose a convolutional mixture density network (CMDN) model that estimates the lateral position error, the yaw angle error, and their corresponding uncertainties from the camera vision. We then establish a vision-based uncertainty-aware lane keeping strategy in which a high-level reinforcement learning policy hierarchically modulates the reference longitudinal speed as well as the low-level lateral control. Finally, we evaluate the robustness of our strategy against the uncertainties of the learned CMDN model coming from unseen or noisy situations, as compared to the conventional lane keeping strategy without taking into account such uncertainties. Our uncertainty-aware strategy outperformed the conventional lane keeping strategy, without a lane departure in our test scenario during high-uncertainty periods with random occurrences of fog and rain situations on the road. The successfully trained deep reinforcement learning agent slows down the vehicle speed and tries to minimize the lateral error during high uncertainty situations similarly to what human drivers would do in such situations.","PeriodicalId":54846,"journal":{"name":"Journal of Dynamic Systems Measurement and Control-Transactions of the Asme","volume":null,"pages":null},"PeriodicalIF":1.7000,"publicationDate":"2021-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"9","resultStr":"{\"title\":\"Vision-Based Uncertainty-Aware Lane Keeping Strategy Using Deep Reinforcement Learning\",\"authors\":\"Myoungho Kim, Joohwan Seo, Mingoo Lee, Jongeun Choi\",\"doi\":\"10.1115/1.4050396\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"\\n Recent deep learning techniques promise high hopes for self-driving cars while there are still many issues to be addressed such as uncertainties (e.g., extreme weather conditions) in learned models. In this work, for the uncertainty-aware lane keeping, we first propose a convolutional mixture density network (CMDN) model that estimates the lateral position error, the yaw angle error, and their corresponding uncertainties from the camera vision. We then establish a vision-based uncertainty-aware lane keeping strategy in which a high-level reinforcement learning policy hierarchically modulates the reference longitudinal speed as well as the low-level lateral control. Finally, we evaluate the robustness of our strategy against the uncertainties of the learned CMDN model coming from unseen or noisy situations, as compared to the conventional lane keeping strategy without taking into account such uncertainties. Our uncertainty-aware strategy outperformed the conventional lane keeping strategy, without a lane departure in our test scenario during high-uncertainty periods with random occurrences of fog and rain situations on the road. The successfully trained deep reinforcement learning agent slows down the vehicle speed and tries to minimize the lateral error during high uncertainty situations similarly to what human drivers would do in such situations.\",\"PeriodicalId\":54846,\"journal\":{\"name\":\"Journal of Dynamic Systems Measurement and Control-Transactions of the Asme\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.7000,\"publicationDate\":\"2021-08-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"9\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Dynamic Systems Measurement and Control-Transactions of the Asme\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1115/1.4050396\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Dynamic Systems Measurement and Control-Transactions of the Asme","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1115/1.4050396","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
Vision-Based Uncertainty-Aware Lane Keeping Strategy Using Deep Reinforcement Learning
Recent deep learning techniques promise high hopes for self-driving cars while there are still many issues to be addressed such as uncertainties (e.g., extreme weather conditions) in learned models. In this work, for the uncertainty-aware lane keeping, we first propose a convolutional mixture density network (CMDN) model that estimates the lateral position error, the yaw angle error, and their corresponding uncertainties from the camera vision. We then establish a vision-based uncertainty-aware lane keeping strategy in which a high-level reinforcement learning policy hierarchically modulates the reference longitudinal speed as well as the low-level lateral control. Finally, we evaluate the robustness of our strategy against the uncertainties of the learned CMDN model coming from unseen or noisy situations, as compared to the conventional lane keeping strategy without taking into account such uncertainties. Our uncertainty-aware strategy outperformed the conventional lane keeping strategy, without a lane departure in our test scenario during high-uncertainty periods with random occurrences of fog and rain situations on the road. The successfully trained deep reinforcement learning agent slows down the vehicle speed and tries to minimize the lateral error during high uncertainty situations similarly to what human drivers would do in such situations.
期刊介绍:
The Journal of Dynamic Systems, Measurement, and Control publishes theoretical and applied original papers in the traditional areas implied by its name, as well as papers in interdisciplinary areas. Theoretical papers should present new theoretical developments and knowledge for controls of dynamical systems together with clear engineering motivation for the new theory. New theory or results that are only of mathematical interest without a clear engineering motivation or have a cursory relevance only are discouraged. "Application" is understood to include modeling, simulation of realistic systems, and corroboration of theory with emphasis on demonstrated practicality.