Autonomous Vehicles Sensor Needs

Abstract

Fully Autonomous vehicles are becoming a reality, with many major players having development trials on actual streets. Yet, we have seen that in spite of the large investments, these systems today are still not superseding human driver limited safe driving capability - even with human supervision. There are also multiple other applications of such technologies, that are not about driving on streets, for industrial, infrastructure, home and health applications, where “robotic moving vehicles” have similar sensing and behavior needs. This presentation highlights the motivations behind the journey from advanced driver assistance systems (ADAS) to full autonomy of vehicles and what are the key driver sensing skills and behaviors that need to be replaced to complete a safe trip. The human drivers carry many “learnings” about the scenes and behaviors they observe from other drivers or pedestrians and this learning to a great extent can compensate for some inaccuracy of sensing by use of anticipation of behavior. The sensing in autonomous cars also has the burden of ensuring “sound” actions with a limited depth of learning, and hence multiple sensing of 3D surround scenes is essential for this robustness. Redundancy in such systems is also essential to ensure self-checking, and graceful exit when partial sensor failure happens. Active and passive sensing with propagating waves, be it optical, mmWave or ultrasonic are key elements for surround sensing. Other sensing modalities to predict automobile behavior, orientation, inclination, inertia and road surface are all essential for decision making. Moreover, without any driver, which is an ultimate goal of AV, the vehicle needs to interact with passengers or other “cargo” which opens another set of sensing needs to ensure their safety and desires. The presentation tries to highlight the key requirements of such sensing modalities, the key gaps that exist today in the capability of each sensing technology, and directions of where research in sensors can help to enable fully safe autonomy. Finally, while robust multi-modal sensing systems are essential, they are not sufficient for the success of such complex systems. The fusion of information of the sensing data, the continuous learning needed for every type of autonomous system, and the communication are all elements of such a robotic vehicle and play a major role in reaching the robustness, sound decisions and accuracies needed. It is becoming evident, that such a need will help drive semiconductor industry for years to come with all the demands and value it creates and hence, the role of research in closing the need gaps is essential for such growth.