Transparent integration of autonomous vehicles simulation tools with a data-centric middleware

Abstract

Simulations are key steps in the design, implementation, and verification of autonomous vehicles (AV). Parallel to this, typical simulation tools fail to integrate the entirety of the aspects related to the complexity of AV applications, such as data communication delay, security, and the integration of software/hardware-in-the-loop and other simulation tools. This work proposes a SmartData-based middleware to integrate AV simulators and external tools. The interface models the data used on a simulator and creates an intermediary layer between the simulator and the external tools by defining the inputs and outputs as SmartData. A message bus is used for communication between SmartData following their Interest relations. Messages are exchanged following a specific protocol. Nevertheless, the architecture presented is agnostic of protocol. Moreover, we present a data-centric AV design integrated into the middleware. The design considers the standardization of the data interfaces between AV components, including sensing, perception, planning, decision, and actuation. Therefore, the presented design promotes a transparent integration of the AV simulation with other simulators (e.g., network simulators), cloud services, fault injection mechanisms, digital twins, and hardware-in-the-loop scenarios. Moreover, the design allows for transparent, runtime component replacement and time synchronization, the modularization of the vehicle components, and the addition of security aspects in the simulation. We present a case-study application with an AV simulation using CARLA, and we measure the end-to-end delay and overhead incurred in the simulation by our middleware. An increase in the end-to-end delay was measured once data communication was not acknowledged in the original scenario, and data was assumed to be ready for processing with no communication delay between sensors, decision-making, and actuation units.