Quantifying the Sim2Real Gap: Model-Based Verification and Validation in Autonomous Ground Systems

Abstract

Quantifying the Sim2Real gap is crucial for validating autonomous ground systems, enabling robust algorithm testing in simulations before real-world deployment, thereby reducing costs and time. This study introduces the Vinnicombe ($\nu$-gap) metric as a quantitative tool for assessing this gap. To achieve this, a non-holonomic skid-steer differential drive robot was used. The $\nu$-gap metric compares two dynamical control systems and returns a value between 0 and 1, where 0 indicates identical systems and 1 indicates significantly different systems. A linear time-invariant dynamic model, optimized through a genetic algorithm, was employed to ensure accurate representation of system behavior across varying conditions. Unlike task-specific metrics focused on localized errors, the $\nu$-gap metric provides a holistic assessment by capturing system-wide differences. The $\nu$-gap metric quantified significant differences with a maximum of 0.64 between real-world and simulated trials highlighting discrepancies in vehicle-environment interactions. Terrain-induced changes in real-world comparisons were quantified with values up to 0.27, reflecting increased compliance and friction on rubber-like surfaces versus concrete. Internal system changes were also identified with $\nu$-gap values between 0.25 and 0.32, demonstrating sensitivity to changes in vehicle dynamics. These findings highlight the $\nu$-gap metric's utility in enhancing simulation fidelity and reducing reliance on resource-intensive real-world testing.