A Hierarchical Temporal Memory Based End-to-End Autonomous Driving System

Abstract

Achieving human-level driving performance in complex environments remains a major challenge in the field of Deep Learning (DL) based end-to-end Autonomous Driving Systems (ADS). In ADS, generalization to rare edge cases poses a serious safety concern with DL based models. The leading solution to this problem is scaling; the construction of larger models and datasets. However, limitations in the computational power available to autonomous vehicles, coupled with the under-representation of safety-critical edge cases in large autonomous driving datasets raise questions over the suitability of scaling for ADS. In this work, we investigate the performance of an alternate, computationally less demanding, Machine Learning (ML) algorithm, Hierarchical Temporal Memory (HTM). Existing HTM models use rudimentary encoding schemes that have thus far limited their application to simple inputs. Motivated by this shortcoming, we first propose a bespoke CNN based encoding scheme suited to the input data used in ADS. We then integrate this encoding scheme into a novel DL-HTM end-to-end ADS. The proposed DL-HTM based end-to-end ADS is trained and evaluated against a conventional DL end-to-end ADS based on the widely used AlexNet model from the literature. Our evaluation results show that the proposed DL-HTM model achieves comparable performance with far fewer trainable parameters than the conventional DL based end-to-end ADS. Results also indicate that the proposed model demonstrates a superior capacity for learning under-represented classes, i.e. edge cases, in the dataset.