Pythia: An Edge-First Agent for State Prediction in High-Dimensional Environments

Abstract

Modern deep learning agents usually operate in low-dimensional environments. They process pixel input, do not offer insights into their thought process, and require significant power and computational resources. These characteristics make them inapplicable for embedded devices. In this letter, we present Pythia, an edge-first framework that uses latent imagination to handle complex environments efficiently and envision future agent states. It utilizes a vector quantized variational autoencoder to reduce the high-dimensional features into a low-dimensional space, making it ideal for modern embedded devices. Moreover, Pythia offers human interpretable feedback and scales well with respect to the design space. Pythia surpassed the other state-of-the-art models in prediction accuracy on both intrinsic and extrinsic metrics.