Comparing Strategies for Visualizing the High-Dimensional Exploration Behavior of CPS Design Agents

Abstract

The design of cyber-physical systems often involves search within high-dimensional design spaces. When evaluating the performance of algorithms in tasks such as these, the patterns of exploration are often informative and can help support algorithm selection. However, accurately representing these patterns in a way that is human understandable while still preserving the nuanced search complexities in the high-dimensional space is nontrivial. This work specifically examines approaches for visualizing the search trajectories of reinforcement learning agents. We assess trajectories on two exemplar problems: the design of a racecar and the design of an aerial vehicle. We compare and contrast the visualizations produced using PCA, t-SNE, UMAP, TriMap, and PaCMAP. Future work should extend this comparison to a wider variety of exemplar design problems and consider the additional challenges posed by set-based design algorithms (e.g., genetic algorithms, particle swarm optimization).