RollingGantryCrane: Automation for unpacking containers into HPC environments

Abstract

Software containers are an important common currency for portability and reproducibility in the modern world of computing. While they are easy to share through public registries, usage documentation is often lacking, effectively leaving users with black boxes. RollingGantryCrane (RGC) is an open-source tool that takes generic software containers and automatically exposes the internal software through LMOD environment modules. Users provide the container URLs they wish to use, and RGC pulls the containers, collects descriptive metadata from public repositories, scans for non-standard executables on each container's search path, and generates LMOD modulefiles with help text and shell functions that transparently expose applications directly to the command line interface. RGC has been used in production since early 2019 on five production systems at The Texas Advanced Computing Center (TACC), allowing users to create bespoke modules and serving over 3000 unique tools from the BioContainers project.