ECSS Experience: Particle Tracing Reinvented

This work describes an implementation of distributed particle tracking that provides a factor 10000x speedup over traditional schemes. While none of the techniques used to achieve this result are completely new, they have been used in combination to great effect in this project. The implementation includes parallel IO using HDF5, a flexible load balancing scheme, and dynamic buffering to achieve excellent performance at scale. The use of HDF5 decouples the size of the simulation generating the data from the particle tracing, providing a more flexible and efficient workflow. The load balancing scheme ensures that heterogeneous particle distributions do not result in a waste of computational resources by maintaining all the MPI tasks occupied at any given time. Dynamic buffering minimizes MPI exchanges across MPI tasks, a critical element in the performance improvements achieved.