Parallel Eulerian-Lagrangian coupling method on hierarchical meshes

An Eulerian-Lagrangian coupling method based on hierarchical meshes is presented, which allows an efficient parallelization on high-performance computing hardware. It features an interleaved execution pattern with non-blocking communication, where the hierarchical mesh structure facilitates the redistribution of the computational load. The Lagrangian and Eulerian solvers use hierarchical Cartesian meshes which share a common coarse mesh level. The domain decomposition is based on a space-filling curve defined on the joint computational mesh, where the load is projected to a coarse mesh level used for the partitioning. The performance of the coupled method is evaluated for the problem of spray modeling in turbulent flow. A solution adaptive mesh is utilized for the large-eddy simulation of the flow field and the Lagrangian tracking method is used for the spray particles. Static and dynamic workload estimators are compared with respect to the alleviation of load imbalances. Liquid fuel spray injection in a constant pressure chamber and in an internal combustion engine serves as applications with varying scale resolution and localized computational load. The parallel efficiency of the approach on high performance systems is demonstrated for meshes with up to $2.8\cdot {10}^9$ cells and $21\cdot {10}^6$ particles. Detailed performance analyses show a performance gain of the novel algorithm of approx. 20% compared to a non-interleaved time step execution for two-way coupled spray injection simulations. Results of strong scaling experiments at different injection phases show a good parallel performance with an efficiency of up to 81% using 262000 MPI processes.