Accelerating advection via approximate block exterior flow maps

Abstract

Flow visualization techniques involving extreme advection workloads are becoming increasingly popular. While these techniques often produce insightful images, the execution times to carry out the corresponding computations are lengthy. With this work, we introduce an alternative to traditional advection, which improves on performance at the cost of decreased accuracy. Our approach centers around block exterior flow maps (BEFMs), which can be used to accelerate flow computations by reducing redundant calculations. Our algorithm uses Lagrangian interpolation, but falls back to Eulerian advection whenever regions of high error are encountered. In our study, we demonstrate that the BEFM-based approach can lead to significant savings in time, with limited loss in accuracy. Introduction A myriad of scientific simulations, including those modeling fluid flow, astrophysics, fusion, thermal hydraulics, and others, model phenomena where constituents move through their volume. This movement is captured by a velocity field stored at every point on the mesh. Further, other vector fields, such as force fields for electricity, magnetism, and gravity, also govern movement and interaction. A wide range of flow visualization techniques are used to understand such vector fields. The large majority of these techniques rely on placing particles in the volume and analyzing the trajectories they follow. Traditionally, the particles are displaced through the volume using an advection step, i.e., solving an ordinary differential equation using a Runge-Kutta integrator. As computational power on modern desktops has increased, flow visualization algorithms have been empowered to consider designs that include more and more particles advecting for longer and longer periods. Techniques such as Line Integral Convolution and Finite-Time Lyapunov Exponents (FTLE) seed particles densely in a volume and examine where these particles end up. For these operations, and many others, only the ending position of the particle is needed, and not the details of the path the particle took to get there. Despite seemingly abundant computational power, some techniques have excessively long running times. For example, ocean modelers often study the FTLE within an ocean with both high seeding density and very long durations for the particles (years of simulation time) [2, 3]. As another example, fusion scientists are interested in FTLE computations inside a tokamak where particles travel for hundreds of rotations [1]. In both cases, FTLE calculations, even on supercomputers, can take tens of minutes. With this work, we consider an alternative to traditional Eulerian advection. The key observation that motivates the work is that, in conditions with dense seeding and long durations, particles will tread the same (or very similar) paths over and over. Where the current paradigm carries out the same computation over and over, we consider a new paradigm where a computation can be carried out a single time, and then reused. That said, we find that, while particle trajectories do often travel quite close to each other, they typically follow their own (slightly) unique paths. Therefore, to effectively reuse computations, we consider a method where we interpolate new trajectories from existing ones, effectively trading accuracy for speed. Our method depends on Block Exterior Flow Maps, or BEFMs. The idea behind BEFMs is to pre-compute known trajectories that lie on block boundaries. It assumes data in blockdecomposed, but this assumption is common when dealing with parallel, distributed-memory computations. When a computeintensive flow visualization algorithm is then calculated, it consults with the BEFMs and does Lagrangian-style interpolation from its known trajectories. While this approach introduces error, it can be considerably faster, since it avoids Eulerian advection steps inside each block. The contributions of the paper are as follows: • Introduction of BEFMs as an operator for accelerating dense particle advection calculations; • A novel method for generating an approximate BEFM that can be used in practice; • A study that evaluates the approximate BEFM approach, including comparisons with traditional advection. Related Work McLouglin et al. recently surveyed the state of the art in flow visualization [4], and the large majority of techniques they described incorporate particle advection. Any of these techniques could possibly benefit from the BEFM approach, although the tradeoff in accuracy is only worthwhile for those that have extreme computational costs, e.g., Line Integral Convolution [5], finite-time Lyapunov exponents [6], and Poincare analysis [7]. One solution for dealing with extreme advection workloads is parallelization. A summary of strategies for parallelizing particle advection problems on CPU clusters can be found in [8]. The basic approaches are to parallelize-over-data, parallelizeover-particles, or a hybrid of the two [9]. Recent results using parallelization-over-data demonstrated streamline computation on up to 32,768 processors and eight billion cells [11]. These parallelization approaches are complementary with our own. That is, traditional parallel approaches can be used in the current way, but the phase where they advect particles through a region could be replaced by our BEFM approach. In terms of precomputation, the most notable related work comes from Nouanesengsy et al. [10]. They precomputed flow patterns within a region and used the resulting statistics to decide which regions to load. While their precomputation and ours have similar elements, we are using the results of the precomputation in different ways: Nouanesengsy et al. for load balancing and ourselves to replace multiple integrations with one interpolation. In terms of accelerating particle advection through approximation, two works stand out. Brunton et al. [18] also looked at accelerating FTLE calculation, but they considered the unsteady state problem, and used previous calculations to accelerate new ones. While this is a compelling approach, it does not help with the steady state problem we consider. Hlwatsch et al. [15] employ an approach where flow is calculated by following hierarchical lines. This approach is well-suited for their use case, where all data fits within the memory of a GPU, but it is not clear how to build and connect hierarchical lines within a distributed memory parallel setting. In contrast, our method, by focusing on flow between exteriors of blocks, is well-suited for this type of parallelism. Bhatia et al. [19] studied edge maps, and the properties of flow across edge maps. While this work clearly has some similar elements to our, their focus was more on topology and accuracy, and less on accelerating particle advection workloads. Scientific visualization algorithms are increasingly using Lagrangian calculations of flow. Jobard et al. [12] presented a Lagrangian-Eulerian advection scheme which incorporated forward advection with a backward tracing Lagrangian step to more accurately shift textures during animation. Salzbrunn et al. delivered a technique for analyzing circulation and detecting vortex cores given predicates from pre-computed sets of streamlines [14] and pathlines [13]. Agranovsky et al. [16] focused on extracting a basis of Lagrangian flows as an in situ compression operator, while Chandler at al. [17] focused on how to interpolate new pathlines from arbitrary existing sets. Of these works, none share our focus on accelerating advection. Method Our method makes use of block exterior flow maps (BEFM). We begin by defining this mapping, in Section . We then describe our method, and how it incorporates these maps, in Section . Block Exterior Flow Map Definition In scientific computing, parallel simulation codes often partition their spatial volume over their compute nodes. Restated, each compute node will operate on one spatial region, and that compute node will be considered the “owner” of that region. Such a region is frequently referred to as a block. For example, a simulation over the spatial region X: [0-1], Y: [0-1], and Z: [0-1] and having N compute nodes could have N blocks, with each block covering a volume of 1 N . Consider a point P that lies on the exterior of a block B. If the velocity field points toward the interior of B at point P, then Eulerian advection of a particle originating at P will take the particle through the interior of B until it exits. In this case, the particle will exit B at some location P′, where P′ is also located on the exterior of B. The BEFM captures this mapping. The BEFM’s domain is all spatial locations on the exterior of blocks, and its range is also spatial locations on the exteriors of blocks. Further, for any given P in the BEFM’s domain, BEFM(P,B) will produce a location that is on B’s exterior. Saying it concisely, the BEFM is the mapping from particles at exteriors of blocks to the locations where those particles will exit the block under Eulerian advection. Figure 1 illustrates an example of a BEFM.