Space-time optical imaging framework: Holographic particle tracking velocimetry

We present a joint optimization framework for four-dimensional (4D) optical imaging: volumes and movements are reconstructed jointly in a higher space-time dimension, enabling faster convergence and better reconstruction quality of both the volumes and movements within a few minutes on modern GPUs. We verified the framework on digital holographic particle tracking velocimetry, a typical 4D optical imaging example. Experimental results are presented to show the efficiency of the proposed technique. © 2021 The Author(s) 1. Theory The temporal evolution of a volume o defines an apparent movement of the volume, known as fluid velocities. Motion flows can be numerically reconstructed from consecutive, rapid measurements of changing particle volumes. Under the assumption of incompressibility, i.e., divergence-free flow, the motion flow vt ∈ R3 relates temporally neighboring particle volumes from time t to t +1 [1]. That is, with x = (r,z): ot(x+vt)≈ ot+1(x), (1) where subscripts (t and t + 1) denote for two neighboring time frames. Given a sequence of indexed 2D images bt(r) of t = 1,2, · · · ,T , at each frame t, we would like to recover simultaneously its volume ot(x) and the associated 3D flow vector vt(x). By taking into consideration of the temporal coherence between neighboring frames, we jointly estimate volumes and fluid flows and this strategy optimizes for neighboring frames together, and hence closing the numerical reconstruction loop. These yield the following optimization problem: min o,v T ∑ t=1 ‖Aot −bt‖2 + τ T−1 ∑ t=1 ‖ot+1(x)−ot(x+vt)‖2 +priors(o)+priors(v), (2) where priors(v) may represent the smoothness and incompressible property [1,2] of the flow, and priors(o) represents the property of the volumes, usually is sparsity. Specifically, Eq. (2) is solved by alternating between solving particle volume o and solving volumetric movement flow v, as in Algorithm 1: min o T ∑ t=1 ‖Aot −bt‖2 + τ T−1 ∑ t=1 ‖ot+1(x)−ot(x+vt)‖2 +priors(o), min v τ T−1 ∑ t=1 ‖ot+1(x)−ot(x+vt)‖2 +priors(v). (3) Algorithm 1 Holo-Flow solver for Eq. (3). function RECONSTRUCT VOLUME AND FLOW(b1,b2, · · · ,bT ) Initialize o0 and v0; while not converge do . Alternating loop ok+1← argmin o T ∑ t=1 ‖Aot −bt‖2 +λ T−1 ∑ t=1 ‖ot+1(x)−ot(x+v t )‖2 +priors(o); . o-solver: volume [3] vk+1← argmin v λ T−1 ∑ t=1 ‖ok t+1(x)−o k t (x+vt)‖2 +priors(v); . v-solver: 3D optical flow [4] return oK and vK ; . At final iteration K In Eq. (2) and Eq. (3), A can be an optical system of holography or light field microscopy, where bt(r) are holograms or light field images. We take the holographic particle tracking velocimetry (HPTV) as an example. Specifically, we solve particles using FASTA iterative shrinkage algorithm [3], and solve flow using standard HornSchunck optical flow [4]. In the following, we evaluate our proposed approach on HPTV based on experimental hologram data. 2. Verification of the proposed technique with holographic PTV tank with moving particles laser diode CCD (a) Experimental setup. (b) Selected hologram frames. (c) Reconstructed fluid flow. Fig. 1: Verification of the proposed technique with holographic PTV. In the experiment, a tank containing seeded particles in high viscosity liquid was illuminated by a laser beam, as shown in Figure 1(a). We spun the water anticlockwisely from top of the tank and captured hologram sequences through a 4F system that consists of two achromatic lenses (focal length of 200 mm and 60 mm), as shown in Figure 1(a). In the flow reconstruction, we downsampled the holograms by a factor of 8 from an original resolution of 2048×2048 to 256×256. The depth was discretized into 101 layers (0.1 mm/layer with a voxel resolution of 55.2μm× 55.2μm× 100μm. The path-line visualization of the recovered flow field is shown in Figure 1(c). As expected, the swirl rotates clock-wisely along the y axis. The reconstruction only shows a part of the swirl because of the limited field of view of the experiment (around 23.6 mm), about one-half of the total tank width. The recovered flow field shown in Figure 1 is consistent with our expectation and the observed hologram frames. For the experiment, the outer iterations are 3, and the inner iteration of the hologram solver is 20, and τ = 0.001. Specific prior weights are tuned to plausible values. The reconstruction of the last experiment takes around 1 min on a Ubuntu 18.04 Linux workstation with 2.70GHz Intel(R) Xeon(R) CPU E5-2680, 62.9GB RAM, and a NVIDIA TITAN X (Pascal) GPU. Roughly 68% of that time was spent on the data transfer between CPU and GPU. Funding Information This work was supported by the KAUST individual baseline funding. References 1. J. Gregson, I. Ihrke, N. Thuerey, and W. Heidrich, “From capture to simulation,” ACM Transactions on Graph. 33, 1–11 (2014). 2. J. Xiong, R. Idoughi, A. A. Aguirre-Pablo, A. B. Aljedaani, X. Dun, Q. Fu, S. T. Thoroddsen, and W. Heidrich, “Rainbow particle imaging velocimetry for dense 3d fluid velocity imaging,” ACM Transactions on Graph. 36, 1–14 (2017). 3. T. Goldstein, C. Studer, and R. Baraniuk, “A field guide to forward-backward splitting with a fasta implementation,” arXiv preprint arXiv:1411.3406 (2014). 4. B. K. Horn and B. G. Schunck, “Determining optical flow,” in Techniques and Applications of Image Understanding, , vol. 281 (International Society for Optics and Photonics, 1981), pp. 319–331.