A case study on raytracing-in-the-loop optimization: focal surface displays

Optimization-based design of optical systems can yield configurations that would be impractical to achieve with manual parameter adjustment. Nonetheless, most approaches are geared toward one-time, offline generation of static configurations to be fabricated physically. Recently, challenging computational imaging problems, such as seeing around corners or through scattering media, have utilized dynamically addressable optical elements to probe scene light transport. A new class of optimization techniques targeted at these dynamic applications has emerged in which stochastic raytracing replaces the fixed operators applied with conventional optimization methods. By modeling optical systems as raytracing operators, more complex non-linear phenomena and larger problem sizes can be considered. We introduce a simple raytracing-in-the-loop optimization model for a head-mounted display (HMD) containing a spatial light modulator (SLM). Using this approach, we are able to compute color images to be displayed in concert with spatially varying SLM phase maps at a resolution that would otherwise be computationally in-feasible. We also consider extensions of this model that may further enhance the performance of the target system.