Bidirectional projective sampling for physics-based differentiable rendering

Abstract

Background

Physics-based differentiable rendering (PBDR) aims to propagate gradients from scene parameters to image pixels or vice versa. The physically correct gradients obtained can be used in various applications, including inverse rendering and machine learning. Currently, two categories of methods are prevalent in the PBDR community: reparameterization and boundary sampling methods. The state-of-the-art boundary sampling methods rely on a guiding structure to calculate the gradients efficiently. They utilize the rays generated in traditional path-tracing methods and project them onto the object silhouette boundary to initialize the guiding structure.

Methods

In this study, we propose an augmentation of previous projective-sampling-based boundary-sampling methods in a bidirectional manner. Specifically, we utilize the rays spawned from the sensors and also employ the rays emitted by the emitters to initialize the guiding structure.

Results

To demonstrate the benefits of our technique, we perform a comparative analysis of differentiable rendering and inverse rendering performance. We utilize a range of synthetic scene examples and evaluate our method against state-of-the-art projective-sampling-based differentiable rendering methods.

Conclusions

The experiments show that our method achieves lower variance gradients in the forward differentiable rendering process and better geometry reconstruction quality in the inverse-rendering results.