Hexagonal mesh-based neural rendering for real-time rendering and fast reconstruction

Abstract

Although recent neural rendering-based methods can achieve high-quality geometry and realistic rendering results in multi-view reconstruction, they incur a heavy computational burden on rendering and training, which limits their application scenarios. To address these challenges, we propose an effective mesh-based neural rendering approach which leverages the characteristic of meshes being able to achieve real-time rendering. Besides, a coarse-to-fine scheme is introduced to efficiently extract the initial mesh so as to significantly reduce the reconstruction time. More importantly, we suggest a hexagonal mesh model to preserve surface regularity by constraining the second-order derivatives of its vertices, where only low level of positional encoding is engaged for neural rendering. Experiments show that our approach significantly reduces the rendering time from several tens of seconds to 0.05s compared to methods based on implicit representation. And it can quickly achieve state-of-the-art results in novel view synthesis and reconstruction. Our full implementation will be made publicly available at https://github.com/FuchengSu/FastMesh.