Latitude-oriented hierarchical enhancement network for omnidirectional image super-resolution

Abstract

Omnidirectional image super-resolution (ODISR) holds significant application potential in various industrial scenarios, such as virtual reality and autonomous driving. However, most existing super-resolution methods focus on standard 2D images and yield unsatisfactory ODISR performance, because omnidirectional images (ODIs) typically adopt the equirectangular projection (ERP) format, suffering from serious geometric distortion and differentiated texture features related to the latitude. In this paper, we propose a novel latitude-oriented hierarchical enhancement network (LOHE-Net) for ODISR, which allows features at different latitudes to obtain hierarchical enhancement. Specifically, we first exploit a hierarchical enhancement unit to divide an ERP feature map into different sub-regions according to the latitude and then perform distinct enhancement for these sub-regions, which can effectively address the differentiation of texture features, adapt the geometric distortion, and derive high-frequency information across latitudes in ERP ODIs. Subsequently, we introduce a distillation and spatial enhancement unit to progressively extract important information and further refine it in the spatial domain, boosting the representation ability with low computational cost. Extensive quantitative and qualitative experiments validate the superior ODISR performance and computational efficiency of our LOHE-Net.