Deep Learning Attention-Ranked Media Space Generation for Virtual Reality Equirectangular Scene Augmentation

Abstract

Virtual Reality has fastened its growth radicalising industries such as tertiary education, marketing, and entertainment. Developments in virtual world-building like the Metaverse yields challenges such as the prohibitive technical skill requirement. This work constructed a method of generating attention-ranked media spaces through deep learning as a solution to this issue mitigating unskilled demand for scene augmentation. Two segmentation tasks were addressed: true-perspective view-port media space inferencing, and gaze attention predictions for equirectangular 360-degree projections. Combining results produced ranked spaces providing multimedia implantation locations. Ablation studies assessed TranSalNet, a leading attention Transformer, for attention-saliency accounting for model pre-encoders. This was compared against U-Net for media space generation. Weak attention supervision and architecture overparameterisation limitations were addressed with modified Salient Object Subitizing and DT-Fixup algorithms respectively. These contributions yielded an overall improvement from second-best models demonstrating experimental success.