FRISNET: A Fast Real-Time Instance Segmentation Network Fusing Frequency Domain and Multilevel Features

Abstract

It is challenging to obtain accurate location information of instances and segmentation masks, considering the intricacy and diversity of practical scenarios. This article presents a fast real-time instance segmentation network (FRISNET) by fusing the information from the frequency domain and space domain. Based on you only look at coefficients (YOLACT), which is the fastest instance segmentation method, the frequency domain representation is introduced into a convolutional neural network (CNN). By fast Fourier transform (FFT), features of different frequencies extracted from the frequency domain are fused with the characteristic map of the spatial domain. Accurate global location information and clear semantic information are obtained using CNN. To take advantage of the high-resolution information of target location and feature information from the bottom level, as well as the supervision information located at the entirely identical level, a brand fresh bottom-up feature fusion branch and skip connection at the same level are introduced based on the top-down feature pyramid network (FPN) feature fusion network, enabling the feature extraction network to possess diverse feature representation. The proposed instance segmentation model is trained on open standard datasets of PASCAL segmentation boundary detection (PASCAL SBD) and Microsoft Common Objects in Context (MS COCO). The results show that the proposed method improves instance segmentation accuracy. It achieves a mean average precision (mAP) of 34.5 at 31.77 frames/s (FPS) on MS COCO. This performance is 1.17% higher than YOLACT++ with the ResNet-50 architecture. The model’s speed also shows potential for use in motion planning and tactile sensors for robotic grasping tasks. This could further enhance execution efficiency and operational reliability.