Semantic Segmentation with Perceiver IO

Abstract

Recently, in deep learning, the transformer is replacing the convolutional neural network (CNN) due to its performance and simple design. In particular, in recent studies, constructing an encoder of the transformer that effectively extracts features on an image has been widely used. However, even in these cases, models utilizing existing deep neural network structures needed to use a form suitable for each data format according to input modality. Recently, the Perceiver IO [6] has been proposed to overcome this limitation. It can process various data formats through one structure to extract a characteristic value. Also, it uses an output query to output data as we want. In this paper, a semantic segmentation model using the characteristics of the Perceiver IO is presented. Two types of input configuration are suggested, and experimental results show the feasibility of the proposed method.