mobile YOLACT: Toward Lightweight Instance Segmentation for Mobile Devices

In this paper, we present a lightweight instance segmentation model, mobileYOLACT which is designed for mobile environments where the computational resources are limited. We propose several modifications to YOLACT to improve computational efficiency. First, we use a quantized lightweight backbone for feature extraction. Second, we reduce the computational burden with marginal degradation in accuracy by employing the depthwise separable convolution on the entire model. Third, we simplified the structure of prototype mask generation branch. Last, we used TorchScript and NCNN to further optimize the model and deploy it on mobile device. We validate the effectiveness of the proposed method from the experiments on COCO dataset. The proposed model can run at the speed of 21 FPS on Samsung Galaxy S20 with 23 APmask at 0.5 IoU threshold.