Convolutional Neural Networks for Morphologically Similar Fish Species Identification

Seafood comprises the largest globally traded food commodity in the world. Its supply chains are complex, focus on quick distribution, and rely on processing practices that make it difficult to trace products to their source. This has resulted in seafood mislabeling, with investigations revealing mislabeling of more than 30% of marketed seafood products, though the full extent of seafood mislabeling in the U.S. is unknown. When two species are morphologically similar, it is difficult for humans to visually distinguish between them, thus making mislabeling difficult to detect. To address this problem, we present a novel deep-learning-based model to distinguish between morphologically similar fish species in images. Our approach uses transfer learning with state-of-the-art convolutional neural networks (CNN) to build upon previously learned features on millions of images, thereby improving the model’s classification accuracy. We compare three pretrained CNNs: VGG, ResNet, and RegNet. For evaluation, we utilize the FishNet Open Image Database, containing over 85,000 images from electronic monitoring footage of fisheries. We train and test two models: a 4-species classifier of visually-similar tuna species, and a binary classifier of visually-indistinguishable tuna often mislabeled. Our results show CNNs can be used to distinguish between morphologically similar fish species with high accuracy, which otherwise would often be mislabeled by humans.