Tuna defect classification and grading using Twins transformer

Abstract

Ensuring the quality and safety of food products is of paramount importance within the food processing industry. Particularly in the seafood sector, the detection and classification of different quality defects in processed tuna loins poses a significant challenge, usually demanding the visual assessment by seasoned experts. This research proposes a technical solution to the tuna quality inspection using computer vision techniques to identify and localize different types of defects in contrast to what is considered the “standard” of a cleaned product, while additionally assessing the severity level of such defects affecting each individual loin. Image data of tuna defects are acquired under industrial conditions and compose two different datasets: a 4-common-defect dataset (TunaDefect-4) and a 6-extended-defect dataset (TunaDefect-6) including two additional types that are less common but of greater technical challenge. The quality grading process comprises 3 main steps. (1) Initially, preprocessing normalizes image input and augments the image dataset. (2) Then, a semantic segmentation model Twins-PCPVT-L, a pyramid vision transformer with self-attention and conditional positioning encoding, is employed for the TunaDefect-4 dataset. For the TunaDefect-6, a Twins-SVT-L, which amends the former model with locally-group self-attention and global sub-sampled attention, is used. The Twins-PCPVT-L applied to TunaDefect-4 has a mean pixel accuracy (mPA) of 93.96% and a mean IoU of 80.4%; while the Twins-SVT-L on the TunaDefect-6, results in an mPA of 83.82% and mIoU of 66.96%. (3) Lastly, the semantically segmented images are graded by severity ranging from level 0 to 4, where level 0 represents a fully cleaned loin and level 4, being the highest severity level, assigned to loins completely covered by various defects. The accuracy of severity grading is 84% for TunaDefect-4 and 76.6% for TunaDefect-6. Both models run within a total inference and processing time of approximately 0.20 s, faster than the conveyor's transport time. A web application prototype has been developed for the tuna quality classification and grading and is hosted on the Google Cloud Platform (GCP). The developed application responds in timely manner, to be used as a complementary identification and grading tool, with the potential to be integrated as an inline processing solution to further provide practicality to the industry.