[Evaluation of the performance of the artificial intelligence - enabled snail identification system for recognition of Oncomelania hupensis robertsoni and Tricula].

Abstract

Objective: To evaluate the performance of the artificial intelligence (AI)-enabled snail identification system for recognition of Oncomelania hupensis robertsoni and Tricula in schistosomiasis-endemic areas of Yunnan Province.

Methods: Fifty O. hupensis robertsoni and 50 Tricula samples were collected from Yongbei Township, Yongsheng County, Lijiang City, a schistosomiasis-endemic area in Yunnan Province in May 2024. A total of 100 snail sample images were captured with smartphones, including front-view images of 25 O. hupensis robertsoni and 25 Tricula samples (upward shell opening) and back-view images of 25 O. hupensis robertsoni and 25 Tricula samples (downward shell opening). Snail samples were identified as O. hupensis robertsoni or Tricula by schistosomiasis control experts with a deputy senior professional title and above according to image quality and morphological characteristics. A standard dataset for snail image classification was created, and served as a gold standard for recognition of snail samples. A total of 100 snail sample images were recognized with the AI-enabled intelligent snail identification system based on a WeChat mini program in smartphones. Schistosomiasis control professionals were randomly sampled from stations of schistosomisis prevention and control and centers for disease control and prevention in 18 schistosomiasis-endemic counties (districts, cities) of Yunnan Province, for artificial identification of 100 snail sample images. All professionals are assigned to two groups according the median years of snail survey experiences, and the effect of years of snail survey experiences on O. hupensis robertsoni sample image recognition was evaluated. A receiver operating characteristic (ROC) curve was plotted, and the sensitivity, specificity, accuracy, Youden's index and the area under the curve (AUC) of the AI-enabled intelligent snail identification system and artificial identification were calculated for recognition of snail sample images. The snail sample image recognition results of AI-enabled intelligent snail identification system and artificial identification were compared with the gold standard, and the internal consistency of artificial identification results was evaluated with the Cronbach's coefficient alpha.

Results: A total of 54 schistosomiasis control professionals were sampled for artificial identification of snail sample image recognition, with a response rate of 100% (54/54), and the accuracy, sensitivity, specificity, Youden's index, and AUC of artificial identification were 90%, 86%, 94%, 0.80 and 0.90 for recognition of snail sample images, respectively. The overall Cronbach's coefficient alpha of artificial identification was 0.768 for recognition of snail sample images, and the Cronbach's coefficient alpha was 0.916 for recognition of O. hupensis robertsoni snail sample images and 0.925 for recognition of Tricula snail sample images. The overall accuracy of artificial identification was 90% for recognition of snail sample images, and there was no significant difference in the accuracy of artificial identification for recognition of O. hupensis robertsoni (86%) and Tricula snail sample images (94%) (χ² = 1.778, P > 0.05). There was no significant difference in the accuracy of artificial identification for recognition of snail sample images with upward (88%) and downward shell openings (92%) (χ² = 0.444, P > 0.05), and there was a significant difference in the accuracy of artificial identification for recognition of snail sample images between schistosomiasis control professionals with snail survey experiences of 6 years and less (75%) and more than 6 years (90%) (χ² = 7.792, P < 0.05). The accuracy, sensitivity, specificity and AUC of the AI-enabled intelligent snail identification system were 88%, 100%, 76% and 0.88 for recognition of O. hupensis robertsoni snail sample images, and there was no significant difference in the accuracy of recognition of O. hupensis robertsoni snail sample images between the AI-enabled intelligent snail identification system and artificial identification (χ² = 0.204, P > 0.05). In addition, there was no significant difference in the accuracy of artificial identification for recognition of snail sample images with upward (90%) and downward shell openings (86%) (χ² = 0.379, P > 0.05), and there was a significant difference in the accuracy of artificial identification for recognition of snail sample images between schistosomiasis control professionals with snail survey experiences of 6 years and less and more than 6 years (χ² = 5.604, P_adjusted < 0.025).

Conclusions: The accuracy of recognition of snail sample images is comparable between the AI-enabled intelligent snail identification system and artificial identification by schistosomiasis control professionals, and the AI-enabled intelligent snail identification system is feasible for recognition of O. hupensis robertsoni and Tricula in Yunnan Province.