Variability of interobserver interpretation of selected helminth ova in the development of a training image set.

IF 2.3 4区医学 Q2 PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH

International Health Pub Date : 2025-01-09 DOI:10.1093/inthealth/ihae085

Rupert Stephen Charles S Chua, Kiersten A Henderson, Lorenzo Maria C de Guzman, Vicki Foss, Nathaniel Schub, Cameron Bell, John Robert C Medina, Taggart G Siao, Myra S Mistica, Maria Luz B Belleza, Marie Cris R Modequillo, Nadine Joyce C Torres, Vicente Y Belizario

{"title":"Variability of interobserver interpretation of selected helminth ova in the development of a training image set.","authors":"Rupert Stephen Charles S Chua, Kiersten A Henderson, Lorenzo Maria C de Guzman, Vicki Foss, Nathaniel Schub, Cameron Bell, John Robert C Medina, Taggart G Siao, Myra S Mistica, Maria Luz B Belleza, Marie Cris R Modequillo, Nadine Joyce C Torres, Vicente Y Belizario","doi":"10.1093/inthealth/ihae085","DOIUrl":null,"url":null,"abstract":"Background: Diagnosis of soil-transmitted helminthiasis and schistosomiasis for surveillance relies on microscopic detection of ova in Kato-Katz (KK) prepared slides. Artificial intelligence (AI)-based platforms for parasitic eggs may be developed using a robust image set with defined labels by reference microscopists. This study aimed to determine interobserver variability among reference microscopists in identifying parasite ova.Methods: Images of parasite ova taken from KK prepared slides were labelled according to species by two reference microscopists (M1 and M2). A third reference microscopist (M3) labelled images when the first two did not agree. Frequency, percent agreement, κ statistics and variability score (VS) were generated for analysis.Results: M1 and M2 agreed on 89.24% of the labelled images (κ=0.86, p<0.001). M3 had agreement with M1 and M2 (κ=0.30, p<0.001 and κ=0.28, p<0.001), resolving 89.29% of disagreement between them. The labelling of Schistosoma japonicum had the highest VS (κ=0.487, p=0.101) among the targeted ova. Reference microscopists were able to reliably reach consensus in 99.0% of the dataset.Conclusions: Training AI using this image set may provide more objective and reliable readings compared with that of reference microscopists.","PeriodicalId":49060,"journal":{"name":"International Health","volume":" ","pages":""},"PeriodicalIF":2.3000,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Health","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1093/inthealth/ihae085","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH","Score":null,"Total":0}

引用次数: 0

Abstract

Background: Diagnosis of soil-transmitted helminthiasis and schistosomiasis for surveillance relies on microscopic detection of ova in Kato-Katz (KK) prepared slides. Artificial intelligence (AI)-based platforms for parasitic eggs may be developed using a robust image set with defined labels by reference microscopists. This study aimed to determine interobserver variability among reference microscopists in identifying parasite ova.

Methods: Images of parasite ova taken from KK prepared slides were labelled according to species by two reference microscopists (M1 and M2). A third reference microscopist (M3) labelled images when the first two did not agree. Frequency, percent agreement, κ statistics and variability score (VS) were generated for analysis.

Results: M1 and M2 agreed on 89.24% of the labelled images (κ=0.86, p<0.001). M3 had agreement with M1 and M2 (κ=0.30, p<0.001 and κ=0.28, p<0.001), resolving 89.29% of disagreement between them. The labelling of Schistosoma japonicum had the highest VS (κ=0.487, p=0.101) among the targeted ova. Reference microscopists were able to reliably reach consensus in 99.0% of the dataset.

Conclusions: Training AI using this image set may provide more objective and reliable readings compared with that of reference microscopists.

查看原文本刊更多论文

在训练图像集的开发中，选定的蠕虫卵的观察者间解释的可变性。

背景：用于监测的土壤传播性寄生虫病和血吸虫病的诊断依赖于显微镜检测Kato-Katz （KK）制备的载玻片中的卵。参考显微镜可以使用具有定义标签的鲁棒图像集开发基于人工智能（AI）的寄生卵平台。本研究的目的是确定在鉴定寄生虫卵的参考显微镜之间的观察者差异。方法：用两台参考显微镜（M1和M2）对KK制备的载玻片上的寄生虫卵进行分类标记。当前两位不同意时，第三位参考显微镜（M3）标记图像。生成频率、一致性百分比、κ统计量和变异性评分（VS）进行分析。结果：M1和M2对标记图像的一致性为89.24% (κ=0.86)，结论：与参考显微镜相比，使用该图像集训练AI可以提供更客观可靠的读数。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Health PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH-

CiteScore

4.50

自引率

0.00%

发文量

审稿时长

>12 weeks

期刊介绍： International Health is an official journal of the Royal Society of Tropical Medicine and Hygiene. It publishes original, peer-reviewed articles and reviews on all aspects of global health including the social and economic aspects of communicable and non-communicable diseases, health systems research, policy and implementation, and the evaluation of disease control programmes and healthcare delivery solutions. It aims to stimulate scientific and policy debate and provide a forum for analysis and opinion sharing for individuals and organisations engaged in all areas of global health.