Ana Leni Frei, Amjad Khan, Raphaël Oberson, Stefan Reinhard, Yara Banz, Frédérique Meeuwsen, Andrew Janowczyk, Rainer Grobholz, Heather E Dawson, Alessandro Lugli, Marius Ilié, Jeroen van der Laak, Inti Zlobec
{"title":"由医学生、住院医师和病理学家进行的计算机辅助肿瘤细胞分数(TCF)评估提高了观察者之间的一致性,同时突出了自动化偏差的风险。","authors":"Ana Leni Frei, Amjad Khan, Raphaël Oberson, Stefan Reinhard, Yara Banz, Frédérique Meeuwsen, Andrew Janowczyk, Rainer Grobholz, Heather E Dawson, Alessandro Lugli, Marius Ilié, Jeroen van der Laak, Inti Zlobec","doi":"10.1007/s00428-025-04163-w","DOIUrl":null,"url":null,"abstract":"<p><p>The potential for computer-aided diagnostics (CAD) to augment learning outcomes for medical education in histopathology are vast. However, over-reliance in CAD tools could lead to important errors if the output is inaccurate, such as automation bias. Here, we analyze the influence of a CAD tool for tumor cell fraction (TCF) estimation, named TCFCAD, on medical students with limited histopathological expertise, compared to pathology residents and practicing pathologists. Participants were third-year medical students from the University of Bern (n = 18), residents and pathologists from the European Masters in Molecular Pathology (EMMP; n = 23), and 63 practicing pathologists stratified by < 20 or > 20 years of professional experience (n = 28 and 32, respectively). Each group evaluated 10 colorectal cancer regions of interest (ROI) with and without TCFCAD assistance. The ground truth (GT) was evaluated by a laborious count of all tumor and non-tumor cells per ROI. All groups demonstrated reduced variability in TCF scores with TCFCAD assistance. The standard deviation (SD) of the TCF scores compared to the GT values before and after TCFCAD assistance were 9.09 to 4.95 for medical students, 9.93 to 5.55 for EMMP, and 9.98 to 5.69 and 9.9 to 6.13 for pathologists with < 20 and > 20 years of experience. For one image, both medical students and EMMP participants' scores swayed to the TCFCAD's output despite (1) tool output being inaccurate and (2) participants' original scores without TCFCAD assistance being closer to the GT value. This confirmed a trend of medical students following the tool's recommendation across all ROIs. All groups benefitted most from TCFCAD output for a tumor with marked inflammatory infiltrates. Although TCFCAD improved interobserver agreement, independently of professional experience, there is a demonstrated danger in over-relying on its output. This important bias must be addressed when training students and pathologists on the use of CAD tools.</p>","PeriodicalId":23514,"journal":{"name":"Virchows Archiv","volume":" ","pages":""},"PeriodicalIF":3.1000,"publicationDate":"2025-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Computer-aided tumor cell fraction (TCF) estimation by medical students, residents, and pathologists improves inter-observer agreement while highlighting the risk of automation bias.\",\"authors\":\"Ana Leni Frei, Amjad Khan, Raphaël Oberson, Stefan Reinhard, Yara Banz, Frédérique Meeuwsen, Andrew Janowczyk, Rainer Grobholz, Heather E Dawson, Alessandro Lugli, Marius Ilié, Jeroen van der Laak, Inti Zlobec\",\"doi\":\"10.1007/s00428-025-04163-w\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>The potential for computer-aided diagnostics (CAD) to augment learning outcomes for medical education in histopathology are vast. However, over-reliance in CAD tools could lead to important errors if the output is inaccurate, such as automation bias. Here, we analyze the influence of a CAD tool for tumor cell fraction (TCF) estimation, named TCFCAD, on medical students with limited histopathological expertise, compared to pathology residents and practicing pathologists. Participants were third-year medical students from the University of Bern (n = 18), residents and pathologists from the European Masters in Molecular Pathology (EMMP; n = 23), and 63 practicing pathologists stratified by < 20 or > 20 years of professional experience (n = 28 and 32, respectively). Each group evaluated 10 colorectal cancer regions of interest (ROI) with and without TCFCAD assistance. The ground truth (GT) was evaluated by a laborious count of all tumor and non-tumor cells per ROI. All groups demonstrated reduced variability in TCF scores with TCFCAD assistance. The standard deviation (SD) of the TCF scores compared to the GT values before and after TCFCAD assistance were 9.09 to 4.95 for medical students, 9.93 to 5.55 for EMMP, and 9.98 to 5.69 and 9.9 to 6.13 for pathologists with < 20 and > 20 years of experience. For one image, both medical students and EMMP participants' scores swayed to the TCFCAD's output despite (1) tool output being inaccurate and (2) participants' original scores without TCFCAD assistance being closer to the GT value. This confirmed a trend of medical students following the tool's recommendation across all ROIs. All groups benefitted most from TCFCAD output for a tumor with marked inflammatory infiltrates. Although TCFCAD improved interobserver agreement, independently of professional experience, there is a demonstrated danger in over-relying on its output. This important bias must be addressed when training students and pathologists on the use of CAD tools.</p>\",\"PeriodicalId\":23514,\"journal\":{\"name\":\"Virchows Archiv\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":3.1000,\"publicationDate\":\"2025-07-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Virchows Archiv\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1007/s00428-025-04163-w\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PATHOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Virchows Archiv","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s00428-025-04163-w","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PATHOLOGY","Score":null,"Total":0}
Computer-aided tumor cell fraction (TCF) estimation by medical students, residents, and pathologists improves inter-observer agreement while highlighting the risk of automation bias.
The potential for computer-aided diagnostics (CAD) to augment learning outcomes for medical education in histopathology are vast. However, over-reliance in CAD tools could lead to important errors if the output is inaccurate, such as automation bias. Here, we analyze the influence of a CAD tool for tumor cell fraction (TCF) estimation, named TCFCAD, on medical students with limited histopathological expertise, compared to pathology residents and practicing pathologists. Participants were third-year medical students from the University of Bern (n = 18), residents and pathologists from the European Masters in Molecular Pathology (EMMP; n = 23), and 63 practicing pathologists stratified by < 20 or > 20 years of professional experience (n = 28 and 32, respectively). Each group evaluated 10 colorectal cancer regions of interest (ROI) with and without TCFCAD assistance. The ground truth (GT) was evaluated by a laborious count of all tumor and non-tumor cells per ROI. All groups demonstrated reduced variability in TCF scores with TCFCAD assistance. The standard deviation (SD) of the TCF scores compared to the GT values before and after TCFCAD assistance were 9.09 to 4.95 for medical students, 9.93 to 5.55 for EMMP, and 9.98 to 5.69 and 9.9 to 6.13 for pathologists with < 20 and > 20 years of experience. For one image, both medical students and EMMP participants' scores swayed to the TCFCAD's output despite (1) tool output being inaccurate and (2) participants' original scores without TCFCAD assistance being closer to the GT value. This confirmed a trend of medical students following the tool's recommendation across all ROIs. All groups benefitted most from TCFCAD output for a tumor with marked inflammatory infiltrates. Although TCFCAD improved interobserver agreement, independently of professional experience, there is a demonstrated danger in over-relying on its output. This important bias must be addressed when training students and pathologists on the use of CAD tools.
期刊介绍:
Manuscripts of original studies reinforcing the evidence base of modern diagnostic pathology, using immunocytochemical, molecular and ultrastructural techniques, will be welcomed. In addition, papers on critical evaluation of diagnostic criteria but also broadsheets and guidelines with a solid evidence base will be considered. Consideration will also be given to reports of work in other fields relevant to the understanding of human pathology as well as manuscripts on the application of new methods and techniques in pathology. Submission of purely experimental articles is discouraged but manuscripts on experimental work applicable to diagnostic pathology are welcomed. Biomarker studies are welcomed but need to abide by strict rules (e.g. REMARK) of adequate sample size and relevant marker choice. Single marker studies on limited patient series without validated application will as a rule not be considered. Case reports will only be considered when they provide substantial new information with an impact on understanding disease or diagnostic practice.