{"title":"儿童正常阑尾和阑尾炎 ED PoCUS 视频训练数据集的观察者间一致性。","authors":"James W Tsung, Maytal Firnberg, Philip Sosa","doi":"10.1186/s13089-024-00386-1","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Educational video datasets can be an effective method for training in emergency department (ED) point-of-care ultrasound (PoCUS). A video dataset for normal appendix and appendicitis in children using ED PoCUS images was developed to assess interobserver agreement, as measured by Cohen's Kappa on key sonographic findings.</p><p><strong>Methods: </strong>Three sets of 25 ED PoCUS videos were selected and curated from pediatric patients with normal appendix and acute appendicitis. Four participant ED sonologist-physicians were trained on the first set of 25 videos showing normal appendix or normal bowel in patients without appendicitis to note if normal appendix was seen in any part or in it's entirety from tip-to-cecum. They were then tested on the second set of similar videos. A third set of 25 videos from patients who had appendicitis where participant sonologists were asked to note if appendicitis was present or absent, with and without appendicolith or perforation. Cohen's Kappa was calculated in aggregate and stratified by experience vs. novice against a senior sonologist-physician aware of all patient outcomes for visualization of: 1. any part of normal appendix, 2. normal appendix visualized from tip to cecum 3. any part of appendicitis, 4. appendicolith, 5. appendiceal perforation.</p><p><strong>Results: </strong>Cohen's Kappa for any part of normal appendix, 0.71, 95% CI (0.58-0.85); normal appendix tip-to-cecum, 0.43, 95% CI (0.19-0.67), appendicitis, 0.53, 95%CI (0.34-0.70), appendicolith, 0.63, 95%CI (0.43-0.84), perforated appendicitis, 0.46, 95%CI (0.22-0.70). Stratified by experienced vs. novice: any part of normal appendix, 0.75 vs. 0.68; normal appendix tip-to-cecum, 0.50 vs. 0.36; appendicitis, 0.78 vs. 0.31; appendicolith, 0.75 vs. 0.5; perforated appendicitis, 0.5 vs 0.42.</p><p><strong>Conclusions: </strong>This educational video dataset may be used to train sonologist-physicians in ED PoCUS scanning for normal appendix and appendicitis in children. Sonologist experience affected interobserver agreement with respect to visualization of entire normal appendix and appendicitis.</p>","PeriodicalId":36911,"journal":{"name":"Ultrasound Journal","volume":null,"pages":null},"PeriodicalIF":3.4000,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11303676/pdf/","citationCount":"0","resultStr":"{\"title\":\"Interobserver agreement of an ED PoCUS video training dataset of normal appendix and appendicitis in children.\",\"authors\":\"James W Tsung, Maytal Firnberg, Philip Sosa\",\"doi\":\"10.1186/s13089-024-00386-1\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>Educational video datasets can be an effective method for training in emergency department (ED) point-of-care ultrasound (PoCUS). A video dataset for normal appendix and appendicitis in children using ED PoCUS images was developed to assess interobserver agreement, as measured by Cohen's Kappa on key sonographic findings.</p><p><strong>Methods: </strong>Three sets of 25 ED PoCUS videos were selected and curated from pediatric patients with normal appendix and acute appendicitis. Four participant ED sonologist-physicians were trained on the first set of 25 videos showing normal appendix or normal bowel in patients without appendicitis to note if normal appendix was seen in any part or in it's entirety from tip-to-cecum. They were then tested on the second set of similar videos. A third set of 25 videos from patients who had appendicitis where participant sonologists were asked to note if appendicitis was present or absent, with and without appendicolith or perforation. Cohen's Kappa was calculated in aggregate and stratified by experience vs. novice against a senior sonologist-physician aware of all patient outcomes for visualization of: 1. any part of normal appendix, 2. normal appendix visualized from tip to cecum 3. any part of appendicitis, 4. appendicolith, 5. appendiceal perforation.</p><p><strong>Results: </strong>Cohen's Kappa for any part of normal appendix, 0.71, 95% CI (0.58-0.85); normal appendix tip-to-cecum, 0.43, 95% CI (0.19-0.67), appendicitis, 0.53, 95%CI (0.34-0.70), appendicolith, 0.63, 95%CI (0.43-0.84), perforated appendicitis, 0.46, 95%CI (0.22-0.70). Stratified by experienced vs. novice: any part of normal appendix, 0.75 vs. 0.68; normal appendix tip-to-cecum, 0.50 vs. 0.36; appendicitis, 0.78 vs. 0.31; appendicolith, 0.75 vs. 0.5; perforated appendicitis, 0.5 vs 0.42.</p><p><strong>Conclusions: </strong>This educational video dataset may be used to train sonologist-physicians in ED PoCUS scanning for normal appendix and appendicitis in children. Sonologist experience affected interobserver agreement with respect to visualization of entire normal appendix and appendicitis.</p>\",\"PeriodicalId\":36911,\"journal\":{\"name\":\"Ultrasound Journal\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.4000,\"publicationDate\":\"2024-08-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11303676/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Ultrasound Journal\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1186/s13089-024-00386-1\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"Medicine\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ultrasound Journal","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1186/s13089-024-00386-1","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Medicine","Score":null,"Total":0}
引用次数: 0
摘要
背景:教育视频数据集是培训急诊科(ED)护理点超声(PoCUS)的有效方法。我们利用急诊科 PoCUS 图像开发了一个儿童正常阑尾和阑尾炎的视频数据集,以评估观察者之间的一致性(通过关键声像图结果的 Cohen's Kappa 进行测量):方法: 从阑尾正常和急性阑尾炎的儿科患者中挑选并整理了三组共 25 个 ED PoCUS 视频。四名参加培训的急诊室超声医师-物理师对第一组 25 个视频进行了培训,这些视频显示了无阑尾炎患者的正常阑尾或正常肠道,培训内容是注意阑尾从顶端到尾部的任何部分或全部是否正常。然后对第二组类似视频进行测试。第三组是 25 个阑尾炎患者的视频,要求声学专家注意阑尾炎是否存在,有无阑尾结石或穿孔。科恩卡帕(Cohen's Kappa)是根据经验与新手进行分层计算的,新手与了解所有患者结果的资深声学医师进行比较,以观察到以下情况:1.正常阑尾的任何部分;2.从阑尾顶端到盲肠的正常阑尾;3.阑尾炎的任何部分;4.阑尾结石;5.阑尾穿孔:结果:正常阑尾任何部位的科恩卡帕值为 0.71,95%CI (0.58-0.85);正常阑尾顶端至盲肠的科恩卡帕值为 0.43,95%CI (0.19-0.67);阑尾炎的科恩卡帕值为 0.53,95%CI (0.34-0.70);阑尾结石的科恩卡帕值为 0.63,95%CI (0.43-0.84);阑尾穿孔的科恩卡帕值为 0.46,95%CI (0.22-0.70)。根据有经验者与新手进行分层:正常阑尾的任何部位,0.75 vs. 0.68;正常阑尾顶端到盲肠,0.50 vs. 0.36;阑尾炎,0.78 vs. 0.31;阑尾结石,0.75 vs. 0.5;穿孔性阑尾炎,0.5 vs. 0.42:该教育视频数据集可用于培训声学医师-医师在 ED PoCUS 扫描正常阑尾和儿童阑尾炎方面的能力。声学医师的经验会影响观察者之间在观察整个正常阑尾和阑尾炎方面的一致性。
Interobserver agreement of an ED PoCUS video training dataset of normal appendix and appendicitis in children.
Background: Educational video datasets can be an effective method for training in emergency department (ED) point-of-care ultrasound (PoCUS). A video dataset for normal appendix and appendicitis in children using ED PoCUS images was developed to assess interobserver agreement, as measured by Cohen's Kappa on key sonographic findings.
Methods: Three sets of 25 ED PoCUS videos were selected and curated from pediatric patients with normal appendix and acute appendicitis. Four participant ED sonologist-physicians were trained on the first set of 25 videos showing normal appendix or normal bowel in patients without appendicitis to note if normal appendix was seen in any part or in it's entirety from tip-to-cecum. They were then tested on the second set of similar videos. A third set of 25 videos from patients who had appendicitis where participant sonologists were asked to note if appendicitis was present or absent, with and without appendicolith or perforation. Cohen's Kappa was calculated in aggregate and stratified by experience vs. novice against a senior sonologist-physician aware of all patient outcomes for visualization of: 1. any part of normal appendix, 2. normal appendix visualized from tip to cecum 3. any part of appendicitis, 4. appendicolith, 5. appendiceal perforation.
Results: Cohen's Kappa for any part of normal appendix, 0.71, 95% CI (0.58-0.85); normal appendix tip-to-cecum, 0.43, 95% CI (0.19-0.67), appendicitis, 0.53, 95%CI (0.34-0.70), appendicolith, 0.63, 95%CI (0.43-0.84), perforated appendicitis, 0.46, 95%CI (0.22-0.70). Stratified by experienced vs. novice: any part of normal appendix, 0.75 vs. 0.68; normal appendix tip-to-cecum, 0.50 vs. 0.36; appendicitis, 0.78 vs. 0.31; appendicolith, 0.75 vs. 0.5; perforated appendicitis, 0.5 vs 0.42.
Conclusions: This educational video dataset may be used to train sonologist-physicians in ED PoCUS scanning for normal appendix and appendicitis in children. Sonologist experience affected interobserver agreement with respect to visualization of entire normal appendix and appendicitis.