Anthony V. D'Antoni, Nancy Kamel, R. Shane Tubbs, Morgan G. McCartan, Laine W. Strobel, Kathleen C. Bubb
{"title":"解剖荟萃分析关键评估工具的心理测量特性。","authors":"Anthony V. D'Antoni, Nancy Kamel, R. Shane Tubbs, Morgan G. McCartan, Laine W. Strobel, Kathleen C. Bubb","doi":"10.1002/ca.24263","DOIUrl":null,"url":null,"abstract":"<div>\n \n <p>The hallmark of evidence-based anatomy (EBA) is the anatomical meta-analysis (AMA). The Critical Appraisal Tool for Anatomical Meta-Analysis (CATAM) was recently published to enable users to appraise AMAs quickly and effectively. The tool is valuable for students and clinicians who need to judge the quality of AMAs, which informs clinical decision making and results in better patient care. Subjective measures of the tool's face and content validity have been established, but establishing its reliability provides a more objective measure of the instrument's dependability. This study investigated the interrater reliability (IRR) of the CATAM between novice and expert raters. Three graduate students and three professors (two anatomists and one pharmacist) read the original CATAM paper, and then had a post hoc meeting to discuss scoring with the tool. Three recent AMAs (published between 2017 and 2022) were randomly chosen from PubMed, and all six raters scored the papers blindly. The intraclass correlation coefficient (ICC) statistic was used to calculate the interrater reliability (IRR) between all scores, and then the ICCs between novice and expert scores were compared. Cronbach's alpha (internal consistency) of the CATAM was also calculated (SPSS 25, Armonk, NY). ICC for AMA-1 was 0.999 (95% CI, 0.997–0.999), <i>p</i> = 0.000, and alpha was 0.999. ICC for AMA-2 was 0.994 (95% CI, 0.988–0.998), <i>p</i> = 0.000, and alpha was 0.994. ICC for AMA-3 was 0.998 (95% CI, 0.995–0.999), <i>p</i> = 0.000, and alpha was 0.998. ANOVA showed no significant differences (<i>p</i> > 0.05) in mean ICCs between raters. The CATAM is a robust tool with excellent IRR (ICC > 0.990) and internal consistency (alpha > 0.990). No significant difference in ICC scores between novices and experts suggests the tool does not require prior expert knowledge to be effective. Now that the reliability of the CATAM is established, it can be more widely adopted by students and physicians worldwide to evaluate the quality of AMAs. The CATAM offers widespread applicability, and can be adopted in medical education, journal clubs, and clinical seminars to critically evaluate AMAs.</p>\n </div>","PeriodicalId":50687,"journal":{"name":"Clinical Anatomy","volume":"38 3","pages":"355-361"},"PeriodicalIF":2.3000,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Psychometric Properties of the Critical Appraisal Tool for Anatomical Meta-Analysis\",\"authors\":\"Anthony V. D'Antoni, Nancy Kamel, R. Shane Tubbs, Morgan G. McCartan, Laine W. Strobel, Kathleen C. Bubb\",\"doi\":\"10.1002/ca.24263\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n <p>The hallmark of evidence-based anatomy (EBA) is the anatomical meta-analysis (AMA). The Critical Appraisal Tool for Anatomical Meta-Analysis (CATAM) was recently published to enable users to appraise AMAs quickly and effectively. The tool is valuable for students and clinicians who need to judge the quality of AMAs, which informs clinical decision making and results in better patient care. Subjective measures of the tool's face and content validity have been established, but establishing its reliability provides a more objective measure of the instrument's dependability. This study investigated the interrater reliability (IRR) of the CATAM between novice and expert raters. Three graduate students and three professors (two anatomists and one pharmacist) read the original CATAM paper, and then had a post hoc meeting to discuss scoring with the tool. Three recent AMAs (published between 2017 and 2022) were randomly chosen from PubMed, and all six raters scored the papers blindly. The intraclass correlation coefficient (ICC) statistic was used to calculate the interrater reliability (IRR) between all scores, and then the ICCs between novice and expert scores were compared. Cronbach's alpha (internal consistency) of the CATAM was also calculated (SPSS 25, Armonk, NY). ICC for AMA-1 was 0.999 (95% CI, 0.997–0.999), <i>p</i> = 0.000, and alpha was 0.999. ICC for AMA-2 was 0.994 (95% CI, 0.988–0.998), <i>p</i> = 0.000, and alpha was 0.994. ICC for AMA-3 was 0.998 (95% CI, 0.995–0.999), <i>p</i> = 0.000, and alpha was 0.998. ANOVA showed no significant differences (<i>p</i> > 0.05) in mean ICCs between raters. The CATAM is a robust tool with excellent IRR (ICC > 0.990) and internal consistency (alpha > 0.990). No significant difference in ICC scores between novices and experts suggests the tool does not require prior expert knowledge to be effective. Now that the reliability of the CATAM is established, it can be more widely adopted by students and physicians worldwide to evaluate the quality of AMAs. The CATAM offers widespread applicability, and can be adopted in medical education, journal clubs, and clinical seminars to critically evaluate AMAs.</p>\\n </div>\",\"PeriodicalId\":50687,\"journal\":{\"name\":\"Clinical Anatomy\",\"volume\":\"38 3\",\"pages\":\"355-361\"},\"PeriodicalIF\":2.3000,\"publicationDate\":\"2025-01-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Clinical Anatomy\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/ca.24263\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ANATOMY & MORPHOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Clinical Anatomy","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/ca.24263","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ANATOMY & MORPHOLOGY","Score":null,"Total":0}
Psychometric Properties of the Critical Appraisal Tool for Anatomical Meta-Analysis
The hallmark of evidence-based anatomy (EBA) is the anatomical meta-analysis (AMA). The Critical Appraisal Tool for Anatomical Meta-Analysis (CATAM) was recently published to enable users to appraise AMAs quickly and effectively. The tool is valuable for students and clinicians who need to judge the quality of AMAs, which informs clinical decision making and results in better patient care. Subjective measures of the tool's face and content validity have been established, but establishing its reliability provides a more objective measure of the instrument's dependability. This study investigated the interrater reliability (IRR) of the CATAM between novice and expert raters. Three graduate students and three professors (two anatomists and one pharmacist) read the original CATAM paper, and then had a post hoc meeting to discuss scoring with the tool. Three recent AMAs (published between 2017 and 2022) were randomly chosen from PubMed, and all six raters scored the papers blindly. The intraclass correlation coefficient (ICC) statistic was used to calculate the interrater reliability (IRR) between all scores, and then the ICCs between novice and expert scores were compared. Cronbach's alpha (internal consistency) of the CATAM was also calculated (SPSS 25, Armonk, NY). ICC for AMA-1 was 0.999 (95% CI, 0.997–0.999), p = 0.000, and alpha was 0.999. ICC for AMA-2 was 0.994 (95% CI, 0.988–0.998), p = 0.000, and alpha was 0.994. ICC for AMA-3 was 0.998 (95% CI, 0.995–0.999), p = 0.000, and alpha was 0.998. ANOVA showed no significant differences (p > 0.05) in mean ICCs between raters. The CATAM is a robust tool with excellent IRR (ICC > 0.990) and internal consistency (alpha > 0.990). No significant difference in ICC scores between novices and experts suggests the tool does not require prior expert knowledge to be effective. Now that the reliability of the CATAM is established, it can be more widely adopted by students and physicians worldwide to evaluate the quality of AMAs. The CATAM offers widespread applicability, and can be adopted in medical education, journal clubs, and clinical seminars to critically evaluate AMAs.
期刊介绍：
Clinical Anatomy is the Official Journal of the American Association of Clinical Anatomists and the British Association of Clinical Anatomists. The goal of Clinical Anatomy is to provide a medium for the exchange of current information between anatomists and clinicians. This journal embraces anatomy in all its aspects as applied to medical practice. Furthermore, the journal assists physicians and other health care providers in keeping abreast of new methodologies for patient management and informs educators of new developments in clinical anatomy and teaching techniques. Clinical Anatomy publishes original and review articles of scientific, clinical, and educational interest. Papers covering the application of anatomic principles to the solution of clinical problems and/or the application of clinical observations to expand anatomic knowledge are welcomed.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
