Inter-rater variability in scoring of Addenbrooke's Cognitive Examination-Third Edition (ACE-III) protocols.

IF 1.4 4区心理学 Q4 CLINICAL NEUROLOGY

Applied Neuropsychology-Adult Pub Date : 2024-09-01 Epub Date: 2022-06-15 DOI:10.1080/23279095.2022.2083964

Miranda J Say, Ciarán O'Driscoll

{"title":"Inter-rater variability in scoring of Addenbrooke's Cognitive Examination-Third Edition (ACE-III) protocols.","authors":"Miranda J Say, Ciarán O'Driscoll","doi":"10.1080/23279095.2022.2083964","DOIUrl":null,"url":null,"abstract":"Background: Despite its wide use in dementia diagnosis on the basis of cut-off points, the inter-rater variability of the Addenbrooke's Cognitive Examination-Third Edition (ACE-III) has been poorly studied.Methods: Thirty-one healthcare professionals from an older adults' mental health team scored two ACE-III protocols based on mock patients in a computerised form. Scoring accuracy, as well as total and domain-specific scoring variability, were calculated; factors relevant to participants were obtained, including their level of experience and self-rated confidence administering the ACE-III.Results: There was considerable inter-rater variability (up to 18 points for one of the cases), and one case's mean score was significantly higher (by nearly four points) than the true score. The Fluency, Visuospatial and Attention domains had greater levels of variability than Language and Memory. Higher scoring accuracy was not associated with either greater levels of experience or higher self-confidence in administering the ACE-III.Conclusions: The results suggest that the ACE-III is susceptible to scoring error and considerable inter-rater variability, which highlights the critical importance of initial, and continued, administration and scoring training.","PeriodicalId":51308,"journal":{"name":"Applied Neuropsychology-Adult","volume":null,"pages":null},"PeriodicalIF":1.4000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Neuropsychology-Adult","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1080/23279095.2022.2083964","RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2022/6/15 0:00:00","PubModel":"Epub","JCR":"Q4","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Background: Despite its wide use in dementia diagnosis on the basis of cut-off points, the inter-rater variability of the Addenbrooke's Cognitive Examination-Third Edition (ACE-III) has been poorly studied.

Methods: Thirty-one healthcare professionals from an older adults' mental health team scored two ACE-III protocols based on mock patients in a computerised form. Scoring accuracy, as well as total and domain-specific scoring variability, were calculated; factors relevant to participants were obtained, including their level of experience and self-rated confidence administering the ACE-III.

Results: There was considerable inter-rater variability (up to 18 points for one of the cases), and one case's mean score was significantly higher (by nearly four points) than the true score. The Fluency, Visuospatial and Attention domains had greater levels of variability than Language and Memory. Higher scoring accuracy was not associated with either greater levels of experience or higher self-confidence in administering the ACE-III.

Conclusions: The results suggest that the ACE-III is susceptible to scoring error and considerable inter-rater variability, which highlights the critical importance of initial, and continued, administration and scoring training.

查看原文本刊更多论文

Addenbrooke's Cognitive Examination-Third Edition (ACE-III) 规程评分的评分者之间的差异。

背景：尽管阿登布鲁认知检查第三版（ACE-III）在痴呆症诊断中被广泛使用，但对其评分者之间的差异性却研究甚少：方法：来自一个老年人心理健康小组的 31 名专业医护人员根据计算机化表格中的模拟患者对两个 ACE-III 方案进行了评分。计算了评分的准确性以及总评分和特定领域评分的变异性；获得了与参与者相关的因素，包括他们的经验水平和自评的实施 ACE-III 的信心：结果：评分者之间的差异相当大（其中一个病例的差异高达 18 分），一个病例的平均得分明显高于真实得分（高出近 4 分）。与语言和记忆相比，流畅性、视觉空间和注意力领域的变异性更大。较高的评分准确性既与较丰富的经验无关，也与在实施 ACE-III 时较强的自信心无关：研究结果表明，ACE-III容易出现评分错误，评分者之间的差异也很大，这就凸显了初次和持续的施测和评分培训的重要性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Applied Neuropsychology-Adult CLINICAL NEUROLOGY-PSYCHOLOGY

CiteScore

4.50

自引率

11.80%

发文量

134

期刊介绍： pplied Neuropsychology-Adult publishes clinical neuropsychological articles concerning assessment, brain functioning and neuroimaging, neuropsychological treatment, and rehabilitation in adults. Full-length articles and brief communications are included. Case studies of adult patients carefully assessing the nature, course, or treatment of clinical neuropsychological dysfunctions in the context of scientific literature, are suitable. Review manuscripts addressing critical issues are encouraged. Preference is given to papers of clinical relevance to others in the field. All submitted manuscripts are subject to initial appraisal by the Editor-in-Chief, and, if found suitable for further considerations are peer reviewed by independent, anonymous expert referees. All peer review is single-blind and submission is online via ScholarOne Manuscripts.