The Reliability of Expert Diagnosis of Childhood Apraxia of Speech.

IF 2.2 2区医学 Q1 AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY

Journal of Speech Language and Hearing Research Pub Date : 2024-09-26 Epub Date: 2023-08-29 DOI:10.1044/2023_JSLHR-22-00677

Elizabeth Murray, Shelley Velleman, Jonathan L Preston, Robert Heard, Akhila Shibu, Patricia McCabe

{"title":"The Reliability of Expert Diagnosis of Childhood Apraxia of Speech.","authors":"Elizabeth Murray, Shelley Velleman, Jonathan L Preston, Robert Heard, Akhila Shibu, Patricia McCabe","doi":"10.1044/2023_JSLHR-22-00677","DOIUrl":null,"url":null,"abstract":"Purpose: The current standard for clinical diagnosis of childhood apraxia of speech (CAS) is expert clinician judgment. The psychometric properties of this standard are not well understood; however, they are important for improving clinical diagnosis. The purpose of this study is to determine the extent to which experts agree on the clinical diagnosis of CAS using two cohorts of children with mixed speech sound disorders (SSDs).Method: Speech samples of children with SSDs were obtained from previous and ongoing research from video recordings of children aged 3-8 years (n = 36) and audio recordings of children aged 8-17 years (n = 56). A total of 23 expert, English-speaking clinicians were recruited internationally. Three of these experts rated each speech sample to provide a description of the observed features and a diagnosis. Intrarater reliability was acceptable at 85% agreement.Results: Interrater reliability on the presence or absence of CAS among experts was poor both as a categorical diagnosis (κ = .187, 95% confidence interval [CI] [0.089, 0.286]) and on a continuous \"likelihood of CAS\" scale (0-100; intraclass correlation = .183, 95% CI [.037, .347]). Reliability was similar across the video-recorded and audio-only samples. There was greater agreement on other diagnoses (such as articulation disorder) than on the diagnosis of CAS, although these too did not meet the predetermined standard. Likelihood of CAS was greater in children who presented with more American Speech-Language-Hearing Association CAS consensus features.Conclusions: Different expert raters had different thresholds for applying the diagnosis of CAS. If expert clinician judgment is to be used for diagnosis of CAS or other SSDs, further standardization and calibration is needed to increase interrater reliability. Diagnosis may require operationalized checklists or reliable measures that operate along a diagnostic continuum.Supplemental material: https://doi.org/10.23641/asha.23949105.","PeriodicalId":51254,"journal":{"name":"Journal of Speech Language and Hearing Research","volume":" ","pages":"3309-3326"},"PeriodicalIF":2.2000,"publicationDate":"2024-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Speech Language and Hearing Research","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1044/2023_JSLHR-22-00677","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/8/29 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: The current standard for clinical diagnosis of childhood apraxia of speech (CAS) is expert clinician judgment. The psychometric properties of this standard are not well understood; however, they are important for improving clinical diagnosis. The purpose of this study is to determine the extent to which experts agree on the clinical diagnosis of CAS using two cohorts of children with mixed speech sound disorders (SSDs).

Method: Speech samples of children with SSDs were obtained from previous and ongoing research from video recordings of children aged 3-8 years (n = 36) and audio recordings of children aged 8-17 years (n = 56). A total of 23 expert, English-speaking clinicians were recruited internationally. Three of these experts rated each speech sample to provide a description of the observed features and a diagnosis. Intrarater reliability was acceptable at 85% agreement.

Results: Interrater reliability on the presence or absence of CAS among experts was poor both as a categorical diagnosis (κ = .187, 95% confidence interval [CI] [0.089, 0.286]) and on a continuous "likelihood of CAS" scale (0-100; intraclass correlation = .183, 95% CI [.037, .347]). Reliability was similar across the video-recorded and audio-only samples. There was greater agreement on other diagnoses (such as articulation disorder) than on the diagnosis of CAS, although these too did not meet the predetermined standard. Likelihood of CAS was greater in children who presented with more American Speech-Language-Hearing Association CAS consensus features.

Conclusions: Different expert raters had different thresholds for applying the diagnosis of CAS. If expert clinician judgment is to be used for diagnosis of CAS or other SSDs, further standardization and calibration is needed to increase interrater reliability. Diagnosis may require operationalized checklists or reliable measures that operate along a diagnostic continuum.

Supplemental material: https://doi.org/10.23641/asha.23949105.

查看原文本刊更多论文

专家诊断儿童语言障碍的可靠性。

目的：目前，儿童语言障碍（CAS）的临床诊断标准是临床专家的判断。这一标准的心理测量特性尚不十分明确；但是，这些特性对于改进临床诊断非常重要。本研究的目的是利用两组混合性言语发声障碍（SSD）儿童，确定专家对 CAS 临床诊断的一致程度：方法：从以往和正在进行的研究中获取混合言语失调儿童的语音样本，包括 3-8 岁儿童的视频记录（36 人）和 8-17 岁儿童的音频记录（56 人）。我们在国际上共招募了 23 位讲英语的临床专家。其中三位专家对每个语音样本进行评分，以提供观察到的特征描述和诊断结果。研究结果表明，双方的一致性达到了 85% 的可接受水平：结果：无论是分类诊断（κ = .187，95% 置信区间 [CI][0.089，0.286]）还是连续的 "CAS 可能性 "量表（0-100；类内相关性 = .183，95% 置信区间 [.037，.347]），专家之间关于是否存在 CAS 的相互间可靠性都很差。视频录像样本和纯音频样本的可靠性相似。与 CAS 诊断相比，其他诊断（如发音障碍）的一致性更高，尽管这些诊断也未达到预定标准。具有更多美国言语-语言-听力协会 CAS 一致特征的儿童患 CAS 的可能性更大：结论：不同的专家评分者对 CAS 诊断的阈值不同。如果要使用临床专家的判断来诊断 CAS 或其他 SSD，则需要进一步标准化和校准，以提高评定者之间的可靠性。诊断可能需要操作化的核对表或可靠的测量方法，这些方法可沿着诊断连续体进行操作。补充材料：https://doi.org/10.23641/asha.23949105。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Speech Language and Hearing Research AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY-REHABILITATION

CiteScore

4.10

自引率

19.20%

发文量

538

审稿时长

4-8 weeks

期刊介绍： Mission: JSLHR publishes peer-reviewed research and other scholarly articles on the normal and disordered processes in speech, language, hearing, and related areas such as cognition, oral-motor function, and swallowing. The journal is an international outlet for both basic research on communication processes and clinical research pertaining to screening, diagnosis, and management of communication disorders as well as the etiologies and characteristics of these disorders. JSLHR seeks to advance evidence-based practice by disseminating the results of new studies as well as providing a forum for critical reviews and meta-analyses of previously published work. Scope: The broad field of communication sciences and disorders, including speech production and perception; anatomy and physiology of speech and voice; genetics, biomechanics, and other basic sciences pertaining to human communication; mastication and swallowing; speech disorders; voice disorders; development of speech, language, or hearing in children; normal language processes; language disorders; disorders of hearing and balance; psychoacoustics; and anatomy and physiology of hearing.