Automated remote speech-based testing of individuals with cognitive decline: Bayesian agreement of transcription accuracy.

IF 4 Q1 CLINICAL NEUROLOGY

Alzheimer''s and Dementia: Diagnosis, Assessment and Disease Monitoring Pub Date : 2024-10-06 eCollection Date: 2024-10-01 DOI:10.1002/dad2.70011

Alexandra König, Stefanie Köhler, Johannes Tröger, Emrah Düzel, Wenzel Glanz, Michaela Butryn, Elisa Mallick, Josef Priller, Slawek Altenstein, Annika Spottke, Okka Kimmich, Björn Falkenburger, Antje Osterrath, Jens Wiltfang, Claudia Bartels, Ingo Kilimann, Christoph Laske, Matthias H Munk, Sandra Roeske, Ingo Frommann, Daniel C Hoffmann, Frank Jessen, Michael Wagner, Nicklas Linz, Stefan Teipel

{"title":"Automated remote speech-based testing of individuals with cognitive decline: Bayesian agreement of transcription accuracy.","authors":"Alexandra König, Stefanie Köhler, Johannes Tröger, Emrah Düzel, Wenzel Glanz, Michaela Butryn, Elisa Mallick, Josef Priller, Slawek Altenstein, Annika Spottke, Okka Kimmich, Björn Falkenburger, Antje Osterrath, Jens Wiltfang, Claudia Bartels, Ingo Kilimann, Christoph Laske, Matthias H Munk, Sandra Roeske, Ingo Frommann, Daniel C Hoffmann, Frank Jessen, Michael Wagner, Nicklas Linz, Stefan Teipel","doi":"10.1002/dad2.70011","DOIUrl":null,"url":null,"abstract":"Introduction: We investigated the agreement between automated and gold-standard manual transcriptions of telephone chatbot-based semantic verbal fluency testing.Methods: We examined 78 cases from the Screening over Speech in Unselected Populations for Clinical Trials in AD (PROSPECT-AD) study, including cognitively normal individuals and individuals with subjective cognitive decline, mild cognitive impairment, and dementia. We used Bayesian Bland-Altman analysis of word count and the qualitative features of semantic cluster size, cluster switches, and word frequencies.Results: We found high levels of agreement for word count, with a 93% probability of a newly observed difference being below the minimally important difference. The qualitative features had fair levels of agreement. Word count reached high levels of discrimination between cognitively impaired and unimpaired individuals, regardless of transcription mode.Discussion: Our results support the use of automated speech recognition particularly for the assessment of quantitative speech features, even when using data from telephone calls with cognitively impaired individuals in their homes.Highlights: High levels of agreement were found between automated and gold-standard manual transcriptions of telephone chatbot-based semantic verbal fluency testing, particularly for word count.The qualitative features had fair levels of agreement.Word count reached high levels of discrimination between cognitively impaired and unimpaired individuals, regardless of transcription mode.Automated speech recognition for the assessment of quantitative and qualitative speech features, even when using data from telephone calls with cognitively impaired individuals in their homes, seems feasible and reliable.","PeriodicalId":53226,"journal":{"name":"Alzheimer''s and Dementia: Diagnosis, Assessment and Disease Monitoring","volume":"16 4","pages":"e70011"},"PeriodicalIF":4.0000,"publicationDate":"2024-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11456616/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Alzheimer''s and Dementia: Diagnosis, Assessment and Disease Monitoring","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/dad2.70011","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/10/1 0:00:00","PubModel":"eCollection","JCR":"Q1","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction: We investigated the agreement between automated and gold-standard manual transcriptions of telephone chatbot-based semantic verbal fluency testing.

Methods: We examined 78 cases from the Screening over Speech in Unselected Populations for Clinical Trials in AD (PROSPECT-AD) study, including cognitively normal individuals and individuals with subjective cognitive decline, mild cognitive impairment, and dementia. We used Bayesian Bland-Altman analysis of word count and the qualitative features of semantic cluster size, cluster switches, and word frequencies.

Results: We found high levels of agreement for word count, with a 93% probability of a newly observed difference being below the minimally important difference. The qualitative features had fair levels of agreement. Word count reached high levels of discrimination between cognitively impaired and unimpaired individuals, regardless of transcription mode.

Discussion: Our results support the use of automated speech recognition particularly for the assessment of quantitative speech features, even when using data from telephone calls with cognitively impaired individuals in their homes.

Highlights: High levels of agreement were found between automated and gold-standard manual transcriptions of telephone chatbot-based semantic verbal fluency testing, particularly for word count.The qualitative features had fair levels of agreement.Word count reached high levels of discrimination between cognitively impaired and unimpaired individuals, regardless of transcription mode.Automated speech recognition for the assessment of quantitative and qualitative speech features, even when using data from telephone calls with cognitively impaired individuals in their homes, seems feasible and reliable.

查看原文本刊更多论文

对认知能力下降者进行自动远程语音测试：转录准确性的贝叶斯协议

简介我们研究了基于电话聊天机器人的语义言语流利性测试的自动转录与黄金标准人工转录之间的一致性：我们研究了78个病例，这些病例来自 "针对AD临床试验的非选择人群言语筛查（PROSPECT-AD）"研究，包括认知能力正常的人和主观认知能力下降、轻度认知障碍和痴呆的人。我们使用贝叶斯布兰德-阿尔特曼分析法对词数以及语义群大小、群切换和词频等定性特征进行了分析：结果：我们发现字数的一致性很高，新观察到的差异低于最小重要差异的概率为 93%。定性特征的一致程度一般。无论采用哪种转录模式，字数在认知障碍者和非认知障碍者之间的区分度都很高：讨论：我们的研究结果支持使用自动语音识别技术，尤其是在评估定量语音特征时，即使使用的是认知障碍者在家中的电话数据：基于电话聊天机器人的语义言语流畅性测试的自动转录和黄金标准人工转录之间的一致性很高，尤其是在字数方面，定性特征的一致性一般。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Alzheimer''s and Dementia: Diagnosis, Assessment and Disease Monitoring Medicine-Psychiatry and Mental Health

CiteScore

7.80

自引率

7.50%

发文量

101

审稿时长

8 weeks

期刊介绍： Alzheimer''s & Dementia: Diagnosis, Assessment & Disease Monitoring (DADM) is an open access, peer-reviewed, journal from the Alzheimer''s Association® that will publish new research that reports the discovery, development and validation of instruments, technologies, algorithms, and innovative processes. Papers will cover a range of topics interested in the early and accurate detection of individuals with memory complaints and/or among asymptomatic individuals at elevated risk for various forms of memory disorders. The expectation for published papers will be to translate fundamental knowledge about the neurobiology of the disease into practical reports that describe both the conceptual and methodological aspects of the submitted scientific inquiry. Published topics will explore the development of biomarkers, surrogate markers, and conceptual/methodological challenges. Publication priority will be given to papers that 1) describe putative surrogate markers that accurately track disease progression, 2) biomarkers that fulfill international regulatory requirements, 3) reports from large, well-characterized population-based cohorts that comprise the heterogeneity and diversity of asymptomatic individuals and 4) algorithmic development that considers multi-marker arrays (e.g., integrated-omics, genetics, biofluids, imaging, etc.) and advanced computational analytics and technologies.