The development of a novel, standardized, norm-referenced Arabic Discourse Assessment Tool (ADAT), including an examination of psychometric properties of discourse measures in aphasia

IF 1.5 3区医学 Q2 AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY

International Journal of Language & Communication Disorders Pub Date : 2024-06-18 DOI:10.1111/1460-6984.13083

Reem S. W. Alyahya

{"title":"The development of a novel, standardized, norm-referenced Arabic Discourse Assessment Tool (ADAT), including an examination of psychometric properties of discourse measures in aphasia","authors":"Reem S. W. Alyahya","doi":"10.1111/1460-6984.13083","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>People with aphasia (PWA) typically exhibit deficits in spoken discourse. Discourse analysis is the gold standard approach to assess language deficits beyond sentence level. However, the available discourse assessment tools are biased towards English and European languages and Western culture. Additionally, there is a lack of consensus on which discourse measures to use and limited evidence of the psychometric properties of published discourse measurements.</p>\n </section>\n \n <section>\n \n <h3> Aims</h3>\n \n <p>(1) To develop a standardized, norm-referenced, culturally and linguistically appropriate Arabic Discourse Assessment Tool (ADAT); and (2) to examine the psychometric properties of content and construct validity and interrater reliability of different discourse measures elicited using three discourse genres (descriptive, narrative and procedural) in neurotypical control adults and matched PWA.</p>\n </section>\n \n <section>\n \n <h3> Methods & Procedures</h3>\n \n <p>Discourse samples were collected using three novel discourse stimuli that are sensitive to the Arabic language and culture from 70 neurotypical control adults and a matched group of 50 PWA. Transcription agreement was assessed. A standard approach was used to evaluate construct validity and interrater reliability for 16 discourse measures that assess fluency, language productivity, information content, lexical–semantics, lexical diversity, grammatical category, grammatical structure and syntactic complexity. Strong measures were identified based on their psychometric properties, and normative data were established on these measures. Discourse performance of PWA was then examined using the newly developed tool (ADAT).</p>\n </section>\n \n <section>\n \n <h3> Outcomes & Results</h3>\n \n <p>Transcription agreement was extremely high for all discourse stimuli in both groups. Eight discourse measures were proven to have consistently very high construct validity and consistently very good to excellent reliability across the three stimuli in both neurotypical control and aphasia groups: lexical information units, content information units, words per minute, discourse duration, number of different words, number of complete sentences and proportion of open and closed class words. Norms were established on these measures, and cut-off scores of impairments were determined. Other measures showed low construct validity and variable or poor reliability across the two groups.</p>\n </section>\n \n <section>\n \n <h3> Conclusions & Implications</h3>\n \n <p>The newly developed, standardized, and norm-referenced tool (ADAT) consist of three discourse stimuli and eight high-quality discourse measures that assess multiple aspects of spoken discourse and were able to differentiate PWA from neurotypical adults consistently. ADAT also includes normative data and cut-off impairment scores. The tool has great potential to enhance clinical practice and research with Arabic speakers. Evidence was provided that not all discourse measures are of high quality, as some are vulnerable to differences between raters, discourse stimuli and groups. Clinicians and researchers can use ADAT for accurate aphasia assessments, better management plans and to monitor therapy effectiveness. ADAT can be further validated in other clinical populations with language impairments.</p>\n </section>\n \n <section>\n \n <h3> WHAT THIS PAPER ADDS</h3>\n \n <section>\n \n <h3> What is already known on the subject</h3>\n \n <div>\n <ul>\n \n <li>Discourse analysis is the gold standard approach to assess language deficits beyond sentence level. However, existing discourse assessment tools are biased towards English and European languages and Western culture. Additionally, there is a lack of consensus on which discourse measures to use in aphasia, and limited evidence of the psychometric properties of published discourse measurements.</li>\n </ul>\n </div>\n </section>\n \n <section>\n \n <h3> What this paper adds to existing knowledge</h3>\n \n <div>\n <ul>\n \n <li>A novel, standardized, norm-referenced Arabic Discourse Assessment Tool (ADAT) was developed and validated in this study. ADAT was further validated among PWA. The study provides evidence that not all discourse measures are of high quality and thus should not be used with confidence. Specific measures are vulnerable to the type of stimuli, the rater and/or the tested group. On the other hand, eight discourse measures were identified to be reliable between different raters and across different stimuli for the two groups, and they were able to differentiate the discourse performance of PWA from neurotypical control adults. Normative data derived from neurotypical control adults were established on these strong measures, and the performance of PWA was classified as impaired based on these norms.</li>\n </ul>\n </div>\n </section>\n \n <section>\n \n <h3> What are the potential or actual clinical implications of this work?</h3>\n \n <div>\n <ul>\n \n <li>The present study provides a novel, standardized, norm-referenced, validated discourse assessment tool that is culturally and linguistically appropriate for use by Arabic speakers (ADAT). ADAT holds immense potential to enhance clinical practice and research with Arabic speakers. The study also identified strong discourse measures that can be used to assess language productivity, information content, lexical–semantics, lexical diversity, grammatical category, and syntactic complexity for accurate and comprehensive assessments. This will lead to better rehabilitation management by guiding the development of tailored client-centred interventions. ADAT can be utilized in clinical and research settings in PWA and has the potential to be further validated with other clinical populations.</li>\n </ul>\n </div>\n </section>\n </section>\n </div>","PeriodicalId":49182,"journal":{"name":"International Journal of Language & Communication Disorders","volume":"59 5","pages":"2103-2117"},"PeriodicalIF":1.5000,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/1460-6984.13083","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Language & Communication Disorders","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/1460-6984.13083","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Background

People with aphasia (PWA) typically exhibit deficits in spoken discourse. Discourse analysis is the gold standard approach to assess language deficits beyond sentence level. However, the available discourse assessment tools are biased towards English and European languages and Western culture. Additionally, there is a lack of consensus on which discourse measures to use and limited evidence of the psychometric properties of published discourse measurements.

Aims

(1) To develop a standardized, norm-referenced, culturally and linguistically appropriate Arabic Discourse Assessment Tool (ADAT); and (2) to examine the psychometric properties of content and construct validity and interrater reliability of different discourse measures elicited using three discourse genres (descriptive, narrative and procedural) in neurotypical control adults and matched PWA.

Methods & Procedures

Discourse samples were collected using three novel discourse stimuli that are sensitive to the Arabic language and culture from 70 neurotypical control adults and a matched group of 50 PWA. Transcription agreement was assessed. A standard approach was used to evaluate construct validity and interrater reliability for 16 discourse measures that assess fluency, language productivity, information content, lexical–semantics, lexical diversity, grammatical category, grammatical structure and syntactic complexity. Strong measures were identified based on their psychometric properties, and normative data were established on these measures. Discourse performance of PWA was then examined using the newly developed tool (ADAT).

Outcomes & Results

Transcription agreement was extremely high for all discourse stimuli in both groups. Eight discourse measures were proven to have consistently very high construct validity and consistently very good to excellent reliability across the three stimuli in both neurotypical control and aphasia groups: lexical information units, content information units, words per minute, discourse duration, number of different words, number of complete sentences and proportion of open and closed class words. Norms were established on these measures, and cut-off scores of impairments were determined. Other measures showed low construct validity and variable or poor reliability across the two groups.

Conclusions & Implications

The newly developed, standardized, and norm-referenced tool (ADAT) consist of three discourse stimuli and eight high-quality discourse measures that assess multiple aspects of spoken discourse and were able to differentiate PWA from neurotypical adults consistently. ADAT also includes normative data and cut-off impairment scores. The tool has great potential to enhance clinical practice and research with Arabic speakers. Evidence was provided that not all discourse measures are of high quality, as some are vulnerable to differences between raters, discourse stimuli and groups. Clinicians and researchers can use ADAT for accurate aphasia assessments, better management plans and to monitor therapy effectiveness. ADAT can be further validated in other clinical populations with language impairments.

WHAT THIS PAPER ADDS

What is already known on the subject

Discourse analysis is the gold standard approach to assess language deficits beyond sentence level. However, existing discourse assessment tools are biased towards English and European languages and Western culture. Additionally, there is a lack of consensus on which discourse measures to use in aphasia, and limited evidence of the psychometric properties of published discourse measurements.

What this paper adds to existing knowledge

A novel, standardized, norm-referenced Arabic Discourse Assessment Tool (ADAT) was developed and validated in this study. ADAT was further validated among PWA. The study provides evidence that not all discourse measures are of high quality and thus should not be used with confidence. Specific measures are vulnerable to the type of stimuli, the rater and/or the tested group. On the other hand, eight discourse measures were identified to be reliable between different raters and across different stimuli for the two groups, and they were able to differentiate the discourse performance of PWA from neurotypical control adults. Normative data derived from neurotypical control adults were established on these strong measures, and the performance of PWA was classified as impaired based on these norms.

What are the potential or actual clinical implications of this work?

The present study provides a novel, standardized, norm-referenced, validated discourse assessment tool that is culturally and linguistically appropriate for use by Arabic speakers (ADAT). ADAT holds immense potential to enhance clinical practice and research with Arabic speakers. The study also identified strong discourse measures that can be used to assess language productivity, information content, lexical–semantics, lexical diversity, grammatical category, and syntactic complexity for accurate and comprehensive assessments. This will lead to better rehabilitation management by guiding the development of tailored client-centred interventions. ADAT can be utilized in clinical and research settings in PWA and has the potential to be further validated with other clinical populations.

Abstract Image

查看原文本刊更多论文

开发新型、标准化、常模参照的阿拉伯语话语评估工具 (ADAT)，包括研究失语症患者话语测量的心理测量特性。

背景：失语症患者（PWA）通常在口语表达方面存在缺陷。话语分析是评估句子以上语言障碍的金标准方法。然而，现有的话语评估工具偏重于英语、欧洲语言和西方文化。目的：(1) 开发一个标准化、标准参照、文化和语言上合适的阿拉伯语话语评估工具 (ADAT)；(2) 在神经畸形对照组成人和配对的 PWA 中，使用三种话语体裁（描述性、叙述性和程序性）检查不同话语测量的内容和结构有效性以及施测者间可靠性的心理测量特性：使用三种对阿拉伯语和阿拉伯文化敏感的新颖话语刺激，从 70 名神经畸形对照组成人和 50 名匹配的 PWA 群体中收集话语样本。对转录一致性进行了评估。采用标准方法评估了 16 种话语测量方法的结构效度和互测信度，这些方法评估了流畅性、语言效率、信息内容、词汇-语义、词汇多样性、语法类别、语法结构和句法复杂性。根据心理测量特性确定了强测量指标，并建立了这些测量指标的常模数据。然后使用新开发的工具（ADAT）对 PWA 的话语表现进行了检验：在两组的所有话语刺激中，转录一致性都非常高。在神经畸形控制组和失语症组的三种刺激中，有八项话语测量指标被证明具有非常高的建构效度和非常好到极好的可靠性：词汇信息单位、内容信息单位、每分钟词汇量、话语持续时间、不同词汇的数量、完整句子的数量以及开放类和封闭类词汇的比例。对这些测量方法进行了规范，并确定了障碍的临界分数。其他测量指标在两组中的建构效度较低，信度不一或较差：新开发的标准化常模参照工具（ADAT）由三个话语刺激和八个高质量的话语测量指标组成，可评估口语话语的多个方面，能够一致地区分 PWA 和神经畸形成人。ADAT 还包括常模数据和损伤临界分数。该工具在加强阿拉伯语者的临床实践和研究方面具有巨大潜力。有证据表明，并非所有的话语测量都是高质量的，因为有些测量容易受到评分者、话语刺激和群体之间差异的影响。临床医生和研究人员可以使用 ADAT 进行准确的失语症评估，制定更好的管理计划并监测治疗效果。ADAT 还可以在其他有语言障碍的临床人群中进一步验证：关于该主题的已知信息话语分析是评估句子水平以上语言障碍的金标准方法。然而，现有的话语评估工具偏重于英语、欧洲语言和西方文化。此外，对于在失语症中使用哪种话语测量方法还缺乏共识，已发表的话语测量方法的心理测量特性证据也很有限。本文对现有知识的补充本研究开发并验证了一种新颖、标准化、常模参照的阿拉伯语话语评估工具（ADAT）。ADAT 在 PWA 中得到了进一步验证。该研究提供的证据表明，并非所有的话语测量方法都是高质量的，因此不能放心使用。特定的测量方法容易受到刺激类型、评分者和/或受测群体的影响。另一方面，研究发现，有八种话语测量方法在不同评分者之间以及两组受测者在不同刺激下的表现都是可靠的，而且这些方法还能区分 PWA 和神经正常对照组成人的话语表现。在这些强大的测量指标上建立了来自神经正常对照组成人的标准数据，并根据这些标准将 PWA 的表现归类为受损。这项工作有哪些潜在或实际的临床意义？本研究提供了一种新颖、标准化、常模参照、经过验证的话语评估工具（ADAT），该工具在文化和语言上适合阿拉伯语使用者使用。ADAT 在加强阿拉伯语者的临床实践和研究方面具有巨大的潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Journal of Language & Communication Disorders AUDIOLOGY & SPEECH-LANGUAGE PATHOLOGY-REHABILITATION

CiteScore

3.30

自引率

12.50%

发文量

116

审稿时长

6-12 weeks

期刊介绍： The International Journal of Language & Communication Disorders (IJLCD) is the official journal of the Royal College of Speech & Language Therapists. The Journal welcomes submissions on all aspects of speech, language, communication disorders and speech and language therapy. It provides a forum for the exchange of information and discussion of issues of clinical or theoretical relevance in the above areas.