Anthony Robinson, Eathan Breaux, Marissa Huber, Matthew Calamia
{"title":"78 Utility of the D-KEFS Color Word Interference Test as a Measure of Performance Validity in Adults Referred for a Psychoeducational Evaluation","authors":"Anthony Robinson, Eathan Breaux, Marissa Huber, Matthew Calamia","doi":"10.1017/s1355617723009360","DOIUrl":null,"url":null,"abstract":"Objective:Previous investigations have demonstrated the clinical utility of the Delis-Kaplan Executive Function System (D-KEFS) Color Word Interference Test (CWIT) as an embedded validity indicator in mixed clinical samples and traumatic brain injury. The present study sought to cross-validate previously identified indicators and cutoffs in a sample of adults referred for psychoeducational testing.Participants and Methods:Archival data from 267 students and community members self-referred for a psychoeducational evaluation at a university clinic in the South were analyzed. Referrals included assessment for attention-deficit hyperactivity disorder, specific learning disorder, autism spectrum disorder, or other disorders (e.g., anxiety, depression). Individuals were administered subtests of the D-KEFS including the CWIT and several standalone and embedded performance validity indicators as part of the evaluation. Criterion measures included The b Test, Victoria Symptom Validity Test, Medical Symptom Validity Test, Dot Counting Test, and Reliable Digit Span. Individuals who failed 0 criterion measures were included in the credible group <jats:italic>(n</jats:italic> = 164) and individuals failing 2 or more criterion measures were included in the non-credible group <jats:italic>(n</jats:italic> = 31). Because a subset of the sample were seeking external incentives (e.g., accommodations), individuals who failed only 1 of the criterion measures were excluded <jats:italic>(n</jats:italic> = 72). Indicators of interest included all test conditions examined separately, the inverted Stroop index (i.e., better performance on the interference trial than the word reading or color naming trials), inhibition and inhibition/switching composite, and sum of all conditions.Results:Receiver Operating Characteristics (ROC) curves were significant for all four conditions (p < .001) and the inverted stroop index (p = .032). However, only conditions 2, 3 and 4 met minimal acceptable classification accuracy (AUC = .72 - 81). ROC curves with composite indicators were also significant (p < .001), with all three composite indicators meeting minimal acceptable classification accuracy (AUC = .71- .80). At the previously identified cutoff of age corrected scale score of 6 for all four conditions, specificity was high (.88 -.91), with varying sensitivity (.23 - .45). At the previously identified cutoff of .75 for the inverted stroop index, specificity was high (.87) while sensitivity was low (.19). Composite indicators yielded high specificity (.88 - .99) at previously established cutoffs with sensitivity varying from low to moderate (.19 - .48). Increasing the cutoffs (i.e., requiring higher age corrected scale score to pass) for composite indicators increased sensitivity while still maintaining high specificity. For example, increasing the total score cutoff from 18 to 28 resulted in moderate sensitivity (.26 vs .52) with specificity of .91.Conclusions:While a cutoff of 6 resulted in high specificity for most conditions, the sum of all four conditions exhibited the strongest classification accuracy and appears to be the most robust indicator which is consistent with previous research (Eglit et al., 2019). However, a cutoff of 28 as opposed to 18 may be most appropriate for psychoeducational samples. Overall, the results suggest that the D-KEFS CWIT can function as a measure of performance validity in addition to a measure of processing speed/executive functioning.","PeriodicalId":49995,"journal":{"name":"Journal of the International Neuropsychological Society","volume":"5 1","pages":""},"PeriodicalIF":2.6000,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the International Neuropsychological Society","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1017/s1355617723009360","RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Objective:Previous investigations have demonstrated the clinical utility of the Delis-Kaplan Executive Function System (D-KEFS) Color Word Interference Test (CWIT) as an embedded validity indicator in mixed clinical samples and traumatic brain injury. The present study sought to cross-validate previously identified indicators and cutoffs in a sample of adults referred for psychoeducational testing.Participants and Methods:Archival data from 267 students and community members self-referred for a psychoeducational evaluation at a university clinic in the South were analyzed. Referrals included assessment for attention-deficit hyperactivity disorder, specific learning disorder, autism spectrum disorder, or other disorders (e.g., anxiety, depression). Individuals were administered subtests of the D-KEFS including the CWIT and several standalone and embedded performance validity indicators as part of the evaluation. Criterion measures included The b Test, Victoria Symptom Validity Test, Medical Symptom Validity Test, Dot Counting Test, and Reliable Digit Span. Individuals who failed 0 criterion measures were included in the credible group (n = 164) and individuals failing 2 or more criterion measures were included in the non-credible group (n = 31). Because a subset of the sample were seeking external incentives (e.g., accommodations), individuals who failed only 1 of the criterion measures were excluded (n = 72). Indicators of interest included all test conditions examined separately, the inverted Stroop index (i.e., better performance on the interference trial than the word reading or color naming trials), inhibition and inhibition/switching composite, and sum of all conditions.Results:Receiver Operating Characteristics (ROC) curves were significant for all four conditions (p < .001) and the inverted stroop index (p = .032). However, only conditions 2, 3 and 4 met minimal acceptable classification accuracy (AUC = .72 - 81). ROC curves with composite indicators were also significant (p < .001), with all three composite indicators meeting minimal acceptable classification accuracy (AUC = .71- .80). At the previously identified cutoff of age corrected scale score of 6 for all four conditions, specificity was high (.88 -.91), with varying sensitivity (.23 - .45). At the previously identified cutoff of .75 for the inverted stroop index, specificity was high (.87) while sensitivity was low (.19). Composite indicators yielded high specificity (.88 - .99) at previously established cutoffs with sensitivity varying from low to moderate (.19 - .48). Increasing the cutoffs (i.e., requiring higher age corrected scale score to pass) for composite indicators increased sensitivity while still maintaining high specificity. For example, increasing the total score cutoff from 18 to 28 resulted in moderate sensitivity (.26 vs .52) with specificity of .91.Conclusions:While a cutoff of 6 resulted in high specificity for most conditions, the sum of all four conditions exhibited the strongest classification accuracy and appears to be the most robust indicator which is consistent with previous research (Eglit et al., 2019). However, a cutoff of 28 as opposed to 18 may be most appropriate for psychoeducational samples. Overall, the results suggest that the D-KEFS CWIT can function as a measure of performance validity in addition to a measure of processing speed/executive functioning.
期刊介绍:
The Journal of the International Neuropsychological Society is the official journal of the International Neuropsychological Society, an organization of over 4,500 international members from a variety of disciplines. The Journal of the International Neuropsychological Society welcomes original, creative, high quality research papers covering all areas of neuropsychology. The focus of articles may be primarily experimental, applied, or clinical. Contributions will broadly reflect the interest of all areas of neuropsychology, including but not limited to: development of cognitive processes, brain-behavior relationships, adult and pediatric neuropsychology, neurobehavioral syndromes (such as aphasia or apraxia), and the interfaces of neuropsychology with related areas such as behavioral neurology, neuropsychiatry, genetics, and cognitive neuroscience. Papers that utilize behavioral, neuroimaging, and electrophysiological measures are appropriate.
To assure maximum flexibility and to promote diverse mechanisms of scholarly communication, the following formats are available in addition to a Regular Research Article: Brief Communication is a shorter research article; Rapid Communication is intended for "fast breaking" new work that does not yet justify a full length article and is placed on a fast review track; Case Report is a theoretically important and unique case study; Critical Review and Short Review are thoughtful considerations of topics of importance to neuropsychology and include meta-analyses; Dialogue provides a forum for publishing two distinct positions on controversial issues in a point-counterpoint format; Special Issue and Special Section consist of several articles linked thematically; Letter to the Editor responds to recent articles published in the Journal of the International Neuropsychological Society; and Book Review, which is considered but is no longer solicited.