Response Rates, Reliability, and Convergent Validity in Experience Sampling Data.

IF 3.5 2区心理学 Q1 PSYCHOLOGY, CLINICAL

Assessment Pub Date : 2024-12-10 DOI:10.1177/10731911241300002

Jeffrey S Simons, Stephen A Maisto, Raluca M Simons, Jessica A Keith, Tibor P Palfai, Kyle J Walters, Surabhi Swaminath, Kawon Kim, Patrick J Ronan

{"title":"Response Rates, Reliability, and Convergent Validity in Experience Sampling Data.","authors":"Jeffrey S Simons, Stephen A Maisto, Raluca M Simons, Jessica A Keith, Tibor P Palfai, Kyle J Walters, Surabhi Swaminath, Kawon Kim, Patrick J Ronan","doi":"10.1177/10731911241300002","DOIUrl":null,"url":null,"abstract":"<p><p>This study examined associations of compliance rate with the reliability and convergent validity of intoxication and negative affect assessments in experience sampling method (ESM) data in three samples (Veterans, Sexual Minority Men, and College Students). Convergent validity was operationalized as within-person associations between daily aggregates of random in situ assessments and retrospective daily assessments or transdermal alcohol assessments. Measures with lower ICC require more assessments for a reliable aggregate (e.g., daily mean). In this regard, the number of completed assessments and intraclass correlation (ICC), rather than compliance with the protocol per se, determines reliability. Although convergent validity was correlated with compliance rate, the relatively weak associations reflect that there are individuals with excellent compliance yet poor convergent validity as well as individuals with poor compliance and excellent convergent validity. The pattern of results does not show a clear threshold for compliance (e.g., 80%) that differentiates good versus poor validity.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241300002"},"PeriodicalIF":3.5000,"publicationDate":"2024-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Assessment","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1177/10731911241300002","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, CLINICAL","Score":null,"Total":0}

引用次数: 0

Abstract

This study examined associations of compliance rate with the reliability and convergent validity of intoxication and negative affect assessments in experience sampling method (ESM) data in three samples (Veterans, Sexual Minority Men, and College Students). Convergent validity was operationalized as within-person associations between daily aggregates of random in situ assessments and retrospective daily assessments or transdermal alcohol assessments. Measures with lower ICC require more assessments for a reliable aggregate (e.g., daily mean). In this regard, the number of completed assessments and intraclass correlation (ICC), rather than compliance with the protocol per se, determines reliability. Although convergent validity was correlated with compliance rate, the relatively weak associations reflect that there are individuals with excellent compliance yet poor convergent validity as well as individuals with poor compliance and excellent convergent validity. The pattern of results does not show a clear threshold for compliance (e.g., 80%) that differentiates good versus poor validity.

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Assessment PSYCHOLOGY, CLINICAL-

CiteScore

8.90

自引率

2.60%

发文量

期刊介绍： Assessment publishes articles in the domain of applied clinical assessment. The emphasis of this journal is on publication of information of relevance to the use of assessment measures, including test development, validation, and interpretation practices. The scope of the journal includes research that can inform assessment practices in mental health, forensic, medical, and other applied settings. Papers that focus on the assessment of cognitive and neuropsychological functioning, personality, and psychopathology are invited. Most papers published in Assessment report the results of original empirical research, however integrative review articles and scholarly case studies will also be considered.