Advances in Methods and Practices in Psychological Science最新文献

{"title":"How Many Participants Do I Need to Test an Interaction? Conducting an Appropriate Power Analysis and Achieving Sufficient Power to Detect an Interaction","authors":"Nicolas Sommet, David L. Weissman, Nicolas Cheutin, Andrew J. Elliot","doi":"10.1177/25152459231178728","DOIUrl":"https://doi.org/10.1177/25152459231178728","url":null,"abstract":"Power analysis for first-order interactions poses two challenges: (a) Conducting an appropriate power analysis is difficult because the typical expected effect size of an interaction depends on its shape, and (b) achieving sufficient power is difficult because interactions are often modest in size. This article consists of three parts. In the first part, we address the first challenge. We first use a fictional study to explain the difference between power analyses for interactions and main effects. Then, we introduce an intuitive taxonomy of 12 types of interactions based on the shape of the interaction (reversed, fully attenuated, partially attenuated) and the size of the simple slopes (median, smaller, larger), and we offer mathematically derived sample-size recommendations to detect each interaction with a power of .80/.90/.95 (for two-tailed tests in between-participants designs). In the second part, we address the second challenge. We first describe a preregistered metastudy (159 studies from recent articles in influential psychology journals) showing that the median power to detect interactions of a typical size is .18. Then, we use simulations (≈900,000,000 data sets) to generate power curves for the 12 types of interactions and test three approaches to increase power without increasing sample size: (a) preregistering one-tailed tests (+21% gain), (b) using a mixed design (+75% gain), and (c) preregistering contrast analysis for a fully attenuated interaction (+62% gain). In the third part, we introduce INT×Power ( www.intxpower.com ), a web application that enables users to draw their interaction and determine the sample size needed to reach the power of their choice with the option of using/combining these approaches.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":"234 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136260479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Tutorial on Causal Inference in Longitudinal Data With Time-Varying Confounding Using G-Estimation","authors":"W. W. Loh, Dongning Ren","doi":"10.1177/25152459231174029","DOIUrl":"https://doi.org/10.1177/25152459231174029","url":null,"abstract":"In psychological research, longitudinal study designs are often used to examine the effects of a naturally observed predictor (i.e., treatment) on an outcome over time. But causal inference of longitudinal data in the presence of time-varying confounding is notoriously challenging. In this tutorial, we introduce g-estimation, a well-established estimation strategy from the causal inference literature. G-estimation is a powerful analytic tool designed to handle time-varying confounding variables affected by treatment. We offer step-by-step guidance on implementing the g-estimation method using standard parametric regression functions familiar to psychological researchers and commonly available in statistical software. To facilitate hands-on usage, we provide software code at each step using the open-source statistical software R. All the R code presented in this tutorial are publicly available online.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46760911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.","authors":"Yotam Erel,&nbsp;Katherine Adams Shannon,&nbsp;Junyi Chu,&nbsp;Kim Scott,&nbsp;Melissa Kline Struhl,&nbsp;Peng Cao,&nbsp;Xincheng Tan,&nbsp;Peter Hart,&nbsp;Gal Raz,&nbsp;Sabrina Piccolo,&nbsp;Catherine Mei,&nbsp;Christine Potter,&nbsp;Sagi Jaffe-Dax,&nbsp;Casey Lew-Williams,&nbsp;Joshua Tenenbaum,&nbsp;Katherine Fairchild,&nbsp;Amit Bermano,&nbsp;Shari Liu","doi":"10.1177/25152459221147250","DOIUrl":"10.1177/25152459221147250","url":null,"abstract":"<p><p>Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing \"LEFT\" versus \"RIGHT\" and \"ON\" versus \"OFF\" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.</p>","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":"6 2","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/55/6e/nihms-1916587.PMC10471135.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10152159","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Beyond the Mean: Can We Improve the Predictive Power of Psychometric Scales?","authors":"Yngwie Asbjørn Nielsen, Isabel Thielmann, Stefan Pfattheicher","doi":"10.1177/25152459231177713","DOIUrl":"https://doi.org/10.1177/25152459231177713","url":null,"abstract":"Two participants completing a psychometric scale may leave wildly different responses yet attain the same mean score. Moreover, the mean score often does not represent the bulk of participants’ responses, which may be skewed, kurtotic, or bimodal. Even so, researchers in psychological science often aggregate item scores using an unweighted mean or a sum score, thereby neglecting a substantial amount of information. In the present contribution, we explore whether other summary statistics of a scale (e.g., the standard deviation, the median, or the kurtosis) can capture and leverage some of this neglected information to improve prediction of a broad range of outcome measures: life satisfaction, mental health, self-esteem, counterproductive work behavior, and social value orientation. Overall, across 32 psychometric scales and three data sets (total N = 8,376), we show that the mean is the strongest predictor of all five outcomes considered, with little to no additional variance explained by other summary statistics. These results provide justification for the current practice of relying on the mean score but hopefully inspire future research to explore the predictive power of other summary statistics for relevant outcomes. For this purpose, we provide a tutorial and example code for R.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46022858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Primer on Structural Equation Model Diagrams and Directed Acyclic Graphs: When and How to Use Each in Psychological and Epidemiological Research","authors":"Zachary J. Kunicki, Meghan L. Smith, E. Murray","doi":"10.1177/25152459231156085","DOIUrl":"https://doi.org/10.1177/25152459231156085","url":null,"abstract":"Many psychological researchers use some form of a visual diagram in their research processes. Model diagrams used with structural equation models (SEMs) and causal directed acyclic graphs (DAGs) can guide causal-inference research. SEM diagrams and DAGs share visual similarities, often leading researchers familiar with one to wonder how the other differs. This article is intended to serve as a guide for researchers in the psychological sciences and psychiatric epidemiology on the distinctions between these methods. We offer high-level overviews of SEMs and causal DAGs using a guiding example. We then compare and contrast the two methodologies and describe when each would be used. In brief, SEM diagrams are both a conceptual and statistical tool in which a model is drawn and then tested, whereas causal DAGs are exclusively conceptual tools used to help guide researchers in developing an analytic strategy and interpreting results. Causal DAGs are explicitly tools for causal inference, whereas the results of a SEM are only sometimes interpreted causally. A DAG may be thought of as a “qualitative schematic” for some SEMs, whereas SEMs may be thought of as an “algebraic system” for a causal DAG. As psychology begins to adopt more causal-modeling concepts and psychiatric epidemiology begins to adopt more latent-variable concepts, the ability of researchers to understand and possibly combine both of these tools is valuable. Using an applied example, we provide sample analyses, code, and write-ups for both SEM and causal DAG approaches.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42921040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Corrigendum: Journal N-Pact Factors From 2011 to 2019: Evaluating the Quality of Social/Personality Journals With Respect to Sample Size and Statistical Power","authors":"","doi":"10.1177/25152459231175075","DOIUrl":"https://doi.org/10.1177/25152459231175075","url":null,"abstract":"JRP 330 238 (17) 438 (14) 325 (26) 302 (15) 369 (14) 280 (11) 330 (18) 500 (20) 438 (22) EJP 261 111 (13) 194 (16) 392 (14) 217 (7) 200 (12) 261 (15) 422 (15) 576 (11) 496 (10) JP 251 239 (14) 198 (22) 354 (16) 359 (10) 406 (19) 239 (23) 251 (11) 240 (44) 286 (30) PS:S 200 72 (26) 82 (24) 174 (20) 104 (21) 200 (36) 231 (35) 248 (23) 386 (52) 202 (32) SPPS 186 128 (26) 134 (24) 210 (25) 172 (26) 186 (38) 178 (29) 300 (39) 364 (26) 383 (33) JPSP 179 98 (70) 108 (75) 102 (73) 116 (71) 220 (64) 179 (81) 225 (77) 225 (75) 320 (45) PSPB 139 105 (51) 78 (47) 117 (50) 151 (62) 130 (84) 139 (57) 186 (74) 220 (51) 208 (51) EJSP 131 96 (31) 93 (29) 78 (24) 139 (29) 131 (34) 126 (30) 219 (27) 153 (42) 169 (60) JESP 113 94 (66) 69 (53) 120 (68) 92 (56) 113 (89) 102 (46) 204 (60) 206 (81) 275 (70)","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42797342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Selecting the Number and Labels of Topics in Topic Modeling: A Tutorial","authors":"S. Weston, Ian Shryock, Ryan Light, Phillip A. Fisher","doi":"10.1177/25152459231160105","DOIUrl":"https://doi.org/10.1177/25152459231160105","url":null,"abstract":"Topic modeling is a type of text analysis that identifies clusters of co-occurring words, or latent topics. A challenging step of topic modeling is determining the number of topics to extract. This tutorial describes tools researchers can use to identify the number and labels of topics in topic modeling. First, we outline the procedure for narrowing down a large range of models to a select number of candidate models. This procedure involves comparing the large set on fit metrics, including exclusivity, residuals, variational lower bound, and semantic coherence. Next, we describe the comparison of a small number of models using project goals as a guide and information about topic representative and solution congruence. Finally, we describe tools for labeling topics, including frequent and exclusive words, key examples, and correlations among topics.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42107656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using Market-Research Panels for Behavioral Science: An Overview and Tutorial","authors":"Aaron J. Moss, David J. Hauser, Cheskie Rosenzweig, Shalom N Jaffe, Jonathan Robinson, L. Litman","doi":"10.1177/25152459221140388","DOIUrl":"https://doi.org/10.1177/25152459221140388","url":null,"abstract":"Behavioral scientists looking to run online studies are confronted with a bevy of options. Where to recruit participants? Which tools to use for survey creation and study management? How to maintain data quality? In this tutorial, we highlight the unique capabilities of market-research panels and demonstrate how researchers can effectively sample from such panels. Unlike the microtask platforms most academics are familiar with (e.g., MTurk and Prolific), market-research panels have access to more than 100 million potential participants worldwide, provide more representative samples, and excel at demographic targeting. However, efficiently gathering data from online panels requires integration between the panel and a researcher’s survey in ways that are uncommon on microtask sites. For example, panels allow researchers to target participants according to preprofiled demographics (“Level 1” targeting, e.g., parents) and demographics that are not preprofiled but are screened for within the survey (“Level 2” targeting, e.g., parents of autistic children). In this article, we demonstrate how to sample hard-to-reach groups using market-research panels. We also describe several best practices for conducting research using online panels, including setting in-survey quotas to control sample composition and managing data quality. Our aim is to provide researchers with enough information to determine whether market-research panels are right for their research and to outline the necessary considerations for using such panels.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":"6 1","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41396512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"PsyCalibrator: An Open-Source Package for Display Gamma Calibration and Luminance and Color Measurement","authors":"Zhicheng Lin, Qimin Ma, Yang Zhang","doi":"10.1177/25152459221151151","DOIUrl":"https://doi.org/10.1177/25152459221151151","url":null,"abstract":"Studies in vision, psychology, and neuroscience often present visual stimuli on digital screens. Crucially, the appearance of visual stimuli depends on properties such as luminance and color, making it critical to measure them. Yet conventional luminance-measuring equipment is not only expensive but also onerous to operate (particularly for novices). Building on previous work, here we present an open-source integrated software package—PsyCalibrator (https://github.com/yangzhangpsy/PsyCalibrator)—that takes advantage of consumer hardware (SpyderX, Spyder5) and makes luminance/color measurement and gamma calibration accessible and flexible. Gamma calibration based on visual methods (without photometers) is also implemented. PsyCalibrator requires MATLAB (or its free alternative, GNU Octave) and works in Windows, macOS, and Linux. We first validated measurements from SpyderX and Spyder5 by comparing them with professional, high-cost photometers (ColorCAL MKII Colorimeter and Photo Research PR-670 SpectraScan). Validation results show (a) excellent accuracy in linear correction and luminance/color measurement and (b) for practical purposes, low measurement variances. We offer a detailed tutorial on using PsyCalibrator to measure luminance/color and calibrate displays. Finally, we recommend reporting templates to describe simple (e.g., computer-generated shapes) and complex (e.g., naturalistic images and videos) visual stimuli.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43447182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bayesian Repeated-Measures Analysis of Variance: An Updated Methodology Implemented in JASP","authors":"D. van den Bergh, E. Wagenmakers, F. Aust","doi":"10.1177/25152459231168024","DOIUrl":"https://doi.org/10.1177/25152459231168024","url":null,"abstract":"Analysis of variance (ANOVA) is widely used to assess the influence of one or more experimental (or quasi-experimental) manipulations on a continuous outcome. Traditionally, ANOVA is carried out in a frequentist manner using p values, but a Bayesian alternative has been proposed. Assuming that the proposed Bayesian ANOVA is closely modeled after its frequentist counterpart, one may be surprised to find that the two can yield very different conclusions when the design involves multiple repeated-measures factors. We illustrate such a discrepancy with a real data set from a two-factorial within-subject experiment. For this data set, the results of a frequentist and Bayesian ANOVA are in a disagreement about which main effect accounts for the variance in the data. The reason for this disagreement is that frequentist and the proposed Bayesian ANOVA use different model specifications. As currently implemented, the proposed Bayesian ANOVA assumes that there are no individual differences in the magnitude of effects. We suspect that this assumption is neither obvious to nor desired by most analysts because it is untenable in most applications. We argue here that the Bayesian ANOVA should be revised to allow for individual differences. As a default, we suggest the standard frequentist model specification but discuss a recently proposed alternative and provide guidance on how to choose the appropriate model specification. We end by discussing the implications of the revised model specification for previously published results of Bayesian ANOVAs.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41864814","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}