John R. Busenbark, Scott D. Graffin, R. Campbell, Eric Y. Lee
{"title":"A Marginal Effects Approach to Interpreting Main Effects and Moderation","authors":"John R. Busenbark, Scott D. Graffin, R. Campbell, Eric Y. Lee","doi":"10.1177/1094428120976838","DOIUrl":"https://doi.org/10.1177/1094428120976838","url":null,"abstract":"This Short Methodological Report builds on research about moderation practices by focusing on a marginal effects approach to interpreting how a main effect is informed by the presence of a moderating variable. Following a content analysis of published studies and a survey of management researchers, our findings suggest there is a great deal of confusion about the ways in which to interpret how a main effect may fluctuate owing to a moderating variable. We therefore provide explicit instructions on how to implement and interpret a marginal effects approach that depicts the nature of a main effect in the presence of a moderator. We use different scenarios and examples to illustrate how researchers can employ the marginal effects technique, which provides an indication of the relationship between the independent and dependent variables over different values of the moderator. We argue and demonstrate that the marginal effects approach helps resolve conflicting findings that may arise from using other prevailing techniques to interpret both main effects and moderation.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"25 1","pages":"147 - 169"},"PeriodicalIF":9.5,"publicationDate":"2021-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428120976838","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44971423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Cortina, Hannah M. Markell-Goldstein, Jennifer P. Green, Yingyi Chang
{"title":"How Are We Testing Interactions in Latent Variable Models? Surging Forward or Fighting Shy?:","authors":"J. Cortina, Hannah M. Markell-Goldstein, Jennifer P. Green, Yingyi Chang","doi":"10.25384/SAGE.C.4665197.V1","DOIUrl":"https://doi.org/10.25384/SAGE.C.4665197.V1","url":null,"abstract":"Latent variable models and interaction effects have both been common in the organizational sciences for some time. Methods for incorporating interactions into latent variable models have existed si...","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"26-54"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"69207895","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Building Transparency and Trustworthiness in Inductive Research Through Computer-Aided Qualitative Data Analysis Software","authors":"P. O’Kane, Anne D. Smith, Michael P. Lerman","doi":"10.1177/1094428119865016","DOIUrl":"https://doi.org/10.1177/1094428119865016","url":null,"abstract":"Many scholars have called for qualitative research to demonstrate transparency and trustworthiness in the data analysis process. Yet these processes, particularly within inductive research, often remain shrouded in mystery. We suggest that computer-aided/assisted qualitative data analysis software (CAQDAS) can support qualitative researchers in their efforts to present their analysis and findings in a transparent way, thus enhancing trustworthiness. To this end, we propose, describe, and illustrate working examples of six CAQDAS building blocks, three combined CAQDAS techniques, and two coder consistency checks. We argue that these techniques give researchers the language to write about their methods and findings in a transparent manner and that their appropriate use enhances a research project’s trustworthiness. Specific CAQDAS techniques are rarely discussed across an array of inductive research processes. Thus, we see this article as the beginning of a conversation about the utility of CAQDAS to support inductive qualitative research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"104 - 139"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428119865016","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45888085","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Croon’s Bias-Corrected Factor Score Path Analysis for Small- to Moderate-Sample Multilevel Structural Equation Models","authors":"Ben Kelcey, Kyle Cox, N. Dong","doi":"10.1177/1094428119879758","DOIUrl":"https://doi.org/10.1177/1094428119879758","url":null,"abstract":"Maximum likelihood estimation of multilevel structural equation model (MLSEM) parameters is a preferred approach to probe theories involving latent variables in multilevel settings. Although maximum likelihood has many desirable properties, a major limitation is that it often fails to converge and can incur significant bias when implemented in studies with a small to moderate multilevel sample (e.g., fewer than 100 organizations with 10 or less individuals/organization). To address similar limitations in single-level SEM, literature has developed Croon’s bias-corrected factor score path analysis estimator that converges more regularly than maximum likelihood and delivers less biased parameter estimates with small to moderate sample sizes. We derive extensions to this framework for MLSEMs and probe the degree to which the estimator retains these advantages with small to moderate multilevel samples. The estimator emerges as a useful alternative or complement to maximum likelihood because it often outperforms maximum likelihood in small to moderate multilevel samples in terms of convergence, bias, error variance, and power. The proposed estimator is implemented as a function in R using lavaan and is illustrated using a multilevel mediation example.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"55 - 77"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428119879758","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45851821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using Outliers for Theory Building","authors":"M. Gibbert, L. B. Nair, M. Weiss, M. Hoegl","doi":"10.1177/1094428119898877","DOIUrl":"https://doi.org/10.1177/1094428119898877","url":null,"abstract":"Outliers are promising candidates for theory building because they defy expected cause-and-effect relationships. Nonetheless, researchers often treat them as a nuisance and exclude them from further study. In fact, our analysis founds only two article using outliers for theory development in all quantitative articles published from 1993 to 2012 in six major management journals, and less than 5% cared to even mention them (relaying reasons for deleting them, mostly). To rectify this, we provide a roadmap for empirical researchers interested in theory building.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"172 - 181"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428119898877","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43107351","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Clarke, Nick Llewellyn, J. Cornelissen, R. Viney
{"title":"Gesture Analysis and Organizational Research: The Development and Application of a Protocol for Naturalistic Settings","authors":"J. Clarke, Nick Llewellyn, J. Cornelissen, R. Viney","doi":"10.1177/1094428119877450","DOIUrl":"https://doi.org/10.1177/1094428119877450","url":null,"abstract":"Gestures are an underresearched but potentially significant aspect of organizational conduct that is relevant to researchers across a range of theoretical and empirical domains. In engaging the cross-disciplinary field of gesture studies, we develop and apply a protocol for analyzing gestures produced in naturalistic settings during ongoing streams of talk and embodied activity. Analyzing video recordings of entrepreneurial investor pitches, we work through this protocol and demonstrate its usefulness. While doing so, we also explore methodological tensions in gesture studies and draw out methodological arguments as they relate to the analysis of these fleeting and often intricate bodily movements. The article contributes a generally applicable protocol for the analysis of gestures in naturalistic settings, and it assesses the methodological implications of this protocol both for research on entrepreneurship and new venture creation and management and organization research more generally.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"140 - 171"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428119877450","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43387032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using Electronic Confederates for Experimental Research in Organizational Science","authors":"Keith Leavitt, Feng Qiu, Debra L. Shapiro","doi":"10.1177/1094428119889136","DOIUrl":"https://doi.org/10.1177/1094428119889136","url":null,"abstract":"Organizational scholars frequently rely on experiments using human confederates or descriptions of vignette characters to study a range of phenomena. Although experiments with confederates allow for realism and rigor, human confederates have several critical limitations. We present a novel and efficient alternative: the use of responsive electronic confederates for manipulating constructs in dyadic, group, and team contexts. Specifically, we (a) define electronic confederates in an organizational research context, identify their optimal qualities, and review studies that have used them; (b) discuss challenges of utilizing human confederates and how electronic confederates may address these; (c) identify boundary conditions around using electronic confederates and, within these conditions, identify the many types of inquiry that can be aided by electronic confederates; (d) discuss validation strategies for electronic confederates, while increasing their believability to study participants; (e) provide materials for two versions of an adaptable research platform involving electronic confederates; and (f) identify future opportunities for developing novel tools for behavioral research. Our article thus provides a toolkit for organizational researchers that empowers them to utilize electronic confederates in their own research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"3 - 25"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428119889136","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43778800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Rethinking the Gold Standard With Multi-armed Bandits: Machine Learning Allocation Algorithms for Experiments","authors":"Chris Kaibel, Torsten Biemann","doi":"10.1177/1094428119854153","DOIUrl":"https://doi.org/10.1177/1094428119854153","url":null,"abstract":"In experiments, researchers commonly allocate subjects randomly and equally to the different treatment conditions before the experiment starts. While this approach is intuitive, it means that new information gathered during the experiment is not utilized until after the experiment has ended. Based on methodological approaches from other scientific disciplines such as computer science and medicine, we suggest machine learning algorithms for subject allocation in experiments. Specifically, we discuss a Bayesian multi-armed bandit algorithm for randomized controlled trials and use Monte Carlo simulations to compare its efficiency with randomized controlled trials that have a fixed and balanced subject allocation. Our findings indicate that a randomized allocation based on Bayesian multi-armed bandits is more efficient and ethical in most settings. We develop recommendations for researchers and discuss the limitations of our approach.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"78 - 103"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428119854153","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41983757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Cortina, Hannah M. Markell-Goldstein, Jennifer P. Green, Yingyi Chang
{"title":"How Are We Testing Interactions in Latent Variable Models? Surging Forward or Fighting Shy?","authors":"J. Cortina, Hannah M. Markell-Goldstein, Jennifer P. Green, Yingyi Chang","doi":"10.1177/1094428119872531","DOIUrl":"https://doi.org/10.1177/1094428119872531","url":null,"abstract":"Latent variable models and interaction effects have both been common in the organizational sciences for some time. Methods for incorporating interactions into latent variable models have existed since at least Kenny and Judd, and a great many articles and books have developed these methods further. In the present article, we present an empirical review of the methods that organizational science investigators use to test their interaction hypotheses. We show that it is very common for investigators to use fully latent methods to test additive portions of their models, but to abandon such methods when testing the multiplicative portions of their models. By contrast, investigators whose models do not contain interactions tend to stick with fully latent methods throughout. As there is little rational basis for this pattern, it is likely due to continued discomfort regarding the proper application of existing fully latent methods. Thus, we end by offering R code that implements some of the more sophisticated fully latent approaches, and by offering a sequence of decisions that investigators can follow in order to choose the best analytic approach.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 1","pages":"26 - 54"},"PeriodicalIF":9.5,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428119872531","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42487641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Methodological Socialization and Identity: A Bricolage Study of Pathways Toward Qualitative Research in Doctoral Education","authors":"Sebnem Cilesiz, Thomas Greckhamer","doi":"10.1177/1094428120980047","DOIUrl":"https://doi.org/10.1177/1094428120980047","url":null,"abstract":"Trends toward convergence on common methodologies and standardized templates restrict the diversity of qualitative methods in organizational research. Considering that graduate education is a critical process in the socialization of researchers into the norms and dominant practices of their discipline, graduate students’ socialization into research methodologies is vital for understanding methodological convergence. The purpose of our study was to understand how graduate students’ socialization shapes their methodological and paradigmatic preferences. Showcasing methodological bricolage as an alternative to qualitative templates, we constructed a research design that combined thematic, discourse, and narrative analyses to investigate graduate students’ reflections throughout a qualitative methods course introducing alternative research paradigms. Our findings highlight the role of institutional, disciplinary, and personal influences as well as identity work in researchers’ socialization and trace alternative trajectories by which socialization and methodological identity construction processes may unfold. We offer a sketch of methodological socialization and suggest that its understanding should be central to nurturing paradigmatic and methodological plurality in qualitative research. We conclude with implications for future research and for research methods training.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"25 1","pages":"337 - 370"},"PeriodicalIF":9.5,"publicationDate":"2020-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1094428120980047","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41650099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}