Organizational Research Methods最新文献_第2页

Hello World! Building Computational Models to Represent Social and Organizational Theory 你好，世界！建立代表社会和组织理论的计算模型

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-07-25 DOI: 10.1177/10944281241261913

James A. Grand, Michael T. Braun, Goran Kuljanin

{"title":"Hello World! Building Computational Models to Represent Social and Organizational Theory","authors":"James A. Grand, Michael T. Braun, Goran Kuljanin","doi":"10.1177/10944281241261913","DOIUrl":"https://doi.org/10.1177/10944281241261913","url":null,"abstract":"Computational modeling holds significant promise as a tool for improving how theory is developed, expressed, and used to inform empirical research and evaluation efforts. However, the knowledge and skillsets needed to build computational models are rarely developed in the training received by social and organizational scientists. The purpose of this manuscript is to provide an accessible introduction to and reference for building computational models to represent theory. We first discuss important principles and recommendations for “thinking about” theory and developing explanatory accounts in ways that facilitate translating their core assumptions, specifications, and ideas into a computational model. Next, we address some frequently asked questions related to building computational models that introduce several fundamental tasks/concepts involved in building models to represent theory and demonstrate how they can be implemented in the R programming language to produce executable model code. The accompanying supplemental materials describes additional considerations relevant to building and using computational models, provides multiple examples of complete computational model code written in R, and an interactive application offering guided practice on key model-building tasks/concepts in R.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"1 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141764124","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Effects of the Training Sample Size, Ground Truth Reliability, and NLP Method on Language-Based Automatic Interview Scores’ Psychometric Properties 训练样本规模、地面实况可靠性和 NLP 方法对基于语言的自动访谈评分心理测量特性的影响

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-07-25 DOI: 10.1177/10944281241264027

Louis Hickman, Josh Liff, Caleb Rottman, Charles Calderwood

{"title":"The Effects of the Training Sample Size, Ground Truth Reliability, and NLP Method on Language-Based Automatic Interview Scores’ Psychometric Properties","authors":"Louis Hickman, Josh Liff, Caleb Rottman, Charles Calderwood","doi":"10.1177/10944281241264027","DOIUrl":"https://doi.org/10.1177/10944281241264027","url":null,"abstract":"While machine learning (ML) can validly score psychological constructs from behavior, several conditions often change across studies, making it difficult to understand why the psychometric properties of ML models differ across studies. We address this gap in the context of automatically scored interviews. Across multiple datasets, for interview- or question-level scoring of self-reported, tested, and interviewer-rated constructs, we manipulate the training sample size and natural language processing (NLP) method while observing differences in ground truth reliability. We examine how these factors influence the ML model scores’ test–retest reliability and convergence, and we develop multilevel models for estimating the convergent-related validity of ML model scores in similar interviews. When the ground truth is interviewer ratings, hundreds of observations are adequate for research purposes, while larger samples are recommended for practitioners to support generalizability across populations and time. However, self-reports and tested constructs require larger training samples. Particularly when the ground truth is interviewer ratings, NLP embedding methods improve upon count-based methods. Given mixed findings regarding ground truth reliability, we discuss future research possibilities on factors that affect supervised ML models’ psychometric properties.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"37 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141764243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhancing Causal Pursuits in Organizational Science: Targeting the Effect of Treatment on the Treated in Research on Vulnerable Populations 加强组织科学的因果追求：在弱势人群研究中针对治疗对被治疗者的影响

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-05-02 DOI: 10.1177/10944281241246772

Wen Wei Loh, Dongning Ren

{"title":"Enhancing Causal Pursuits in Organizational Science: Targeting the Effect of Treatment on the Treated in Research on Vulnerable Populations","authors":"Wen Wei Loh, Dongning Ren","doi":"10.1177/10944281241246772","DOIUrl":"https://doi.org/10.1177/10944281241246772","url":null,"abstract":"Understanding the experiences of vulnerable workers is an important scientific pursuit. For example, research interest is often in quantifying the impacts of adverse exposures such as discrimination, exclusion, harassment, or job insecurity, among others. However, routine approaches have only focused on the average treatment effect, which encapsulates the impact of an exposure (e.g., discrimination) applied to the entire study population—including those who were not exposed. In this paper, we propose using a more refined causal quantity uniquely suited to address such causal queries: The effect of treatment on the treated (ETT) from the causal inference literature. We explain why the ETT is a more pertinent causal estimand for investigating the experiences of vulnerable workers by highlighting three appealing features: Better interpretability, greater accuracy, and enhanced robustness to violations of empirically untestable causal assumptions. We further describe how to estimate the ETT by introducing and comparing two estimators. Both estimators are conferred with a so-called doubly robust property. We hope the current proposal empowers organizational scholars in their crucial endeavors dedicated to understanding the vulnerable workforce.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"51 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140826389","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Analyzing Social Interaction in Organizations: A Roadmap for Reflexive Choice 分析组织中的社会互动：反思性选择路线图

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-04-22 DOI: 10.1177/10944281241245444

Linda Jakob Sadeh, Avital Baikovich, Tammar B. Zilber

{"title":"Analyzing Social Interaction in Organizations: A Roadmap for Reflexive Choice","authors":"Linda Jakob Sadeh, Avital Baikovich, Tammar B. Zilber","doi":"10.1177/10944281241245444","DOIUrl":"https://doi.org/10.1177/10944281241245444","url":null,"abstract":"This article proposes a framework for reflexive choice in qualitative research, centering on social interaction. Interaction, fundamental to social and organizational life, has been studied extensively. Yet, researchers can get lost in the plethora of methodological tools, hampering reflexive choice. Our proposed framework consists of four dimensions of interaction (content, communication patterns, emotions, and roles), intersecting with five levels of analysis (individual, dyadic, group, organizational, and sociocultural), as well as three overarching analytic principles (following the dynamic, consequential, and contextual nature of interaction). For each intersection between dimension and level, we specify analytical questions, empirical markers, and references to exemplary works. The framework functions both as a compass, indicating potential directions for research design and data collection methods, and as a roadmap, illuminating pathways at the analysis stage. Our contributions are twofold: First, our framework fleshes out the broad spectrum of available methods for analyzing interaction, providing pragmatic tools for the researcher to reflexively choose from. Second, we highlight the broader relevance of maps, such as our own, for enhancing reflexive methodological choices.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"9 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140637754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Advancing Qualitative Meta-Studies (QMS): Current Practices and Reflective Guidelines for Synthesizing Qualitative Research 推进定性元研究（QMS）：综合定性研究的现行做法和反思指南

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-04-17 DOI: 10.1177/10944281241240180

Stefanie Habersang, Markus Reihlen

引用次数: 0

Simulating Virtual Organizations for Research: A Comparative Empirical Evaluation of Text-Based, Video, and Virtual Reality Video Vignettes 为研究模拟虚拟组织：基于文本、视频和虚拟现实视频小故事的比较实证评估

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-04-17 DOI: 10.1177/10944281241246770

Anand P. A. van Zelderen, Theodore C. Masters-Waage, Nicky Dries, Jochen I. Menges, Diana R. Sanchez

{"title":"Simulating Virtual Organizations for Research: A Comparative Empirical Evaluation of Text-Based, Video, and Virtual Reality Video Vignettes","authors":"Anand P. A. van Zelderen, Theodore C. Masters-Waage, Nicky Dries, Jochen I. Menges, Diana R. Sanchez","doi":"10.1177/10944281241246770","DOIUrl":"https://doi.org/10.1177/10944281241246770","url":null,"abstract":"Due to recent technological developments, vignette studies that have traditionally been done in text or video formats can now be done in immersive formats using virtual reality—but are such virtual reality video vignettes superior to traditional vignettes? To address this question, we examine participants’ experiences within a fictitious organization by comparing their responses to a relevant and particularly sensitive organizational phenomenon presented either through written text, a video recording, or a virtual reality experience. The results indicate that participants prefer more immersive methods, and that these increase their attention to critical study details. Moreover, this augments the effect sizes of several measured employee reactions—particularly those with high emotional content—suggesting that virtual reality technology offers a promising avenue for developing ecologically valid vignette studies to measure employee affect. To facilitate and expediate the use of virtual reality video vignettes in organizational research, we provide organizational scholars with a step-by-step instructional guide to develop immersive vignette studies.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"78 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140608136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Framework for Detecting Both Main Effect and Interactive DIF in Multidimensional Forced-Choice Assessments 在多维强制选择测评中检测主效应和交互式 DIF 的框架

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-04-13 DOI: 10.1177/10944281241244760

Kai Liu, Yi Zheng, Daxun Wang, Yan Cai, Yuanyuan Shi, Chongqin Xi, Dongbo Tu

{"title":"A Framework for Detecting Both Main Effect and Interactive DIF in Multidimensional Forced-Choice Assessments","authors":"Kai Liu, Yi Zheng, Daxun Wang, Yan Cai, Yuanyuan Shi, Chongqin Xi, Dongbo Tu","doi":"10.1177/10944281241244760","DOIUrl":"https://doi.org/10.1177/10944281241244760","url":null,"abstract":"In recent decades, multidimensional forced-choice (MFC) tests have gained widespread popularity in organizational settings due to their effectiveness in reducing response biases. Detecting differential item functioning (DIF) is crucial in developing MFC tests, as it relates to test fairness and validity. However, existing methods appear insufficient for detecting DIF induced by the interaction between multiple covariates. Furthermore, for multi-category, ordered or continuous covariates, existing approaches often dichotomize them using a-priori cutoffs, commonly using the median of the covariates. This may lead to information loss and reduced power in detecting MFC DIF. To address these limitations, we propose a method to identify both main effect DIF and interactive DIF. This method can automatically search for the optimal cutoffs for ordered or continuous covariates without pre-defined cutoffs. We introduce the rationale behind the proposed method and evaluate its performance through three Monte Carlo simulation studies. Results demonstrate that the proposed method effectively identifies various DIF forms in MFC tests, thereby increasing detection power. Finally, we provide an empirical application to illustrate the practical applicability of the proposed method.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"9 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140551921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

From Textual Data to Theoretical Insights: Introducing and Applying the Word-Text-Topic Extraction Approach 从文本数据到理论见解：单词-文本-主题提取方法的介绍和应用

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-01-31 DOI: 10.1177/10944281241228186

Jaewoo Jung, Wenjun Zhou, Anne D. Smith

{"title":"From Textual Data to Theoretical Insights: Introducing and Applying the Word-Text-Topic Extraction Approach","authors":"Jaewoo Jung, Wenjun Zhou, Anne D. Smith","doi":"10.1177/10944281241228186","DOIUrl":"https://doi.org/10.1177/10944281241228186","url":null,"abstract":"Text analysis, particularly custom dictionaries and topic modeling, has helped advance management and organization theory. Custom dictionaries involve creating word lists to quantify patterns and infer constructs, while topic modeling extracts themes from textual documents to help understand a theoretical domain. Building on these two approaches, we propose another text analysis approach called word-text-topic extraction (WTT), which enhances the efficiency and relevance of text analysis for the sake of theoretical advancement. Specifically, we first identify relevant words for a researcher's theoretical area of interest using word-embedding algorithms. That step is followed by extracting text segments from the textual corpus using a collocation process. Finally, topic modeling is applied to capture themes relevant to the specific theoretical area of interest. To illustrate the WTT approach, we explored one research area needing further theory development—innovation. Using 841 CEOs’ letters to shareholders, we found that our WTT approach provides nuanced features of innovation that differ across industry contexts. We guide researchers on decisions and considerations related to the WTT approach in order to facilitate its use in future studies.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"99 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-01-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139938974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Five Is the Brightest Star. But by how Much? Testing the Equidistance of Star Ratings in Online Reviews 五是最亮的星。但有多亮？测试在线评论中星级评定的等距性

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-01-08 DOI: 10.1177/10944281231223412

Balázs Kovács

{"title":"Five Is the Brightest Star. But by how Much? Testing the Equidistance of Star Ratings in Online Reviews","authors":"Balázs Kovács","doi":"10.1177/10944281231223412","DOIUrl":"https://doi.org/10.1177/10944281231223412","url":null,"abstract":"Organizational research increasingly relies on online review data to gauge perceived valuation and reputation of organizations and products. Online review platforms typically collect ordinal ratings (e.g., 1 to 5 stars); however, researchers often treat them as a cardinal data, calculating aggregate statistics such as the average, the median, or the variance of ratings. In calculating these statistics, ratings are implicitly assumed to be equidistant. We test whether star ratings are equidistant using reviews from two large-scale online review platforms: Amazon.com and Yelp.com. We develop a deep learning framework to analyze the text of the reviews in order to assess their overall valuation. We find that 4 and 5-star ratings, as well as 1 and 2-star ratings, are closer to each other than 3-star ratings are to 2 and 4-star ratings. An additional online experiment corroborates this pattern. Using simulations, we show that the distortion by non-equidistant ratings is especially harmful in cases when organizations receive only a few reviews and when researchers are interested in estimating variance effects. We discuss potential solutions to solve the issue with rating non-equidistance.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"22 12","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139446588","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The VIF Score. What is it Good For? Absolutely Nothing VIF 分数。有什么用？没什么用

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2023-12-13 DOI: 10.1177/10944281231216381

Arturs Kalnins, Kendall Praitis Hill

{"title":"The VIF Score. What is it Good For? Absolutely Nothing","authors":"Arturs Kalnins, Kendall Praitis Hill","doi":"10.1177/10944281231216381","DOIUrl":"https://doi.org/10.1177/10944281231216381","url":null,"abstract":"Variance inflation factors (VIF scores) are regression diagnostics commonly invoked throughout the social sciences. Researchers typically take the perspective that VIF scores below a numerical rule-of-thumb threshold act as a “silver bullet” to dismiss any and all multicollinearity concerns. Yet, no valid logical basis exists for using VIF thresholds to reject the possibility of multicollinearity-induced type 1 errors. Reporting VIF scores below a threshold does not in any way add to the credibility of statistically significant results among correlated variables. In contrast to this “threshold perspective,” our analysis expands the scope of a perspective that has considered multicollinearity and misspecification. We demonstrate analytically that a regression omitting a relevant variable correlated with included variables that exhibit multicollinearity is susceptible to endogeneity-induced bias inflation and beta polarization, leading to the possible co-existence of type 1 errors and low VIF scores. Further, omitting variables explicitly reduces VIF scores. We conclude that the threshold perspective not only lacks any logical basis but also is fundamentally misleading as a rule-of-thumb. Instrumental variables represent one clear remedy for endogeneity-induced bias inflation. If exogenous instruments are unavailable, we encourage researchers to test only straightforward, unambiguous theory when using variables that exhibit multicollinearity, and to ensure that correlated co-variates exhibit the expected signs.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"24 4","pages":""},"PeriodicalIF":9.5,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139005307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0