Organizational Research Methods最新文献

Efficient Processing of Long Sequence Text Data in Transformer: An Examination of Five Different Approaches

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2025-03-18 DOI: 10.1177/10944281251326062

Zihao Jia, Philseok Lee

{"title":"Efficient Processing of Long Sequence Text Data in Transformer: An Examination of Five Different Approaches","authors":"Zihao Jia, Philseok Lee","doi":"10.1177/10944281251326062","DOIUrl":"https://doi.org/10.1177/10944281251326062","url":null,"abstract":"The advent of machine learning and artificial intelligence has profoundly transformed organizational research, especially with the growing application of natural language processing (NLP). Despite these advances, managing long-sequence text input data remains a persistent and significant challenge in NLP analysis within organizational studies. This study introduces five different approaches for handling long sequence text data: term frequency-inverse document frequency with a random forest algorithm (TF-IDF-RF), Longformer, GPT-4o, truncation with averaged scores and our proposed construct-relevant text-selection approach. We also present analytical strategies for each approach and evaluate their effectiveness by comparing the psychometric properties of the predicted scores. Among them, GPT-4o, the truncation with averaged scores, and the proposed text-selection approach generally demonstrate slightly superior psychometric properties compared to TF-IDF-RF and Longformer. However, no single approach consistently outperforms the others across all psychometric criteria. The discussion explores the practical considerations, limitations, and potential directions for future research on these methods, enriching the dialogue on effective long-sequence text management in NLP-driven organizational research.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"22 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143653938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

What Are Mechanisms? Ways of Conceptualizing and Studying Causal Mechanisms

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2025-03-14 DOI: 10.1177/10944281251318727

Joep P. Cornelissen, Mirjam Werner

引用次数: 0

Shedding Light on the Black Box: Integrating Prediction Models and Explainability Using Explainable Machine Learning

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2025-03-13 DOI: 10.1177/10944281251323248

Yucheng Zhang, Yuyan Zheng, Dan Wang, Xiaowei Gu, Michael J. Zyphur, Lin Xiao, Shudi Liao, Yangyang Deng

{"title":"Shedding Light on the Black Box: Integrating Prediction Models and Explainability Using Explainable Machine Learning","authors":"Yucheng Zhang, Yuyan Zheng, Dan Wang, Xiaowei Gu, Michael J. Zyphur, Lin Xiao, Shudi Liao, Yangyang Deng","doi":"10.1177/10944281251323248","DOIUrl":"https://doi.org/10.1177/10944281251323248","url":null,"abstract":"In contemporary organizational research, when dealing with large heterogeneous datasets and complex relationships, statistical modeling focused on developing substantive explanations typically results in low predictive accuracy. In contrast, machine learning (ML) exhibits remarkable strength for prediction, but suffers from an unexplainable analytical process and output—thus ML is often known as a “black box” approach. The recent development of explainable machine learning (XML) integrates high predictive accuracy with explainability, which combines the advantages inherent in both statistical modeling and ML paradigms. This paper compares XML with statistical modeling and the traditional ML approaches, focusing on an advanced application of XML known as evolving fuzzy system (EFS), which enhances model transparency by clarifying the unique contribution of each modeled predictor. In an illustrative study, we demonstrate two EFS-based XML models and conduct comparative analyses among XML, ML, and statistical models with a commonly-used database in organizational research. Our study offers a thorough description of analysis procedures for implementing XML in organizational research, along with best-practice recommendations for each step as well as Python code to aid future research using XML. Finally, we discuss the benefits of XML for organizational research and its potential development.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"14 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143618540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Feature Topic for ORM: Advanced Analytic Approaches to Theorize From Qualitative Research

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2025-01-31 DOI: 10.1177/10944281251314059

Tine Köhler, Anne Smith, Thomas Greckhamer, Jane Lê

引用次数: 0

Surveying the Upper Echelons: An Update to Cycyota and Harrison (2006) on Top Manager Response Rates and Recommendations for the Future 调查高层：对Cycyota和Harrison（2006）关于高层管理者回应率和未来建议的更新

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2025-01-13 DOI: 10.1177/10944281241310574

Cameron J. Borgholthaus, Alaric Bourgoin, Peter D. Harms, Joshua V. White, Tyler N. A. Fezzey

引用次数: 0

Manipulation in Organizational Research: On Executing and Interpreting Designs from Treatments to Primes 组织研究中的操纵：关于执行和解释从治疗到标本的设计

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-12-17 DOI: 10.1177/10944281241300952

Kira F. Schabram, Christopher G. Myers, Ashley E. Hardin

{"title":"Manipulation in Organizational Research: On Executing and Interpreting Designs from Treatments to Primes","authors":"Kira F. Schabram, Christopher G. Myers, Ashley E. Hardin","doi":"10.1177/10944281241300952","DOIUrl":"https://doi.org/10.1177/10944281241300952","url":null,"abstract":"While other applied sciences systematically distinguish between manipulation designs, organizational research does not. Herein, we disentangle distinct applications that differ in how the manipulation is deployed, analyzed, and interpreted in support of hypotheses. First, we define two archetypes: treatments, experimental designs that expose participants to different levels/types of a manipulation of theoretical interest, and primes, manipulations that are not of theoretical interest but generate variance in a state that is. We position these and creative derivations (e.g., interventions and invariant prompts) as specialized tools in our methodological kit. Second, we review 450 manipulations published in leading organizational journals to identify each type's prevalence and application in our field. From this we derive our guiding thesis that while treatments offer unique advantages (foremost establishing causality), they are not always possible, nor the best fit for a research question; in these cases, a non-causal but accurate test of theory, such as a prime design, may prove superior to a causal but inaccurate test. We conclude by outlining best practices for selection, execution, and evaluation by researchers, reviewers, and readers.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"86 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142841975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Measuring Personality When Stakes Are High: Are Graded Paired Comparisons a More Reliable Alternative to Traditional Forced-Choice Methods? 高风险下的人格测量：分级配对比较法是传统强迫选择法的更可靠替代方法吗？

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-12-13 DOI: 10.1177/10944281241279790

Harriet Lingel, Paul-Christian Bürkner, Klaus G. Melchers, Niklas Schulte

{"title":"Measuring Personality When Stakes Are High: Are Graded Paired Comparisons a More Reliable Alternative to Traditional Forced-Choice Methods?","authors":"Harriet Lingel, Paul-Christian Bürkner, Klaus G. Melchers, Niklas Schulte","doi":"10.1177/10944281241279790","DOIUrl":"https://doi.org/10.1177/10944281241279790","url":null,"abstract":"In graded paired comparisons (GPCs), two items are compared using a multipoint rating scale. GPCs are expected to reduce faking compared with Likert-type scales and to produce more reliable, less ipsative trait scores than traditional binary forced-choice formats. To investigate the statistical properties of GPCs, we simulated 960 conditions in which we varied six independent factors and additionally implemented conditions with algorithmically optimized item combinations. Using Thurstonian IRT models, good reliabilities and low ipsativity of trait score estimates were achieved for questionnaires with 50% unequally keyed item pairs or equally keyed item pairs with an optimized combination of loadings. However, in conditions with 20% unequally keyed item pairs and equally keyed conditions without optimization, reliabilities were lower with evidence of ipsativity. Overall, more response categories led to higher reliabilities and nearly fully normative trait scores. In an empirical example, we demonstrate the identified mechanisms under both honest and faking conditions and study the effects of social desirability matching on reliability. In sum, our studies inform about the psychometric properties of GPCs under different conditions and make specific recommendations for improving these properties.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"29 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142820668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Internet Never Forgets: A Four-Step Scraping Tutorial, Codebase, and Database for Longitudinal Organizational Website Data 互联网永远不会忘记：纵向组织网站数据的四步抓取教程、代码库和数据库

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-11-04 DOI: 10.1177/10944281241284941

Richard F.J. Haans, Marc J. Mertens

{"title":"The Internet Never Forgets: A Four-Step Scraping Tutorial, Codebase, and Database for Longitudinal Organizational Website Data","authors":"Richard F.J. Haans, Marc J. Mertens","doi":"10.1177/10944281241284941","DOIUrl":"https://doi.org/10.1177/10944281241284941","url":null,"abstract":"Websites represent a crucial avenue for organizations to reach customers, attract talent, and disseminate information to stakeholders. Despite their importance, strikingly little work in the domain of organization and management research has tapped into this source of longitudinal big data. In this paper, we highlight the unique nature and profound potential of longitudinal website data and present novel open-source code- and databases that make these data accessible. Specifically, our codebase offers a general-purpose setup, building on four central steps to scrape historical websites using the Wayback Machine. Our open-access CompuCrawl database was built using this four-step approach. It contains websites of North American firms in the Compustat database between 1996 and 2020—covering 11,277 firms with 86,303 firm/year observations and 1,617,675 webpages. We describe the coverage of our database and illustrate its use by applying word-embedding models to reveal the evolving meaning of the concept of “sustainability” over time. Finally, we outline several avenues for future research enabled by our step-by-step longitudinal web scraping approach and our CompuCrawl database.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"140 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142580273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

One Size Does Not Fit All: Unraveling Item Response Process Heterogeneity Using the Mixture Dominance-Unfolding Model (MixDUM) 千篇一律：利用混合优势-折叠模型（MixDUM）揭示项目响应过程的异质性

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-09-14 DOI: 10.1177/10944281241271323

Bo Zhang, R. Philip Chalmers, Lingyue Li, Tianjun Sun, Louis Tay

{"title":"One Size Does Not Fit All: Unraveling Item Response Process Heterogeneity Using the Mixture Dominance-Unfolding Model (MixDUM)","authors":"Bo Zhang, R. Philip Chalmers, Lingyue Li, Tianjun Sun, Louis Tay","doi":"10.1177/10944281241271323","DOIUrl":"https://doi.org/10.1177/10944281241271323","url":null,"abstract":"When modeling responses to items measuring non-cognitive constructs that require introspection (e.g., personality, attitude), most studies have assumed that respondents follow the same item response process—either a dominance or an unfolding one. Nevertheless, the results are not equivocal, as some preliminary evidence suggests that some people use an unfolding response process, whereas others use a dominance response process. To enhance item response modeling, it is critical to develop measurement models that can accommodate heterogeneity in the item response processes. Therefore, we proposed the Mixture Dominance-Unfolding Model (MixDUM) to formally identify this potential population heterogeneity. Monte Carlo simulations showed that MixDUM possessed reasonably good statistical properties. Moreover, ignoring item response process heterogeneity was detrimental to item parameter estimation and led to less accurate selection outcomes. An empirical study was conducted in which respondents completed focal personality scales under either an honest condition or a simulated job application condition, to demonstrate the utility of MixDUM. The findings indicated (1) that MixDUM provided the best fit across scales, (2) that approximately 55–60% of respondents utilized an unfolding response process, (3) that respondents exhibited moderate consistency in their use of response processes across scales, (4) that narcissism consistently negatively predicted the use of an unfolding response process, and (5) that the criterion-related validity of focal personality scores varied across latent classes for certain criteria. To encourage its use, we provided a tutorial on the implementation of MixDUM in the R package mirt.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"27 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142233372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Taking It Easy: Off-the-Shelf Versus Fine-Tuned Supervised Modeling of Performance Appraisal Text 从容应对：现成与微调的绩效考核文本监督建模比较

IF 9.5 2区管理学

Organizational Research Methods Pub Date : 2024-08-28 DOI: 10.1177/10944281241271249

Andrew B. Speer, James Perrotta, Tobias L. Kordsmeyer

{"title":"Taking It Easy: Off-the-Shelf Versus Fine-Tuned Supervised Modeling of Performance Appraisal Text","authors":"Andrew B. Speer, James Perrotta, Tobias L. Kordsmeyer","doi":"10.1177/10944281241271249","DOIUrl":"https://doi.org/10.1177/10944281241271249","url":null,"abstract":"When assessing text, supervised natural language processing (NLP) models have traditionally been used to measure targeted constructs in the organizational sciences. However, these models require significant resources to develop. Emerging “off-the-shelf” large language models (LLM) offer a way to evaluate organizational constructs without building customized models. However, it is unclear whether off-the-shelf LLMs accurately score organizational constructs and what evidence is necessary to infer validity. In this study, we compared the validity of supervised NLP models to off-the-shelf LLM models (ChatGPT-3.5 and ChatGPT-4). Across six organizational datasets and thousands of comments, we found that supervised NLP produced scores were more reliable than human coders. However, and even though not specifically developed for this purpose, we found that off-the-shelf LLMs produce similar psychometric properties as supervised models, though with slightly less favorable psychometric properties. We connect these findings to broader validation considerations and present a decision chart to guide researchers and practitioners on how they can use off-the-shelf LLM models to score targeted constructs, including guidance on how psychometric evidence can be “transported” to new contexts.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"98 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142089968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0