Avoiding algorithm errors in textual analysis: A guide to selecting software, and a research agenda toward generative artificial intelligence

IF 9.8 1区管理学 Q1 BUSINESS

Journal of Business Research Pub Date : 2025-06-28 DOI:10.1016/j.jbusres.2025.115571

Janice Wobst , Rainer Lueg

{"title":"Avoiding algorithm errors in textual analysis: A guide to selecting software, and a research agenda toward generative artificial intelligence","authors":"Janice Wobst , Rainer Lueg","doi":"10.1016/j.jbusres.2025.115571","DOIUrl":null,"url":null,"abstract":"<div><div>The use of textual analysis is expanding in organizational research, yet software packages vary in their compatibility with complex constructs. This study helps researchers select suitable tools by focusing on phrase-based dictionary methods. We empirically evaluate four software packages—LIWC, DICTION, CAT Scanner, and a custom Python tool—using the complex construct of value-based management as a test case. The analysis shows that software from the same methodological family produces highly consistent results, while popular but mismatched tools yield significant errors such as miscounted phrases. Based on this, we develop a structured selection guideline that links construct features with software capabilities. The framework enhances construct validity, supports methodological transparency, and is applicable across disciplines. Finally, we position the approach as a bridge to AI-enabled textual analysis, including prompt-based workflows, reinforcing the continued need for theory-grounded construct design.</div></div>","PeriodicalId":15123,"journal":{"name":"Journal of Business Research","volume":"199 ","pages":"Article 115571"},"PeriodicalIF":9.8000,"publicationDate":"2025-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Business Research","FirstCategoryId":"91","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0148296325003947","RegionNum":1,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"BUSINESS","Score":null,"Total":0}

引用次数: 0

Abstract

The use of textual analysis is expanding in organizational research, yet software packages vary in their compatibility with complex constructs. This study helps researchers select suitable tools by focusing on phrase-based dictionary methods. We empirically evaluate four software packages—LIWC, DICTION, CAT Scanner, and a custom Python tool—using the complex construct of value-based management as a test case. The analysis shows that software from the same methodological family produces highly consistent results, while popular but mismatched tools yield significant errors such as miscounted phrases. Based on this, we develop a structured selection guideline that links construct features with software capabilities. The framework enhances construct validity, supports methodological transparency, and is applicable across disciplines. Finally, we position the approach as a bridge to AI-enabled textual analysis, including prompt-based workflows, reinforcing the continued need for theory-grounded construct design.

查看原文本刊更多论文

避免文本分析中的算法错误：软件选择指南，以及生成人工智能的研究议程

文本分析的使用在组织研究中正在扩展，但是软件包在与复杂结构的兼容性上有所不同。本研究通过关注基于短语的词典方法，帮助研究者选择合适的工具。我们使用基于值的管理的复杂结构作为测试用例，对四个软件包（liwc、DICTION、CAT Scanner和自定义Python工具）进行了实证评估。分析表明，来自相同方法家族的软件产生高度一致的结果，而流行但不匹配的工具产生重大错误，例如错误计数短语。在此基础上，我们开发了一个结构化的选择指南，将构造特性与软件功能联系起来。该框架增强了结构的有效性，支持方法的透明度，并且可以跨学科应用。最后，我们将该方法定位为人工智能文本分析的桥梁，包括基于提示的工作流程，加强了对基于理论的结构设计的持续需求。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Business Research BUSINESS-

CiteScore

20.30

自引率

10.60%

发文量

956

期刊介绍： The Journal of Business Research aims to publish research that is rigorous, relevant, and potentially impactful. It examines a wide variety of business decision contexts, processes, and activities, developing insights that are meaningful for theory, practice, and/or society at large. The research is intended to generate meaningful debates in academia and practice, that are thought provoking and have the potential to make a difference to conceptual thinking and/or practice. The Journal is published for a broad range of stakeholders, including scholars, researchers, executives, and policy makers. It aids the application of its research to practical situations and theoretical findings to the reality of the business world as well as to society. The Journal is abstracted and indexed in several databases, including Social Sciences Citation Index, ANBAR, Current Contents, Management Contents, Management Literature in Brief, PsycINFO, Information Service, RePEc, Academic Journal Guide, ABI/Inform, INSPEC, etc.