From assistance to reliance: Development and validation of the large language model dependence scale

IF 20.1 1区 管理学 Q1 INFORMATION SCIENCE & LIBRARY SCIENCE
Zewei Li , Zheng Zhang , Mingwei Wang , Qi Wu
{"title":"From assistance to reliance: Development and validation of the large language model dependence scale","authors":"Zewei Li ,&nbsp;Zheng Zhang ,&nbsp;Mingwei Wang ,&nbsp;Qi Wu","doi":"10.1016/j.ijinfomgt.2025.102888","DOIUrl":null,"url":null,"abstract":"<div><div>With the rapid advancement of large language models (LLMs), the phenomenon of LLMs dependence has emerged and garnered significant attention. However, previous scales have been insufficient to measure the extent of individuals' dependence on LLMs. The current study aims to utilize the human-computer trust model and addiction theory to develop and validate the LLMs dependence scale (LDS) and to report its psychometric properties. An exploratory structural investigation of LLMs dependence was conducted with a sample of 421 LLMs users (Sample 1), which included items analysis, exploratory factor analysis, and network analysis. Additionally, a formal test was performed with a separate sample of 1030 LLMs users (Sample 2), with the data undergoing structural validation through confirmatory factor analysis, validity testing, and reliability testing. To mitigate the potential negative impacts of LLMs dependence, we employed the NodeIdentifyR algorithm for computational simulation interventions to identify critical intervention nodes within the LLMs dependence network. The results indicated that the LDS (18 items) exhibited a bifactor structure of functional dependence and existential dependence. The confirmatory factor model demonstrated a good fit and the LDS also showed good criterion-related validity. Subsequent simulated results of alleviating interventions suggested that users' existential dependence was primarily driven by their dependence on LLMs to handle tedious and boring tasks, while functional dependence was more influenced by users' belief in the omnipotence of LLMs. In summary, the factor structure of the LDS is clear, and its reliability and validity indices meet psychometric standards, making it an effective tool for measuring LLMs dependence.</div></div>","PeriodicalId":48422,"journal":{"name":"International Journal of Information Management","volume":"83 ","pages":"Article 102888"},"PeriodicalIF":20.1000,"publicationDate":"2025-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Information Management","FirstCategoryId":"91","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0268401225000209","RegionNum":1,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"INFORMATION SCIENCE & LIBRARY SCIENCE","Score":null,"Total":0}
引用次数: 0

Abstract

With the rapid advancement of large language models (LLMs), the phenomenon of LLMs dependence has emerged and garnered significant attention. However, previous scales have been insufficient to measure the extent of individuals' dependence on LLMs. The current study aims to utilize the human-computer trust model and addiction theory to develop and validate the LLMs dependence scale (LDS) and to report its psychometric properties. An exploratory structural investigation of LLMs dependence was conducted with a sample of 421 LLMs users (Sample 1), which included items analysis, exploratory factor analysis, and network analysis. Additionally, a formal test was performed with a separate sample of 1030 LLMs users (Sample 2), with the data undergoing structural validation through confirmatory factor analysis, validity testing, and reliability testing. To mitigate the potential negative impacts of LLMs dependence, we employed the NodeIdentifyR algorithm for computational simulation interventions to identify critical intervention nodes within the LLMs dependence network. The results indicated that the LDS (18 items) exhibited a bifactor structure of functional dependence and existential dependence. The confirmatory factor model demonstrated a good fit and the LDS also showed good criterion-related validity. Subsequent simulated results of alleviating interventions suggested that users' existential dependence was primarily driven by their dependence on LLMs to handle tedious and boring tasks, while functional dependence was more influenced by users' belief in the omnipotence of LLMs. In summary, the factor structure of the LDS is clear, and its reliability and validity indices meet psychometric standards, making it an effective tool for measuring LLMs dependence.
从辅助到依赖:大型语言模型依赖规模的开发和验证
随着大型语言模型(llm)的快速发展,llm依赖现象已经出现并引起了人们的广泛关注。然而,以往的量表不足以衡量个人对法学硕士的依赖程度。本研究旨在利用人机信任模型和成瘾理论开发和验证LLMs依赖量表(LDS),并报告其心理测量特性。以421名法学硕士用户(样本1)为样本,对法学硕士依赖进行了探索性结构调查,包括项目分析、探索性因子分析和网络分析。此外,对1030名llm用户(样本2)进行了正式测试,并通过验证性因子分析、效度测试和信度测试对数据进行了结构验证。为了减轻llm依赖的潜在负面影响,我们采用NodeIdentifyR算法进行计算模拟干预,以识别llm依赖网络中的关键干预节点。结果表明:18个条目呈现功能依赖和存在依赖的双因子结构。验证性因子模型拟合良好,LDS也显示出良好的标准相关效度。随后的缓解干预模拟结果表明,用户的存在依赖主要是由用户对llm处理繁琐无聊任务的依赖驱动的,而功能依赖更多地受到用户对llm无所不能的信念的影响。综上所述,LDS的因子结构清晰,信度和效度指标符合心理测量标准,是衡量LLMs依赖性的有效工具。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
International Journal of Information Management
International Journal of Information Management INFORMATION SCIENCE & LIBRARY SCIENCE-
CiteScore
53.10
自引率
6.20%
发文量
111
审稿时长
24 days
期刊介绍: The International Journal of Information Management (IJIM) is a distinguished, international, and peer-reviewed journal dedicated to providing its readers with top-notch analysis and discussions within the evolving field of information management. Key features of the journal include: Comprehensive Coverage: IJIM keeps readers informed with major papers, reports, and reviews. Topical Relevance: The journal remains current and relevant through Viewpoint articles and regular features like Research Notes, Case Studies, and a Reviews section, ensuring readers are updated on contemporary issues. Focus on Quality: IJIM prioritizes high-quality papers that address contemporary issues in information management.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信