The AI‐IP: Minimizing the guesswork of personality scale item development through artificial intelligence

IF 4.7 2区 心理学 Q1 MANAGEMENT
Ivan Hernandez, Weiwen Nie
{"title":"The AI‐IP: Minimizing the guesswork of personality scale item development through artificial intelligence","authors":"Ivan Hernandez, Weiwen Nie","doi":"10.1111/peps.12543","DOIUrl":null,"url":null,"abstract":"We propose a framework for integrating various modern natural language processing (NLP) models to assist researchers with developing valid psychological scales. Transformer-based deep neural networks offer state-of-the-art performance on various natural language tasks. This project adapts the transformer model GPT-2 to learn the structure of personality items, and generate the largest openly available pool of personality items, consisting of one million new items. We then use that artificial intelligence-based item pool (AI-IP) to provide a subset of potential scale items for measuring a desired construct. To better recommend construct-related items, we train a paired neural network-based classification BERT model to predict the observed correlation between personality items using only their text. We also demonstrate how zero-shot models can help balance desired content domains within the scale. In combination with the AI-IP, these models narrow the large item pool to items most correlated with a set of initial items. We demonstrate the ability of this multimodel framework to develop longer cohesive scales from a small set of construct-relevant items. We found reliability, validity, and fit equivalent for AI-assisted scales compared to","PeriodicalId":48408,"journal":{"name":"Personnel Psychology","volume":" ","pages":""},"PeriodicalIF":4.7000,"publicationDate":"2022-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Personnel Psychology","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1111/peps.12543","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MANAGEMENT","Score":null,"Total":0}
引用次数: 2

Abstract

We propose a framework for integrating various modern natural language processing (NLP) models to assist researchers with developing valid psychological scales. Transformer-based deep neural networks offer state-of-the-art performance on various natural language tasks. This project adapts the transformer model GPT-2 to learn the structure of personality items, and generate the largest openly available pool of personality items, consisting of one million new items. We then use that artificial intelligence-based item pool (AI-IP) to provide a subset of potential scale items for measuring a desired construct. To better recommend construct-related items, we train a paired neural network-based classification BERT model to predict the observed correlation between personality items using only their text. We also demonstrate how zero-shot models can help balance desired content domains within the scale. In combination with the AI-IP, these models narrow the large item pool to items most correlated with a set of initial items. We demonstrate the ability of this multimodel framework to develop longer cohesive scales from a small set of construct-relevant items. We found reliability, validity, and fit equivalent for AI-assisted scales compared to
AI‐IP:通过人工智能最大限度地减少对个性量表项目开发的猜测
我们提出了一个整合各种现代自然语言处理(NLP)模型的框架,以帮助研究人员开发有效的心理量表。基于变压器的深度神经网络在各种自然语言任务上提供了最先进的性能。本项目采用变形模型GPT-2学习人格项目的结构,生成最大的公开可用的人格项目池,包含100万个新项目。然后,我们使用基于人工智能的项目池(AI-IP)来提供潜在规模项目的子集,用于测量期望的结构。为了更好地推荐与构念相关的项目,我们训练了一个基于配对神经网络的分类BERT模型来预测观察到的个性项目之间的相关性,只使用它们的文本。我们还演示了零射击模型如何帮助平衡尺度内所需的内容域。结合AI-IP,这些模型将大型项目池缩小到与一组初始项目最相关的项目。我们展示了这个多模型框架的能力,可以从一组与构建相关的项目中开发出更长的内聚尺度。我们发现人工智能辅助量表的信度、效度和拟合等效与
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
10.20
自引率
5.50%
发文量
57
期刊介绍: Personnel Psychology publishes applied psychological research on personnel problems facing public and private sector organizations. Articles deal with all human resource topics, including job analysis and competency development, selection and recruitment, training and development, performance and career management, diversity, rewards and recognition, work attitudes and motivation, and leadership.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信