Using natural language processing to increase prediction and reduce subgroup differences in personnel selection decisions.

IF 9.4 1区 心理学 Q1 MANAGEMENT
Journal of Applied Psychology Pub Date : 2024-03-01 Epub Date: 2023-10-19 DOI:10.1037/apl0001144
Emily D Campion, Michael A Campion, James Johnson, Thomas R Carretta, Sophie Romay, Bobbie Dirr, Andrew Deregla, Amanda Mouton
{"title":"Using natural language processing to increase prediction and reduce subgroup differences in personnel selection decisions.","authors":"Emily D Campion, Michael A Campion, James Johnson, Thomas R Carretta, Sophie Romay, Bobbie Dirr, Andrew Deregla, Amanda Mouton","doi":"10.1037/apl0001144","DOIUrl":null,"url":null,"abstract":"<p><p>The purpose of this research is to demonstrate how using natural language processing (NLP) on narrative application data can improve prediction and reduce racial subgroup differences in scores used for selection decisions compared to mental ability test scores and numeric application data. We posit there is uncaptured and job-related constructs that can be gleaned from applicant text data using NLP. We test our hypotheses in an operational context across four samples (total <i>N</i> = 1,828) to predict selection into Officer Training School in the U.S. Air Force. Boards of three senior officers make selection decisions using a highly structured rating process based on mental ability tests, numeric application information (e.g., number of past jobs, college grades), and narrative application information (e.g., past job duties, achievements, interests, statements of objectives). Results showed that NLP scores of the narrative application generally (a) predict Board scores when combined with test scores and numeric application information at a level of correlation equivalent to the correlation between human raters (.60), (b) add incremental prediction of Board scores beyond mental ability tests and numeric application information, and (c) reduce subgroup differences between racial minorities and nonracial minorities in Board scores compared to mental ability tests and numeric application information. Moreover, NLP scores predict (a) job (training) performance, (b) job (training) performance beyond mental ability tests and numeric application information, and (c) even job (training) performance beyond Board scores. Scoring of narrative application data using NLP shows promise in addressing the validity-adverse impact dilemma in selection. (PsycInfo Database Record (c) 2024 APA, all rights reserved).</p>","PeriodicalId":15135,"journal":{"name":"Journal of Applied Psychology","volume":null,"pages":null},"PeriodicalIF":9.4000,"publicationDate":"2024-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Applied Psychology","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/apl0001144","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/10/19 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"MANAGEMENT","Score":null,"Total":0}
引用次数: 0

Abstract

The purpose of this research is to demonstrate how using natural language processing (NLP) on narrative application data can improve prediction and reduce racial subgroup differences in scores used for selection decisions compared to mental ability test scores and numeric application data. We posit there is uncaptured and job-related constructs that can be gleaned from applicant text data using NLP. We test our hypotheses in an operational context across four samples (total N = 1,828) to predict selection into Officer Training School in the U.S. Air Force. Boards of three senior officers make selection decisions using a highly structured rating process based on mental ability tests, numeric application information (e.g., number of past jobs, college grades), and narrative application information (e.g., past job duties, achievements, interests, statements of objectives). Results showed that NLP scores of the narrative application generally (a) predict Board scores when combined with test scores and numeric application information at a level of correlation equivalent to the correlation between human raters (.60), (b) add incremental prediction of Board scores beyond mental ability tests and numeric application information, and (c) reduce subgroup differences between racial minorities and nonracial minorities in Board scores compared to mental ability tests and numeric application information. Moreover, NLP scores predict (a) job (training) performance, (b) job (training) performance beyond mental ability tests and numeric application information, and (c) even job (training) performance beyond Board scores. Scoring of narrative application data using NLP shows promise in addressing the validity-adverse impact dilemma in selection. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

使用自然语言处理来增加预测并减少人员选择决策中的子组差异。
本研究的目的是证明,与心智能力测试分数和数字应用数据相比,对叙述性应用数据使用自然语言处理(NLP)可以提高预测能力,并减少用于选择决策的分数的种族亚组差异。我们假设,使用NLP可以从申请人的文本数据中收集到一些未捕获的和与工作相关的结构。我们在四个样本(总数N=1828)的作战背景下测试了我们的假设,以预测美国空军军官训练学校的选择。由三名高级官员组成的董事会根据心理能力测试、数字申请信息(如过去的工作数量、大学成绩)和叙述性申请信息(例如过去的工作职责、成就、兴趣、目标陈述),使用高度结构化的评级过程做出选拔决定。结果表明,叙述性应用程序的NLP分数通常(a)在与测试分数和数字应用程序信息相结合时预测Board分数,其相关性水平相当于人类评分者之间的相关性(.60),(b)在心智能力测试和数字应用程序信息之外增加Board分数的增量预测,以及(c)与心理能力测试和数字应用信息相比,减少少数种族和非少数种族在委员会分数方面的亚组差异。此外,NLP分数可以预测(a)工作(培训)表现,(b)心理能力测试和数字应用信息之外的工作(训练)表现,以及(c)甚至可以预测董事会分数之外的工作表现。使用NLP对叙述性应用数据进行评分,有望解决选择中的有效性不利影响困境。(PsycInfo数据库记录(c)2023 APA,保留所有权利)。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
17.60
自引率
6.10%
发文量
175
期刊介绍: The Journal of Applied Psychology® focuses on publishing original investigations that contribute new knowledge and understanding to fields of applied psychology (excluding clinical and applied experimental or human factors, which are better suited for other APA journals). The journal primarily considers empirical and theoretical investigations that enhance understanding of cognitive, motivational, affective, and behavioral psychological phenomena in work and organizational settings. These phenomena can occur at individual, group, organizational, or cultural levels, and in various work settings such as business, education, training, health, service, government, or military institutions. The journal welcomes submissions from both public and private sector organizations, for-profit or nonprofit. It publishes several types of articles, including: 1.Rigorously conducted empirical investigations that expand conceptual understanding (original investigations or meta-analyses). 2.Theory development articles and integrative conceptual reviews that synthesize literature and generate new theories on psychological phenomena to stimulate novel research. 3.Rigorously conducted qualitative research on phenomena that are challenging to capture with quantitative methods or require inductive theory building.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信