International Journal of Selection and Assessment最新文献_第6页

The performance of large language models on quantitative and verbal ability tests: Initial evidence and implications for unproctored high-stakes testing 大语言模型在定量和言语能力测试中的表现：初步证据及对未经监考的高考的影响

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-05-17 DOI: 10.1111/ijsa.12479

Louis Hickman, Patrick D. Dunlop, Jasper Leo Wolf

{"title":"The performance of large language models on quantitative and verbal ability tests: Initial evidence and implications for unproctored high-stakes testing","authors":"Louis Hickman, Patrick D. Dunlop, Jasper Leo Wolf","doi":"10.1111/ijsa.12479","DOIUrl":"10.1111/ijsa.12479","url":null,"abstract":"Unproctored assessments are widely used in pre-employment assessment. However, widely accessible large language models (LLMs) pose challenges for unproctored personnel assessments, given that applicants may use them to artificially inflate their scores beyond their true abilities. This may be particularly concerning in cognitive ability tests, which are widely used and traditionally considered to be less fakeable by humans than personality tests. Thus, this study compares the performance of LLMs on two common types of cognitive tests: quantitative ability (number series completion) and verbal ability (use a passage of text to determine whether a statement is true). The tests investigated are used in real-world, high-stakes selection. We also examine the performance of the LLMs across different test formats (i.e., open-ended vs. multiple choice). Further, we contrast the performance of two LLMs (Generative Pretrained Transformers, GPT-3.5 and GPT-4) across multiple prompt approaches and “temperature” settings (i.e., a parameter that determines the amount of randomness in the model's output). We found that the LLMs performed well on the verbal ability test but extremely poorly on the quantitative ability test, even when accounting for the test format. GPT-4 outperformed GPT-3.5 across both types of tests. Notably, although prompt approaches and temperature settings did affect LLM test performance, those effects were mostly minor relative to differences across tests and language models. We provide recommendations for securing pre-employment testing against LLM influences. Additionally, we call for rigorous research investigating the prevalence of LLM usage in pre-employment testing as well as on how LLM usage affects selection test validity.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"32 4","pages":"499-511"},"PeriodicalIF":2.6,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.12479","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140964082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Examining the efficacy of inoculation and value-affirmation interventions in improving precandidate reactions among prospective military recruits 研究接种和价值确认干预措施对改善未来新兵候选前反应的效果

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-05-03 DOI: 10.1111/ijsa.12475

Justin R. Feeney, Ben Sylvester, Steve Gooch

{"title":"Examining the efficacy of inoculation and value-affirmation interventions in improving precandidate reactions among prospective military recruits","authors":"Justin R. Feeney, Ben Sylvester, Steve Gooch","doi":"10.1111/ijsa.12475","DOIUrl":"10.1111/ijsa.12475","url":null,"abstract":"This study engaged 4848 first-time, English-speaking prospective Canadian Armed Forces applicants to evaluate pre-application interventions' efficacy on the Practice Canadian Forces Aptitude Test (PCFAT). Using a five-level between-subjects design, participants were randomly assigned to one of the following intervention conditions: inoculation message, value-affirmation message, a combination of both, placebo writing intervention, or a no-intervention control group. The interventions were anchored in inoculation theory and value-affirmation theory and aimed to reduce math anxiety and close the gender gap in test performance. Contrary to expectations, the interventions did not significantly reduce math anxiety or improve problem-solving performance. Consistent with the literature, a negative relationship was found between levels of math anxiety and problem-solving scores, and men outscored women in problem-solving across all conditions. Despite these outcomes, the study lays a foundation for future research on enhancing pre-applicant experiences in an increasingly competitive labor market. Implications and future directions are discussed.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"32 4","pages":"491-498"},"PeriodicalIF":2.6,"publicationDate":"2024-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.12475","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140837435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Promoting equity in hiring: An evaluation of the HireNext Job Posting Assessment 促进招聘公平：对 HireNext 职位发布评估的评价

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-05-02 DOI: 10.1111/ijsa.12477

Sumayya Saleem, Linda White, Michal Perlman, Elizabeth Dhuey

引用次数: 0

Investigating impression management use in asynchronous video interviews across 10 countries 调查印象管理在 10 个国家异步视频访谈中的使用情况

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-04-29 DOI: 10.1111/ijsa.12476

René Arseneault, Nicolas Roulin

{"title":"Investigating impression management use in asynchronous video interviews across 10 countries","authors":"René Arseneault, Nicolas Roulin","doi":"10.1111/ijsa.12476","DOIUrl":"https://doi.org/10.1111/ijsa.12476","url":null,"abstract":"This cross-cultural study investigates how interviewees from 10 culturally-distinct countries differ in their use of impression management (IM) tactics in asynchronous video interviews (AVIs), and the relationship(s) between those tactics and interview performance. A total of 582 participants from ten countries (India, Canada, South Africa, Poland, Spain, Iran, Germany, Chile, Philippines, China) completed an 8-question AVI for a mock position as a manager in a bank. We drew upon GLOBE's cultural framework to predict and explain observed differences in self-reported IM use and performance. We used multi-level modeling to test our hypotheses. Interviewees from our ten countries differed slightly in their IM use for various tactics, but IM use was seldom related to GLOBE cultural dimensions. Partially consistent with previous in-person interview research, honest IM tactics (e.g., self promotion) were positively, but deceptive tactics (e.g., extensive image creation) negatively, associated with interview performance. This research is the first to investigate cross-cultural IM differences in AVIs, thus addressing a critical gap in the selection literature at a time when many organizations conduct interviews virtually to save costs, streamline the hiring process, or simply conduct most of their activities remotely.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"32 3","pages":"461-477"},"PeriodicalIF":2.6,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.12476","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141967618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Rating accuracy, leniency, and rater perceptions when using the RPM and BARS 使用 RPM 和 BARS 时的评分准确性、宽严程度和评分者的看法

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-04-22 DOI: 10.1111/ijsa.12474

Justin R. Feeney, Kabir N. Daljeet, Richard D. Goffin, Travis J. Schneider

引用次数: 0

How do avatar characteristics affect applicants' interactional justice perceptions in artificial intelligence-based job interviews? 在基于人工智能的求职面试中，虚拟化身的特征如何影响求职者的互动公正感？

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-04-12 DOI: 10.1111/ijsa.12472

Qingfei Min, Haoye Sun, Xiaodi Wang, Crystal Zhang

引用次数: 0

Like clockwork: A within-person analysis of time theft in the workplace 像钟表一样工作场所时间盗窃的人际分析

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-04-12 DOI: 10.1111/ijsa.12471

Meagan E. Brock Baskin, James M. Vardaman, Matthew R. Leon

引用次数: 0

The (null) effects of video questions on applicant reactions in asynchronous video interviews: Evidence from an actual hiring context 异步视频面试中视频问题对应聘者反应的（无效）影响：来自实际招聘环境的证据

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-04-01 DOI: 10.1111/ijsa.12470

Nelli Niemitz, Lucas Rietheimer, Cornelius König, Markus Langer

引用次数: 0

How does bias enter the employment interview? Identifying the riskiest applicant characteristics, interviewer characteristics, and sources of potentially biasing information 就业面试中的偏见是如何产生的？识别风险最大的申请人特征、面试官特征以及可能存在偏见的信息来源

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-03-28 DOI: 10.1111/ijsa.12467

Timothy G. Wingate, Sabah Rasheed, Stephen D. Risavy, Chet Robie

引用次数: 0

Playful personnel selection: The use of traditional versus game-related personnel selection methods and their perception from the recruiters’ and applicants’ perspectives 游戏式人才选拔：从招聘人员和求职者的角度看传统与游戏相关人员选拔方法的使用及其影响

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-03-27 DOI: 10.1111/ijsa.12466

Marie L. Ohlms, Klaus G. Melchers, Uwe P. Kanning

引用次数: 0