International Journal of Selection and Assessment最新文献_第2页

All Your Base Are Belong to Us: The Urgent Reality of Unproctored Testing in the Age of LLMs 你所有的基础都属于我们：法学硕士时代无监考考试的紧迫现实

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-03-04 DOI: 10.1111/ijsa.70005

Louis Hickman

{"title":"All Your Base Are Belong to Us: The Urgent Reality of Unproctored Testing in the Age of LLMs","authors":"Louis Hickman","doi":"10.1111/ijsa.70005","DOIUrl":"https://doi.org/10.1111/ijsa.70005","url":null,"abstract":"The release of new generative artificial intelligence (AI) tools, including new large language models (LLMs), continues at a rapid pace. Upon the release of OpenAI's new o1 models, I reconducted Hickman et al.'s (2024) analyses examining how well LLMs perform on a quantitative ability (number series) test. GPT-4 scored below the 20th percentile (compared to thousands of human test takers), but o1 scored at the 95th percentile. In response to these updated findings and Lievens and Dunlop's (2025) article about the effects of LLMs on the validity of pre-employment assessments, I make an urgent call to action for selection and assessment researchers and practitioners. A recent survey suggests that a large proportion of applicants are already using generative AI tools to complete high-stakes assessments, and it seems that no current assessments will be safe for long. Thus, I offer possibilities for the future of testing, detail their benefits and drawbacks, and provide recommendations. These possibilities are: increased use of proctoring, adding strict time limits, using LLM detection software, using think-aloud (or similar) protocols, collecting and analyzing trace data, emphasizing samples over signs, and redesigning assessments to allow LLM use during completion. Several of these possibilities inspire future research to modernize assessment. Future research should seek to improve our understanding of how to design valid assessments that allow LLM use, how to effectively use trace test-taker data, and whether think-aloud protocols can help differentiate experts and novices.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"33 2","pages":""},"PeriodicalIF":2.6,"publicationDate":"2025-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.70005","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143533255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Social Desirability Tendency in Personality-Based Job Interviews—A Question of Interview Format? 个性求职面试中的社会可取倾向——一个面试形式问题？

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-03-04 DOI: 10.1111/ijsa.70006

Valerie Schröder, Anna Luca Heimann, Pia Ingold, Nicolas Roulin, Marianne Schmid Mast, Manuel Bachmann, Martin Kleinmann

引用次数: 0

Attitudes Toward Cybervetting in Germany: Impact on Organizational Attractiveness Depends on Social Media Platform 德国人对网络审查的态度：社交媒体平台对组织吸引力的影响

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-02-17 DOI: 10.1111/ijsa.70003

Philipp Schäpers, Franz W. Mönke, Chiara-Maria Frieler, Nicolas Roulin, Johannes Basch

{"title":"Attitudes Toward Cybervetting in Germany: Impact on Organizational Attractiveness Depends on Social Media Platform","authors":"Philipp Schäpers, Franz W. Mönke, Chiara-Maria Frieler, Nicolas Roulin, Johannes Basch","doi":"10.1111/ijsa.70003","DOIUrl":"https://doi.org/10.1111/ijsa.70003","url":null,"abstract":"Cybervetting, assessing social media in personnel selection, is widely used. However, individuals concerned often perceive this practice negatively. We propose that attitudes toward cybervetting may depend on the platform used and the cultural context. Thus, we transfer the attitudes toward cybervetting scale to a context with strict data regulations: Germany. In an online between-subjects experiment with platform users and non-users (N = 100 working professionals and students), we examined attitudes toward cybervetting on different social media platforms (professional: LinkedIn vs. personal: Instagram) and their relationship with organizational attractiveness. We found that German participants viewed cybervetting on professional platforms with more skepticism than American participants. Hierarchical regression analysis revealed higher perceived fairness, lower invasion of privacy, and higher organizational attractiveness when cybervetting was done on professional platforms.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"33 1","pages":""},"PeriodicalIF":2.6,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.70003","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143424083","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Why Participant Perceptions of Assessment Center Exercises Matter: Justice, Motivation, Self-Efficacy, and Performance 为什么参与者对评估中心练习的感知很重要：公正、动机、自我效能和绩效

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-02-04 DOI: 10.1111/ijsa.70002

Sylvia G. Roch, Kathryn Devon

引用次数: 0

Are Games Always Fun and Fair? A Comparison of Reactions to Different Game-Based Assessments 游戏总是有趣和公平的吗？对不同游戏评估的反应比较

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-01-27 DOI: 10.1111/ijsa.12520

Marie Luise Ohlms, Klaus G. Melchers

{"title":"Are Games Always Fun and Fair? A Comparison of Reactions to Different Game-Based Assessments","authors":"Marie Luise Ohlms, Klaus G. Melchers","doi":"10.1111/ijsa.12520","DOIUrl":"https://doi.org/10.1111/ijsa.12520","url":null,"abstract":"Game-based assessment (GBA) has garnered attention in the personnel selection and assessment context owing to its postulated potential to improve applicant reactions. However, GBAs can differ considerably depending on their specific design. Therefore, we sought to determine whether test taker reactions to GBAs vary owing to the different manifestations that GBAs may take on, and to test takers' individual preferences for such assessments. In an experimental study, each of N = 147 participants was shown six different GBAs and asked to rate several applicant reaction variables concerning these assessments. We found that reactions to GBAs were not inherently positive even though GBAs were generally perceived as enjoyable. However, perceptions of fairness and organizational attractiveness varied considerably between GBAs. Participants' age and experience with video games were related to reactions but had less impact than the different GBAs. Our results suggest that a technology-as-designed approach, which considers GBAs as a combination of multiple components (e.g., game elements), is crucial in GBA research to provide generalizable results for theory and practice.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"33 1","pages":""},"PeriodicalIF":2.6,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.12520","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143119821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Comparing Proctored and Unproctored Cognitive Ability Testing in High-Stakes Personnel Selection 高风险人才选拔中有监督与无监督的认知能力测试比较

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-01-27 DOI: 10.1111/ijsa.70001

Tore Nøttestad Norrøne, Morten Nordmo

{"title":"Comparing Proctored and Unproctored Cognitive Ability Testing in High-Stakes Personnel Selection","authors":"Tore Nøttestad Norrøne, Morten Nordmo","doi":"10.1111/ijsa.70001","DOIUrl":"https://doi.org/10.1111/ijsa.70001","url":null,"abstract":"<div>\u0000 \u0000 New advances in computerized adaptive testing (CAT) have increased the feasibility of high-stakes unproctored testing of general mental ability (GMA) in personnel selection contexts. This study presents the results from a within-subject investigation of the convergent validity of unproctored tests. Three batteries of cognitive ability tests were administered during personnel selection in the Norwegian Armed Forces. A total of 537 candidates completed two sets of proctored fixed-length GMA tests before and during the selection process. In addition, an at-home unproctored CAT battery of tests was administered before the selection process began. Differences and similarities between the convergent validity of the tests were evaluated. The convergent validity coefficients did not significantly differ between proctored and unproctored batteries, both on observed GMA scores and the latent factor level. The distribution and standardized residuals of test scores comparing proctored-proctored and proctored-unproctored were overall quite similar and showed no evidence of score inflation or deflation in the unproctored tests. The similarities between proctored and unproctored results also extended to the moderately searchable words similarity test. Although some unlikely individual cases were observed, the overall results suggest that the unproctored tests maintained their convergent validity.</div>","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"33 1","pages":""},"PeriodicalIF":2.6,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143119822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Meta-Analysis of Accent Bias in Employee Interviews: The Effects of Gender and Accent Stereotypes, Interview Modality, and Other Moderating Features 员工面试中口音偏见的元分析：性别、口音刻板印象、面试方式和其他调节特征的影响

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-01-23 DOI: 10.1111/ijsa.12519

Henri T. Maindidze, Jason G. Randall, Michelle P. Martin-Raugh, Katrisha M. Smith

{"title":"A Meta-Analysis of Accent Bias in Employee Interviews: The Effects of Gender and Accent Stereotypes, Interview Modality, and Other Moderating Features","authors":"Henri T. Maindidze, Jason G. Randall, Michelle P. Martin-Raugh, Katrisha M. Smith","doi":"10.1111/ijsa.12519","DOIUrl":"https://doi.org/10.1111/ijsa.12519","url":null,"abstract":"To address concerns of subtle discrimination against stigmatized groups, we meta-analyze the magnitude and moderators of bias against non-standard accents in employment interview evaluations. Results from a multi-level random-effects meta-analysis (unique effects: k = 41, N = 7,596; multi-level effects accounting for dependencies: k = 120, N = 20,873) demonstrate that standard-accented (SA) interviewees are consistently favored over non-standard-accented (NSA) interviewees (d = 0.46). Accent bias is stronger against women compared to men, particularly when evaluator samples are predominantly female, and was strongly predicted by interviewers' stereotypes of NSA interviewees as less competent and, to a lesser extent, as less warm. Accent bias was not significantly impacted by perceptions of comprehensibility, accentedness, accent type, interview modality, study rigor, or job speaking skill requirements.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"33 1","pages":""},"PeriodicalIF":2.6,"publicationDate":"2025-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.12519","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143118622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Toward Theory-Based Volitional Personality Development Interventions at Work 基于理论的意志人格发展干预研究

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-01-19 DOI: 10.1111/ijsa.70000

Sofie Dupré, Bart Wille

{"title":"Toward Theory-Based Volitional Personality Development Interventions at Work","authors":"Sofie Dupré, Bart Wille","doi":"10.1111/ijsa.70000","DOIUrl":"https://doi.org/10.1111/ijsa.70000","url":null,"abstract":"<div>\u0000 \u0000 In this article, we respond to four commentaries (Li et al., 2024; Hennecke & Ingold, 2025; Perossa & Connelly, 2024; Ones et al., 2024) on our article “Personality development goals at work: A new frontier in personality assessment in organizations.” We start by addressing four overarching considerations from the commentaries, including (a) how to approach PDG assessment, (b) the feasibility of personality development interventions, (c) potential trade-offs involved, and (d) the value of personality development beyond established HR practices. Next, in an attempt to integrate these considerations and stimulate future research in this area, we outline three critical elements of what we believe can be the foundation of theory-based personality development interventions at work. For this purpose, we first posit that personality development at work can be rethought such that the focus shifts from “changing an employee's trait levels” to “expanding that employee's comfort zone across a range of personality states.” Second, to have sustained effects, interventions need to accomplish more than simply “learning new behaviors,” by effectively targeting all layers of personality—behavioral, cognitive, and emotional. Finally, we introduce optimal functioning, encompassing both performance and well-being aspects, as the ultimate criterion for evaluating the success of personality development interventions. We hope these reactions and integrative ideas will inspire future research on personality development goals assessment and personality development interventions in the work context.</div>","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"33 1","pages":""},"PeriodicalIF":2.6,"publicationDate":"2025-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143116278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating the Impact of Faking on the Criterion-Related Validity of Personality Assessments 评估作假对人格评估标准效度的影响

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2025-01-06 DOI: 10.1111/ijsa.12518

Andrew B. Speer, Angie Y. Delacruz, Takudzwa Chawota, Lauren J. Wegmeyer, Andrew P. Tenbrink, Carter Gibson, Chris Frost

{"title":"Evaluating the Impact of Faking on the Criterion-Related Validity of Personality Assessments","authors":"Andrew B. Speer, Angie Y. Delacruz, Takudzwa Chawota, Lauren J. Wegmeyer, Andrew P. Tenbrink, Carter Gibson, Chris Frost","doi":"10.1111/ijsa.12518","DOIUrl":"https://doi.org/10.1111/ijsa.12518","url":null,"abstract":"Personality assessments are commonly used in hiring, but concerns about faking have raised doubts about their effectiveness. Qualitative reviews show mixed and inconsistent impacts of faking on criterion-related validity. To address this, a series of meta-analyses were conducted using matched samples of honest and motivated respondents (i.e., instructed to fake, applicants). In 80 paired samples, the average difference in validity coefficients between honest and motivated samples across five-factor model traits ranged from 0.05 to 0.08 (largest for conscientiousness and emotional stability), with the validity ratio ranging from 64% to 72%. Validity was attenuated when candidates faked regardless of sample type, trait relevance, or the importance of impression management, though variation existed across criterion types. Both real applicant samples (k = 25) and instructed response conditions (k = 55) showed a reduction in validity across honest and motivated conditions, including when managerial ratings of job performance were the criterion. Thus, faking impacted the validity in operational samples. This suggests that practitioners should be cautious relying upon concurrent validation evidence (for personality inventories) and expect attenuated validity in operational applicant settings, particularly for conscientiousness and emotional stability scales. That said, it is important to highlight that personality assessments generally maintained useful validity even under-motivated conditions.","PeriodicalId":51465,"journal":{"name":"International Journal of Selection and Assessment","volume":"33 1","pages":""},"PeriodicalIF":2.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ijsa.12518","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143112513","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Attention To Detail and Cyber Skill: Associated Beyond General Intelligence in Cyber-Soldier Conscripts 对细节的关注和网络技能：与网络士兵征召的一般情报相关

IF 2.6 4区管理学

International Journal of Selection and Assessment Pub Date : 2024-12-23 DOI: 10.1111/ijsa.12517

Pär-Anders Albinsson, Patrik Lif

引用次数: 0