The illusion of validity: how effort inflates the perceived validity of interview questions

IF 4 2区心理学 Q2 MANAGEMENT

European Journal of Work and Organizational Psychology Pub Date : 2022-11-11 DOI:10.1080/1359432X.2022.2144240

Don C. Zhang, E. Kausel

{"title":"The illusion of validity: how effort inflates the perceived validity of interview questions","authors":"Don C. Zhang, E. Kausel","doi":"10.1080/1359432X.2022.2144240","DOIUrl":null,"url":null,"abstract":"ABSTRACT Interviewers are often confident in the validity of their interview questions. What drives this confidence and is it justified? In three studies, we found that question creators judged their own interview questions as more valid than when the same questions are judged by an evaluator. We also found that effort expenditure inflated the perceived validity of interview questions but not question quality. Question creators’ perceptions of validity were primarily driven by their self-confidence, and not the question quality. As an intervention, we nudged participants into holding more favourable attitudes towards better questions (i.e., structured questions) by allowing them to choose a subset of them from a pre-written list. Together, we found that while effort expenditure was responsible for the illusion of validity when evaluating unstructured (i.e., low-quality) questions, the same mechanism could also be used to improve interviewers’ acceptance of structured questions. Implications for structured interviews and the scientist-practitioner gap are discussed.","PeriodicalId":48240,"journal":{"name":"European Journal of Work and Organizational Psychology","volume":"32 1","pages":"256 - 271"},"PeriodicalIF":4.0000,"publicationDate":"2022-11-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Work and Organizational Psychology","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1080/1359432X.2022.2144240","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MANAGEMENT","Score":null,"Total":0}

引用次数: 0

Abstract

ABSTRACT Interviewers are often confident in the validity of their interview questions. What drives this confidence and is it justified? In three studies, we found that question creators judged their own interview questions as more valid than when the same questions are judged by an evaluator. We also found that effort expenditure inflated the perceived validity of interview questions but not question quality. Question creators’ perceptions of validity were primarily driven by their self-confidence, and not the question quality. As an intervention, we nudged participants into holding more favourable attitudes towards better questions (i.e., structured questions) by allowing them to choose a subset of them from a pre-written list. Together, we found that while effort expenditure was responsible for the illusion of validity when evaluating unstructured (i.e., low-quality) questions, the same mechanism could also be used to improve interviewers’ acceptance of structured questions. Implications for structured interviews and the scientist-practitioner gap are discussed.

查看原文本刊更多论文

有效性错觉：努力如何提高面试问题的有效性

面试官通常对面试问题的有效性很有信心。是什么驱动了这种信心?这种信心是否合理?在三项研究中，我们发现，问题创造者认为他们自己的面试问题比由评估者判断同样的问题更有效。我们还发现，努力支出增加了面试问题的感知效度，但没有增加问题的质量。问题创造者对有效性的认知主要是由他们的自信所驱动的，而不是问题的质量。作为干预，我们通过允许参与者从预先编写的列表中选择其中的一个子集，推动参与者对更好的问题(即结构化问题)持更有利的态度。我们发现，在评估非结构化(即低质量)问题时，虽然努力支出是产生效度错觉的原因，但同样的机制也可以用来提高面试官对结构化问题的接受度。对结构化访谈和科学家-从业者差距的影响进行了讨论。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

European Journal of Work and Organizational Psychology Multiple-

CiteScore

8.00

自引率

2.30%

发文量

期刊介绍： The mission of the European Journal of Work and Organizational Psychology is to promote and support the development of Work and Organizational Psychology by publishing high-quality scientific articles that improve our understanding of phenomena occurring in work and organizational settings. The journal publishes empirical, theoretical, methodological, and review articles that are relevant to real-world situations. The journal has a world-wide authorship, readership and editorial board. Submissions from all around the world are invited.