The efficacy of propensity score matching for separating selection and measurement effects across different survey modes

IF 1.6 4区数学 Q2 SOCIAL SCIENCES, MATHEMATICAL METHODS

Journal of Survey Statistics and Methodology Pub Date : 2024-04-17 DOI:10.1093/jssam/smae017

Eliud Kibuchi, Patrick Sturgis, Gabriele B. Durrant, Olga Maslovskaya

引用次数: 0

Abstract

Effective evaluation of data quality between data collected in different modes is complicated by the confounding of selection and measurement effects. This study evaluates the utility of propensity score matching (PSM) as a method that has been proposed as a means of removing selection effects across surveys conducted in different modes. Our results show large differences in estimates for the same variables between parallel face-to-face and online surveys, even after matching on standard demographic variables. Moreover, discrepancies in estimates are still present after matching between surveys conducted in the same (online) mode, where differences in measurement properties can be ruled out a priori. Our findings suggest that PSM has substantial limitations as a method for separating measurement and selection differences across modes and should be used only with caution.

查看原文本刊更多论文

倾向得分匹配在不同调查模式中分离选择效应和测量效应的功效

由于选择效应和测量效应的干扰，有效评估不同模式收集的数据之间的数据质量变得复杂。本研究对倾向得分匹配法（PSM）的实用性进行了评估，该方法被认为是消除不同模式调查中选择效应的一种手段。我们的研究结果表明，即使在对标准人口统计学变量进行匹配后，面对面调查和在线调查之间相同变量的估计值仍存在很大差异。此外，在可以先验排除测量属性差异的情况下，以相同（在线）模式进行的调查之间进行匹配后，估计值的差异仍然存在。我们的研究结果表明，PSM 作为一种分离不同模式下测量和选择差异的方法存在很大的局限性，应谨慎使用。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Survey Statistics and Methodology Multiple-

CiteScore

4.30

自引率

9.50%

发文量

期刊介绍： The Journal of Survey Statistics and Methodology, sponsored by AAPOR and the American Statistical Association, began publishing in 2013. Its objective is to publish cutting edge scholarly articles on statistical and methodological issues for sample surveys, censuses, administrative record systems, and other related data. It aims to be the flagship journal for research on survey statistics and methodology. Topics of interest include survey sample design, statistical inference, nonresponse, measurement error, the effects of modes of data collection, paradata and responsive survey design, combining data from multiple sources, record linkage, disclosure limitation, and other issues in survey statistics and methodology. The journal publishes both theoretical and applied papers, provided the theory is motivated by an important applied problem and the applied papers report on research that contributes generalizable knowledge to the field. Review papers are also welcomed. Papers on a broad range of surveys are encouraged, including (but not limited to) surveys concerning business, economics, marketing research, social science, environment, epidemiology, biostatistics and official statistics. The journal has three sections. The Survey Statistics section presents papers on innovative sampling procedures, imputation, weighting, measures of uncertainty, small area inference, new methods of analysis, and other statistical issues related to surveys. The Survey Methodology section presents papers that focus on methodological research, including methodological experiments, methods of data collection and use of paradata. The Applications section contains papers involving innovative applications of methods and providing practical contributions and guidance, and/or significant new findings.