A Nonparametric Data-Fusion Approach for Identification and Estimation of Nonignorable Missing Data With Shadow Variable.

Abstract

Missing data can pose fundamental challenges to statistical inference, with nonignorable missing or missing not at random (MNAR) presenting the most severe methodological difficulties. Despite substantial advances in MNAR inference methods, several limitations remain. These generally include interpretability issues, non-identifiability, unverifiable parametric assumptions, and reliance on external information for which clear practical guidance is often lacking-particularly in sensitivity and Bayesian methods. In this paper, motivated by a real-world COVID-19 dataset from the Centers for Disease Control and Prevention (CDC), we study an MNAR scenario in the presence of a shadow variable that may itself also be subject to MNAR. A shadow variable is associated with the primary variable that is MNAR, but is conditionally independent of that variable's missingness given the primary variable and other covariates. Under the pattern mixture model framework, we propose a nonparametric inference method that can leverage external data to explicitly address the identification and estimation problems in the primary data. The method requires inference only of an observed data density and an odds of missing function, with the latter forming the core of our approach. We further provide two multiple-imputation-based estimation strategies, enhancing transparency and interpretability. The proposed framework can accommodate both covariate MNAR and outcome MNAR settings, as well as different variable types, and extends the existing shadow variable paradigm for MNAR data by relaxing the common assumption that the shadow variable must be fully observed. We evaluate the proposed method through simulation studies and apply it to CDC COVID-19 surveillance data.