Imputation of missing aggregate EHR audit log data across individual and multiple organizations

Objective

To compare naive versus machine learning imputation strategies’ efficacy for imputing missing data in EHR-vendor generated data, explore subgrouping criteria, and evaluate performance and feasibility for in-house implementation.

Materials and Methods

Missing data imputation experiments involving various types and sizes of organizations were conducted using physician-only aggregate EHR audit log data. Organizations were categorized by teaching status. Based on the coefficient of variation and missing percentage, variables were classified into three categories before imputation. The model with the highest R²-value was selected as the most robust option.

Results

Teaching and non-teaching organizations showed similar R² trends in model selection, though some differences existed within each class. Moreover, the rolling average provided more consistent R² results across various organization sizes, especially for medium and small-sized organizations. XGBoost performed slightly better in large organizations than in small organizations. Comparisons between single- and multi-site organizations revealed higher R²-values for single organizations using their own data for imputation as opposed to merging.

Discussion/Conclusion:

The study introduced a systematic method for classifying variables and determining the best imputation strategy for each class. It also tested the scalability of this approach for individual organizations. Organizations can effectively use this method, including variable classification and tailored imputation methods. Organization size did not significantly affect the imputation process. The rolling average time-series method outperformed the machine learning method, which used non-time-series approaches. Combining data from diverse sites does not necessarily improve machine learning imputation.