A hybrid machine learning and Extended Kalman filtering framework for sensor fusion of low-cost and high-precision dissolved oxygen sensors under environmental variability

Abstract

Low-cost optical (fluorescence) dissolved oxygen (DO) sensors exhibit significant nonlinear biases that vary with temperature and pressure, thereby limiting their standalone accuracy. This work proposes a hybrid framework that integrates Gaussian Process Regression (GPR) for probabilistic bias correction with an Extended Kalman Filter (EKF) for accurate real-time DO estimation using low-cost optical DO sensors. The framework was validated using 3,788 laboratory samples under varied temperature, pressure, and DO conditions. Performance evaluation on an independent 994-sample test dataset against a high-precision reference sensor yielded an RMSE of 0.162 mg/L (RRMSE ≈ 2.1%) and an MAE of 0.101 mg/L, corresponding to an 84.5% reduction in RMSE relative to raw low-cost sensor readings. Statistical consistency was confirmed with a mean NEES of 0.96 and white innovations. The proposed approach effectively tracked rapid deoxygenation transients and executed in < 4 ms per update, enabling real-time embedded deployment for scalable, near-reference-grade DO monitoring with reliable uncertainty awareness.