Improved User Identification through Calibrated Monte-Carlo Dropout

Abstract

This paper presents an enhanced approach to user identification using smartphone and wearable sensor data. Our methodology involves segmenting input data and independently analyzing subsequences with CNNs. During testing, we apply calibrated Monte-Carlo Dropout to measure prediction uncertainty. By leveraging the weights obtained from uncertainty quantification, we integrate the results through weighted averaging, thereby improving overall identification accuracy. The main motivation behind this paper is the need to calibrate the CNN for improved weighted averaging. It has been observed that incorrect predictions often receive high confidence, while correct predictions are assigned lower confidence. To tackle this issue, we have implemented the Ensemble of Near Isotonic Regression (ENIR) as an advanced calibration technique. This ensures that certainty scores more accurately reflect the true likelihood of correctness. Furthermore, our experiment shows that calibrating CNN reduces the need for Monte Carlo samples in uncertainty quantification, thereby reducing computational costs. Our thorough evaluation and comparison of different calibration methods have shown improved accuracy in user identification across multiple datasets. Our results showed notable performance improvements when compared to the latest models available. In particular, our approach achieved better results than DB2 by 1.12% and HAR by 0.3% in accuracy.