SMDRL: Self-supervised mobile device representation learning framework for recording source identification from unlabeled data

Abstract

In mobile recording device source identification, deep learning techniques have been pivotal for extracting deep features from audio signals. Traditional approaches, however, predominantly rely on fully labeled datasets for supervised training, neglecting the vast amounts of unlabeled data typically present in real-world scenarios. This limitation has led to a significant disparity between the performance of existing models and their practical applicability. To bridge this gap, we introduce the Self-supervised Mobile Device Representation Learning (SMDRL) framework. During the pre-training phase, SMDRL utilizes a substantial unlabeled audio dataset enhanced by three novel data augmentation techniques: interpolative noise mixing, time-frequency masking, and partitioned resampling. Employing contrastive learning, these methods facilitate the development of a universal encoder capable of effectively capturing device-specific features from raw audio. Further, we propose the Cross-scale Mobile Device Encoder (CMDEncoder), which integrates a Convolutional Neural Network (CNN) for local feature extraction with Long Short-Term Memory (LSTM) and Transformer-Encoder architectures to handle global-scale information. This encoder is further refined by numerical computations of standard deviations and mean values to enhance global feature interaction. In the fine-tuning phase, the framework employs the Light Weight Enhanced Channel Attention, Propagation in Time-Delay Neural Network (LWECAP-TDNN) classifier, achieving high-precision identification results. Our experimental outcomes demonstrate that the proposed method significantly elevates the accuracy of device source identification, achieving recognition rates of 95.24% and 89.87% on the CCNU Mobile Base and CCNU Mobile Large datasets, respectively. These results represent improvements ranging from 1.44% to 35.69% over baseline methods. To facilitate further research, the source code for this study is made publicly available at https://github.com/CCNUZFW/SMDRL.