Vib2audio: Robust Sound Recovery With Microvibration Sensing via mmWave Radar

The popularity of smartphones presents a potential risk to personal privacy, particularly through covert eavesdropping by external nonacoustic sensors (e.g., high-speed camera and radio frequency (RF) sensor), where RF-based voice perception technology shows great potential due to its insensitivity to the environment and its ability to traverse occlusions. In this article, we introduce Vib2audio, a high-quality audio recovery system that uses RF signals to recover audio from the tiny vibration of sound source. We discover that the extracted phase spectrum lacks components in the high-frequency band. Therefore, we investigate the frequency detectable range of mmWave radar used in our scenario and develop an improved spectrum reconstruction network based on the pix2pix structure to accurately recover the time-frequency spectrum of audio from the limited frequency band of the vibration signal. In addition, we synthesize the audio waveform of voice from the time-frequency spectrum using a Griffin-Lim vocoder. Our extensive experiments indicate that Vib2audio can robustly recover tiny sound source vibration to recognizable high-quality human voice audio from tiny vibration. In our overall objective and subjective performance evaluation, the reconstructed audio by Vib2audio had a mean absolute error (MAE) of 3.6% in the non-intrusive speech quality assessment (NISQA) and achieved a subjective identifiability assessment score of 3.48. Furthermore, the quantitative evaluation results show that Vib2audio is effective and robust to background noise and multisource interference.