Enhancing bone-conducted speech through a pre-trained transformer with low-rank driven sparsity bootstrapping

Abstract

The traditional Transformer architecture encounters substantial challenges in terms of time complexity when dealing with long sequences. Sequential signals, such as speech and serialized image data, inherently exhibit low-rank properties along the temporal axis. By leveraging this low-rank nature effectively, we can not only prune redundant information to enhance model robustness but also devise a sparsity-bootstrapped attention mechanism that significantly reduces the temporal complexity of Transformer-based models. This study is dedicated to applying a self-supervised, pre-trained model that leverages low-rank driven sparsity bootstrapping to enhance bone-conducted speech and address the challenge of scarce paired speech data. This innovative technique enables communication in noisy environments by directly capturing signals from the human skull or larynx. In our experiments, we benchmark our model against five other state-of-the-art recovery models using a comprehensive set of evaluation criteria. Both objective metrics and subjective assessments consistently demonstrate the superiority of our proposed model, indicating its potential to advance bone-conducted speech enhancement technologies.