Token-Prediction-Based Post-Processing for Low-Bitrate Speech Coding

Abstract

Low-bitrate speech coding plays an essential role in speech transmission and storage. However, speech quality degrades noticeably at low bitrates with current coding methods. Therefore, this letter proposes a novel Token-Prediction-based Post-Processing (T3P) model to improve the quality of low-bitrate coded speech. Unlike existing post-processing methods, T3P is a discrete-domain method centered on the prediction and classification of discrete tokens. Specifically, given low-bitrate coded speech features as condition, T3P initiates from a random token and sequentially predicts the token sequences produced by a residual vector quantization (RVQ) based neural codec, which is subsequently decoded to reconstruct the raw speech. Experiments confirm that T3P surpasses flow-matching-based and speech-enhancement-based baselines, achieving a better trade-off between speech quality and efficiency. Empowered by T3P, Encodec achieves performance at just 0.5 kbps that exceeds its original 4 kbps results for 16 kHz speech coding.