Variable Length Joint Source-Channel Coding of Text Using Deep Neural Networks

Abstract

We consider joint source and channel coding of natural language over a noisy channel using deep learning. While the typical approach based on separate source and channel code design minimizes bit error rates, the proposed deep learning approach preserves semantic information of sentences. In particular, unlike previous work which used a fixed-length encoding per sentence, a variable-length neural network encoder is presented. The performance of this new architecture is compared to the one with fixed-length encoding per sentence. We show that the variable-length encoder has a lower word error rate compared with the fixed-length encoder as well as separate source and channel coding schemes across several different communication channels.