Conditional Entropies of k-Deletion/Insertion Channels

The channel output entropy of a transmitted sequence is the entropy of the possible channel outputs, and similarly, the channel input entropy of a received sequence is the entropy of all possible transmitted sequences. The goal of this work is to study these entropy values for the k-deletion and k-insertion channels, where exactly k symbols are deleted or inserted in the transmitted sequence, respectively. If all possible sequences are transmitted with the same probability, then studying the input and output entropies becomes equivalent. For both the 1-deletion and 1-insertion channels, it is shown that among all sequences with a fixed number of runs, the input entropy is minimized for sequences with a skewed distribution of run lengths, and it is maximized for sequences with a balanced distribution of run lengths. Among our results, we establish a conjecture by Atashpendar et al., which claims that for the 1-deletion channel, the input entropy is maximized by the alternating sequences among all binary sequences. This conjecture is also verified for the 2-deletion channel, where it is proved that sequences with a single run minimize the input entropy.