Compression of Next Generation Sequencing Data

Abstract

Summary form only given. FASTQ is the defacto standard for data from next generation sequencing platforms. The FASTQ format uses four lines per read: two lines for header information, one for the sequence itself, and one for the quality scores. The proposed compression scheme treats the various lines of each four line set differently. The highly repetitive headers are encoded using an LZ77 variant. The reads themselves are compressed using a modified LZ78 method which uses a backward adaptive dictionary. The quality factors are encoded using a context based arithmetic coding scheme. Performance results for the proposed method were obtained using data generated via a sequencing simulation of a random 50kbp section from Escherichia coli str. K-12 substr. DH10B chromosome with 35X and 100X coverages. The two best performing methods for FASTQ compression, Fastqz and Fqzcomp were used to compress the same data. These methods performed the best in a competition to compress next generation sequencing data. We also compare the results to two general purpose compressors bzip and LZMA. The results are shown in Table.