Generalized Word Equations: A New Approach to Data Compresion

Let Σ be an alphabet. A generalized word equation, GWE for short, is a set of triples and pairs. A triple is in form (p, q, l) where p, q, l are positive integers. A pair is in form (a, i) where a ∊ S and i is a positive integer. A solution of a word equation e is any word w such that, for each triple (p, q, l) in e, w[p..p + l − 1] = w[q..q + l − 1] and, for each pair (a, i) in e, w[i] = a. If there is only one shortest solution w of e, then we say that e defines w. Observe here that if e defines w, then the solution set of e is {ws : s ∊ Σ*}. The triples and pairs of an equation e are called constraints. If an equation e defines a word w, we say that e is a compressed representation of w. Let G be a GWE with m triples and pairs defining a word w. There is an algorithm reconstructing w from G in O(m+ |w|) worst case time [1]. Therefore decompression is optimal. It is not difficult to prove that in simple modifications of GWE generalize LZ77, LZ78 and LZW algorithms. We consider a natural variant of GWE called pGWE and prove that, for a word w, it is a little more efficent and more general than LZ77 for a reversed word wR. Moreover, it can be proved that GWE approach generalizes the bidirectional scheme. We compared GWE with Straight Line Programs (SLP for short) [2, 3] and prove that if SLP for a word w is of length n, then there is a GWE defining w with n constraints. We are not aware of any reasonable simulation in the other direction. We propose a variant of GWE which compresses an input word w in O(|w|L2) worse case time where L is the longest repeating factor in w. This version was tested on files in Canterbury Corpus. It gives better results than gzip on text files and slightly worse on the other files. It is worth mentioning here that gzip is a result of 20 years studies on LZ77 so it is unfair to compare it with our approach. Our current best approach is significantly worse than bzip2 which is based on the Burrows-Wheeler transform.