On k-Mer-Based and Maximum Likelihood Estimation Algorithms for Trace Reconstruction

Abstract

The goal of the trace reconstruction problem is to recover a string $\mathbf {x}\in \{0,1\}^{n}$ given many independent traces of x, where a trace is a subsequence obtained from deleting bits of x independently with some given probability $p\in [0,1$ ). A recent result of Chase (STOC 2021) shows how x can be determined (in exponential time) from $\exp ({O}(n^{1/5})\log ^{5} n)$ traces. This is the state-of-the-art result on the sample complexity of trace reconstruction. In this paper we consider two kinds of algorithms for the trace reconstruction problem. We first observe that the bound of Chase, which is based on statistics of arbitrary length-k subsequences, can also be obtained by considering the “k-mer statistics”, i.e., statistics regarding occurrences of contiguous k-bit strings (a.k.a, k-mers) in the initial string x, for $k = 2n^{1/5}$ . Mazooji and Shomorony (arXiv.2210.10917) show that such statistics (called k-mer density map) can be estimated within $\varepsilon $ accuracy from $ {\mathrm {poly}} (n, 2^{k}, 1/ {\varepsilon })$ traces. We call an algorithm to be k-mer-based if it reconstructs x given estimates of the k-mer density map. Such algorithms essentially capture all the analyses in the worst-case and smoothed-complexity models of the trace reconstruction problem we know of so far. Our first, and technically more involved, result shows that any k-mer-based algorithm for trace reconstruction must use $\exp (\Omega (n^{1/5} \sqrt {\log n}))$ traces, thus establishing the optimality of this number of traces. The analysis of this result also shows that the analysis technique used by Chase (STOC 2021) is essentially tight, and hence new techniques are needed in order to improve the worst-case upper bound. This result is shown by considering an appropriate class of real polynomials, that have been previously studied in the context of trace estimation (De, O’Donnell, Servedio. Annals of Probability 2019; Nazarov, Peres. STOC 2017), and proving that two of these polynomials are very close to each other on an arc in the complex plane. Our proof of the proximity of such polynomials uses new technical ingredients that allow us to focus on just a few coefficients of these polynomials. Our second, simple, result considers the performance of the Maximum Likelihood Estimator (MLE), which specifically picks the source string that has the maximum likelihood to generate the samples (traces). We show that the MLE algorithm uses a nearly optimal number of traces, i.e., up to a factor of n in the number of samples needed for an optimal algorithm, and show that this factor of n loss may be necessary under general “model estimation” settings.