Diego Díaz-Domínguez, Travis Gagie, Veronica Guerrini, Ben Langmead, Zsuzsanna Lipták, Giovanni Manzini, Francesco Masillo, Vikram Shivakumar
{"title":"Prefix-free parsing for merging big BWTs.","authors":"Diego Díaz-Domínguez, Travis Gagie, Veronica Guerrini, Ben Langmead, Zsuzsanna Lipták, Giovanni Manzini, Francesco Masillo, Vikram Shivakumar","doi":"10.1007/978-3-032-05228-5_6","DOIUrl":"10.1007/978-3-032-05228-5_6","url":null,"abstract":"<p><p>When building Burrows-Wheeler Transforms (BWTs) of truly huge datasets, prefix-free parsing (PFP) can use an unreasonable amount of memory. In this paper we show how if a dataset can be broken down into small datasets that are not very similar to each other - such as collections of many copies of genomes of each of several species, or collections of many copies of each of the human chromosomes - then we can drastically reduce PFP's memory footprint by building the BWTs of the small datasets and then merging them into the BWT of the whole dataset.</p>","PeriodicalId":520001,"journal":{"name":"International Symposium on String Processing and Information Retrieval : SPIRE ... : proceedings. SPIRE (Symposium)","volume":"16073 ","pages":"54-63"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12700358/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145759334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nathaniel K Brown, Lore Depuydt, Mohsen Zakeri, Anas Alhadi, Nour Allam, Dove Begleiter, Nithin Bharathi Kabilan Karpagavalli, Suchith Sridhar Khajjayam, Hamza Wahed, Travis Gagie, Ben Langmead
{"title":"KeBaB: <i>k</i>-mer based breaking for finding long MEMs.","authors":"Nathaniel K Brown, Lore Depuydt, Mohsen Zakeri, Anas Alhadi, Nour Allam, Dove Begleiter, Nithin Bharathi Kabilan Karpagavalli, Suchith Sridhar Khajjayam, Hamza Wahed, Travis Gagie, Ben Langmead","doi":"10.1007/978-3-032-05228-5_2","DOIUrl":"10.1007/978-3-032-05228-5_2","url":null,"abstract":"<p><p>Long maximal exact matches (MEMs) are used in many genomics applications such as read classification and sequence alignment. Li's ropebwt3 finds long MEMs quickly because it can often ignore much of its input, skipping matching steps which are redundant to the final output. In this paper we propose KeBaB, a fast and space efficient <i>k</i>-mer filtration step using a Bloom filter. This approach speeds up MEM-finders such as ropebwt3 even further by letting them ignore even more, breaking the input into substrings called \"pseudo-MEMs\" which are guaranteed to contain all long MEMs. We also show experimentally that KeBaB can accelerate metagenomic classification without significantly reducing accuracy, either by finding all long MEMs or by leveraging the filter to find only the long MEMs present in the <i>t</i> longest pseudo-MEMs.</p>","PeriodicalId":520001,"journal":{"name":"International Symposium on String Processing and Information Retrieval : SPIRE ... : proceedings. SPIRE (Symposium)","volume":"16073 ","pages":"10-17"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12700359/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145759320","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Data Structures for SMEM-Finding in the PBWT.","authors":"Paola Bonizzoni, Christina Boucher, Davide Cozzi, Travis Gagie, Dominik Köppl, Massimiliano Rossi","doi":"10.1007/978-3-031-43980-3_8","DOIUrl":"10.1007/978-3-031-43980-3_8","url":null,"abstract":"<p><p>The positional Burrows-Wheeler Transform (PBWT) was presented as a means to find set-maximal exact matches (SMEMs) in haplotype data via the computation of the divergence array. Although run-length encoding the PBWT has been previously considered, storing the divergence array along with the PBWT in a compressed manner has not been as rigorously studied. We define two queries that can be used in combination to compute SMEMs, allowing us to define smaller data structures that support one or both of these queries. We combine these data structures, enabling the PBWT and the divergence array to be stored in a manner that allows for finding SMEMs. We estimate and compare the memory usage of these data structures, leading to one data structure that is most memory efficient. Lastly, we implement this data structure and compare its performance to prior methods using various datasets taken from the 1000 Genomes Project data.</p>","PeriodicalId":520001,"journal":{"name":"International Symposium on String Processing and Information Retrieval : SPIRE ... : proceedings. SPIRE (Symposium)","volume":"14240 ","pages":"89-101"},"PeriodicalIF":0.0,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11325217/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141989992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Space-time Trade-offs for the LCP Array of Wheeler DFAs.","authors":"Nicola Cotumaccio, Travis Gagie, Dominik Köppl, Nicola Prezza","doi":"10.1007/978-3-031-43980-3_12","DOIUrl":"10.1007/978-3-031-43980-3_12","url":null,"abstract":"<p><p>Recently, Conte et al. generalized the longest-common prefix (LCP) array from strings to Wheeler DFAs, and they showed that it can be used to efficiently determine matching statistics on a Wheeler DFA [DCC 2023]. However, storing the LCP array requires <math><mrow><mi>O</mi> <mfenced><mrow><mi>n</mi> <mi>log</mi> <mi>n</mi></mrow> </mfenced> </mrow> </math> bits, <math><mi>n</mi></math> being the number of states, while the compact representation of Wheeler DFAs often requires much less space. In particular, the BOSS representation of a de Bruijn graph only requires a linear number of bits, if the size of alphabet is constant. In this paper, we propose a sampling technique that allows to access an entry of the LCP array in logarithmic time by only storing a linear number of bits. We use our technique to provide a space-time tradeoff to compute matching statistics on a Wheeler DFA. In addition, we show that by augmenting the BOSS representation of a <math><mi>k</mi></math> -th order de Bruijn graph with a linear number of bits we can navigate the underlying variable-order de Bruijn graph in time logarithmic in <math><mi>k</mi></math> , thus improving a previous bound by Boucher et al. which was linear in <math><mi>k</mi></math> [DCC 2015].</p>","PeriodicalId":520001,"journal":{"name":"International Symposium on String Processing and Information Retrieval : SPIRE ... : proceedings. SPIRE (Symposium)","volume":"14240 ","pages":"143-156"},"PeriodicalIF":0.0,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11301794/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141899308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A simple grammar-based index for finding approximately longest common substrings.","authors":"Travis Gagie, Sana Kashgouli, Gonzalo Navarro","doi":"10.1007/978-3-031-43980-3_19","DOIUrl":"10.1007/978-3-031-43980-3_19","url":null,"abstract":"<p><p>We show how, given positive constants <math><mi>ϵ</mi></math> and <math><mi>δ</mi></math> , and an <math><mi>α</mi></math> -balanced straight-line program with <math><mi>g</mi></math> rules for a text <math><mi>T</mi> <mo>[</mo> <mn>1</mn> <mo>.</mo> <mo>.</mo> <mi>n</mi> <mo>]</mo></math> , we can build an <math><mi>O</mi> <mo>(</mo> <mi>g</mi> <mo>)</mo></math> -space index that, given a pattern <math><mi>P</mi> <mo>[</mo> <mn>1</mn> <mo>.</mo> <mo>.</mo> <mi>m</mi> <mo>]</mo></math> , in <math><mi>O</mi> <mfenced><mrow><mi>m</mi> <mspace></mspace> <msup><mrow><mtext>log</mtext></mrow> <mrow><mi>δ</mi></mrow> </msup> <mspace></mspace> <mi>g</mi></mrow> </mfenced> </math> time finds w.h.p. a substring of <math><mi>P</mi></math> that occurs in <math><mi>T</mi></math> and whose length is at least a ( <math><mn>1</mn> <mo>-</mo> <mi>ϵ</mi></math> ) fraction of the longest common substring of <math><mi>P</mi></math> and <math><mi>T</mi></math> . The correctness can be ensured within the same expected query time.</p>","PeriodicalId":520001,"journal":{"name":"International Symposium on String Processing and Information Retrieval : SPIRE ... : proceedings. SPIRE (Symposium)","volume":"14240 ","pages":"246-252"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12700360/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145759341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"<ArticleTitle xmlns:ns0=\"http://www.w3.org/1998/Math/MathML\">KATKA: A KRAKEN-like tool with <ns0:math><ns0:mi>k</ns0:mi></ns0:math> given at query time.","authors":"Travis Gagie, Sana Kashgouli, Ben Langmead","doi":"10.1007/978-3-031-20643-6_14","DOIUrl":"10.1007/978-3-031-20643-6_14","url":null,"abstract":"<p><p>We describe a new tool, KATKA, that stores a phylogenetic tree <math><mi>T</mi></math> such that later, given a pattern <math><mi>P</mi> <mo>[</mo> <mn>1</mn> <mo>.</mo> <mo>.</mo> <mi>m</mi> <mo>]</mo></math> and an integer <math><mi>k</mi></math> , it can quickly return the root of the smallest subtree of <math><mi>T</mi></math> containing all the genomes in which the <i>k</i>-mer <math><mi>P</mi> <mo>[</mo> <mi>i</mi> <mo>.</mo> <mo>.</mo> <mi>i</mi> <mo>+</mo> <mi>k</mi> <mo>-</mo> <mn>1</mn> <mo>]</mo></math> occurs, for <math><mn>1</mn> <mo>≤</mo> <mi>i</mi> <mo>≤</mo> <mi>m</mi> <mo>-</mo> <mi>k</mi> <mo>+</mo> <mn>1</mn></math> . This is similar to KRAKEN's functionality but with <math><mi>k</mi></math> given at query time instead of at construction time.</p>","PeriodicalId":520001,"journal":{"name":"International Symposium on String Processing and Information Retrieval : SPIRE ... : proceedings. SPIRE (Symposium)","volume":"13617 ","pages":"191-197"},"PeriodicalIF":0.0,"publicationDate":"2022-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11288484/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141857590","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}