Nathaniel K Brown, Travis Gagie, Massimiliano Rossi
{"title":"RLBWT Tricks.","authors":"Nathaniel K Brown, Travis Gagie, Massimiliano Rossi","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Until recently, most experts would probably have agreed we cannot backwards-step in constant time with a run-length compressed Burrows-Wheeler Transform (RLBWT), since doing so relies on rank queries on sparse bitvectors and those inherit lower bounds from predecessor queries. At ICALP '21, however, Nishimoto and Tabei described a new, simple and constant-time implementation. For a permutation <math><mi>π</mi></math> , it stores an <math><mrow><mi>O</mi> <mfenced><mi>r</mi></mfenced> </mrow> </math> -space table - where <math><mi>r</mi></math> is the number of positions <math><mi>i</mi></math> where either <math><mrow><mi>i</mi> <mo>=</mo> <mn>0</mn></mrow> </math> or <math><mrow><mi>π</mi> <mfenced><mrow><mi>i</mi> <mo>+</mo> <mn>1</mn></mrow> </mfenced> <mo>≠</mo> <mi>π</mi> <mfenced><mi>i</mi></mfenced> <mo>+</mo> <mn>1</mn></mrow> </math> - that enables the computation of successive values of <math><mrow><mi>π</mi> <mfenced><mi>i</mi></mfenced> </mrow> </math> by table look-ups and linear scans. Nishimoto and Tabei showed how to increase the number of rows in the table to bound the length of the linear scans such that the query time for computing <math><mrow><mi>π</mi> <mfenced><mi>i</mi></mfenced> </mrow> </math> is constant while maintaining <math><mrow><mi>O</mi> <mfenced><mi>r</mi></mfenced> </mrow> </math> -space. In this paper we refine Nishimoto and Tabei's approach, including a time-space tradeoff, and experimentally evaluate different implementations demonstrating the practicality of part of their result. We show that even without adding rows to the table, in practice we almost always scan only a few entries during queries. We propose a decomposition scheme of the permutation <math><mi>π</mi></math> corresponding to the LF-mapping that allows an improved compression of the data structure, while limiting the query time. We tested our implementation on real-world genomic datasets and found that without compression of the table, backward-stepping is drastically faster than with sparse bitvector implementations but, unfortunately, also uses drastically more space. After compression, backward-stepping is competitive both in time and space with the best existing implementations.</p>","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"233 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11327918/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142000813","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Brandon Jew, Jiajin Li, Sriram Sankararaman, Jae Hoon Sul
{"title":"An efficient linear mixed model framework for meta-analytic association studies across multiple contexts.","authors":"Brandon Jew, Jiajin Li, Sriram Sankararaman, Jae Hoon Sul","doi":"10.4230/LIPIcs.WABI.2021.10","DOIUrl":"10.4230/LIPIcs.WABI.2021.10","url":null,"abstract":"<p><p>Linear mixed models (LMMs) can be applied in the meta-analyses of responses from individuals across multiple contexts, increasing power to detect associations while accounting for confounding effects arising from within-individual variation. However, traditional approaches to fitting these models can be computationally intractable. Here, we describe an efficient and exact method for fitting a multiple-context linear mixed model. Whereas existing exact methods may be cubic in their time complexity with respect to the number of individuals, our approach for multiple-context LMMs (mcLMM) is linear. These improvements allow for large-scale analyses requiring computing time and memory magnitudes of order less than existing methods. As examples, we apply our approach to identify expression quantitative trait loci from large-scale gene expression data measured across multiple tissues as well as joint analyses of multiple phenotypes in genome-wide association studies at biobank scale.</p>","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"2016 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8323485/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"39265573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fast and Space-Efficient Construction of AVL Grammars from the LZ77 Parsing.","authors":"Dominik Kempa, Ben Langmead","doi":"10.4230/LIPIcs.ESA.2021.56","DOIUrl":"https://doi.org/10.4230/LIPIcs.ESA.2021.56","url":null,"abstract":"<p><p>Grammar compression is, next to Lempel-Ziv (LZ77) and run-length Burrows-Wheeler transform (RLBWT), one of the most flexible approaches to representing and processing highly compressible strings. The main idea is to represent a text as a context-free grammar whose language is precisely the input string. This is called a straight-line grammar (SLG). An AVL grammar, proposed by Rytter [Theor. Comput. Sci., 2003] is a type of SLG that additionally satisfies the AVL property: the heights of parse trees for children of every nonterminal differ by at most one. In contrast to other SLG constructions, AVL grammars can be constructed from the LZ77 parsing in compressed time: <math><mrow><mi>𝓞</mi><mrow><mo>(</mo><mi>z</mi><mrow><mi>log</mi></mrow><mi>n</mi></mrow><mo>)</mo></mrow></math> where <math><mrow><mi>z</mi></mrow></math> is the size of the LZ77 parsing and <math><mrow><mi>n</mi></mrow></math> is the length of the input text. Despite these advantages, AVL grammars are thought to be too large to be practical. We present a new technique for rapidly constructing a small AVL grammar from an LZ77 or LZ77-like parse. Our algorithm produces grammars that are always at least five times smaller than those produced by the original algorithm, and usually not more than double the size of grammars produced by the practical Re-Pair compressor [Larsson and Moffat, Proc. IEEE, 2000]. Our algorithm also achieves low peak RAM usage. By combining this algorithm with recent advances in approximating the LZ77 parsing, we show that our method has the potential to construct a run-length BWT in about one third of the time and peak RAM required by other approaches. Overall, we show that AVL grammars are surprisingly practical, opening the door to much faster construction of key compressed data structures.</p>","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"204 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10613516/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71427454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The theory of monads","authors":"N. Jolley","doi":"10.4324/9780429422775-4","DOIUrl":"https://doi.org/10.4324/9780429422775-4","url":null,"abstract":"","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81114878","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Legacy and influence","authors":"N. Jolley","doi":"10.4324/9780429422775-10","DOIUrl":"https://doi.org/10.4324/9780429422775-10","url":null,"abstract":"","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"09 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86350450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Leibniz: life and works","authors":"N. Jolley","doi":"10.4324/9780429422775-2","DOIUrl":"https://doi.org/10.4324/9780429422775-2","url":null,"abstract":"","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"85 6 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79746090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The metaphysics of substances: unity and activity","authors":"N. Jolley","doi":"10.4324/9780429422775-3","DOIUrl":"https://doi.org/10.4324/9780429422775-3","url":null,"abstract":"","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"59 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83584184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Natural and philosophical language","authors":"N. Jolley","doi":"10.4324/9780429422775-6","DOIUrl":"https://doi.org/10.4324/9780429422775-6","url":null,"abstract":"","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73067108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Human and divine freedom","authors":"N. Jolley","doi":"10.4324/9780429422775-7","DOIUrl":"https://doi.org/10.4324/9780429422775-7","url":null,"abstract":"","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"34 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90991959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Mind, knowledge, and ideas","authors":"N. Jolley","doi":"10.4324/9780429422775-5","DOIUrl":"https://doi.org/10.4324/9780429422775-5","url":null,"abstract":"","PeriodicalId":30209,"journal":{"name":"Leibniz International Proceedings in Informatics","volume":"33 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77275794","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}