MetaKSSD: boosting the scalability of the reference taxonomic marker database and the performance of metagenomic profiling using sketch operations.

The performance of metagenomic profiling is constrained by the diversity of taxa present in the reference taxonomic marker database (MarkerDB) used. However, continually updating MarkerDB to include newly determined taxa using existing approaches faces increasing difficulties and will soon become impractical. Here we introduce MetaKSSD, which redefines MarkerDB construction and metagenomic profiling using sketch operations, enhancing MarkerDB scalability and profiling performance. MetaKSSD encompasses 85,202 species in its MarkerDB using just 0.17 GB of storage and profiles 10 GB of data within seconds. Leveraging its comprehensive MarkerDB, MetaKSSD substantially improves profiling results. In a microbiome-phenotype association study, MetaKSSD identified more effective associations than MetaPhlAn4. We profiled 382,016 metagenomic runs using MetaKSSD, conducted extensive sample clustering analyses and suggested potential yet-to-be-discovered niches. MetaKSSD offers functionality for instantaneous searching of similar profiles. It enables the swift transmission of metagenome sketches over the network and real-time online metagenomic analysis, facilitating use by non-expert users.