The MTIST platform: a microbiome time series inference standardized test

Abstract The human gut microbiome is a promising therapeutic target, but interventions are hampered by our limited understanding of microbial ecosystems. Here, we present a platform to develop, evaluate, and score approaches to learn ecological interactions from microbiome time series data. The microbiome time series inference standardized test (MTIST) comprises: a simulation framework for the in silico generation of microbiome study data akin to what is obtained with quantitative next-generation sequencing approaches, a compilation of a large curated data set generated by the simulation framework representing 648 simulated microbiome studies containing 18,360 time series, with a total of 2,182,800 species abundance measurements, and a scoring method to rank ecological inference algorithms. We use the MTIST platform to rank five implementations of microbiome inference approaches, revealing that while all algorithms performed well on ecosystems with few species (3 and 10), all algorithms failed to infer most interaction in a large ecosystem with 100 member species. However, we do find that the strongest interactions within a large ecosystem are inferred with higher success by all algorithms. Finally, we use the MTIST platform to compare different microbiome study designs, characterizing tradeoffs between samples per subject and number of subjects. Interestingly, we find that when only few samples can be collected per subject, ecological inference is most successful when these samples are collected with highest feasible temporal frequency. Taken together, we provide a computational tool to aid the development of better microbiome ecosystem inference approaches, which will be crucial towards the development of reliable and predictable therapeutic approaches that target the microbiome ecosystem.