Limitations of Current Machine-Learning Models in Predicting Enzymatic Functions for Uncharacterized Proteins.

Abstract

Thirty to seventy percent of proteins in any given genome have no assigned function and have been labeled as the protein "unknome". This large knowledge shortfall is one of the final frontiers of biology. Machine-Learning (ML) approaches are enticing, with early successes demonstrating the ability to propagate functional knowledge from experimentally characterized proteins. An open question is the ability of machine-learning approaches to predict enzymatic functions unseen in the training sets. By integrating literature and a combination of bioinformatic approaches, we evaluated individually Enzyme Commission number predictions for over 450 Escherichia coli unknowns made using state-of-the-art machine-learning approaches. We found that current ML methods not only mostly fail to make novel predictions but also make basic logic errors in their predictions that human annotators avoid by leveraging the available knowledge base. This underscores the need to include assessments of prediction uncertainty in model output and to test for 'hallucinations' (logic failures) as a part of model evaluation. Explainable AI (XAI) analysis can be used to identify indicators of prediction errors, potentially identifying the most relevant data to include in the next generation of computational models.