Informed machine learning to reconcile interpretability with fidelity in scientific applications

Abstract

Notwithstanding their impressive performances, unfortunately some of the most powerful machine learning (ML) models are obscure and almost impossible to interpret. Consequently, in the last years, there has been a rapid increase in research about eXplainable Artificial Intelligence, whose objective consists of improving their transparency. In scientific applications, explainability assumes a different flavour and cannot be reduced to pure user understanding but there is a premium also on fidelity, on developing models that reflect the actual mechanisms at play in the investigated phenomena. To this end, Genetic Programming supported Symbolic Regression (GPSR), conceived explicitly to manipulate symbols, can present various competitive advantages in finding a good trade-off between interpretability and realism. However, the search spaces are typically too large and the algorithms have to be steered to converge on the desired solutions. The present work describes techniques to constrain GPSR and to combine it with deep learning tools, so that the final models are expressed in terms of interpretable and realistic mathematical equations. The strategies to guide convergence include dimensional analysis, integration of prior information about symmetries and conservation laws, refinements of the fitness function and robust statistics. The performances are improved according to all the main metrics: accuracy, robustness against noise and outliers, capability of handling data sparsity and interpretability. Great attention has been paid to introducing practical solutions, covering most essential aspects of the data analysis process, from the treatment of the uncertainties to the quantification of the equations’ complexity. All the main applications of supervised ML, from regression to classification, are considered (and the extension to unsupervised and reinforcement learning are not expected to pose major difficulties). Theoretical considerations, systematic numerical tests, simulations with multiphysics codes and the results of actual experiments prove the potential of the proposed improvements.