A unified approach to extract interpretable rules from tree ensembles via Integer Programming

Abstract

Tree ensembles are widely used machine learning models, known for their effectiveness in supervised classification and regression tasks. Their performance derives from aggregating predictions of multiple decision trees, which are renowned for their interpretability properties. However, tree ensemble models do not reliably exhibit interpretable output. Our work aims to extract an optimized list of rules from a trained tree ensemble, providing the user with a condensed, interpretable model that retains most of the predictive power of the full model. Our approach consists of solving a set partitioning problem formulated through Integer Programming. The extracted list of rules is unweighted and defines a partition of the training data, assigning each instance to exactly one rule, and thereby simplifying the explanation process. The proposed method works with tabular or time series data, for both classification and regression tasks, and its flexible formulation can include any arbitrary loss or regularization functions. Our computational experiments offer statistically significant evidence that our method performs comparably to several rule extraction methods in terms of predictive performance and fidelity towards the tree ensemble. Moreover, we empirically show that the proposed method effectively extracts interpretable rules from tree ensembles that are designed for time series data.