RUMBoost: Gradient boosted random utility models

Abstract

This paper introduces the RUMBoost model, a novel discrete choice modelling approach that combines the interpretability and behavioural robustness of Random Utility Models (RUMs) with the generalisation and predictive ability of tree-based ensemble methods. We obtain the full functional form of non-linear utility specifications by replacing each linear parameter in the utility functions of a RUM with an ensemble of gradient boosted regression trees. We introduce additional constraints on the ensembles to ensure three crucial features of the utility specifications: (i) dependency of the utilities of each alternative on only the attributes of that alternative, (ii) monotonicity of marginal utilities, and (iii) an intrinsically interpretable functional form, where the exact response of the model is known throughout the entire input space. Furthermore, we introduce an optimisation-based smoothing technique that replaces the piece-wise constant utility values of alternative attributes with monotonic piece-wise cubic splines to identify non-linear parameters with defined gradient. We demonstrate the potential of RUMBoost compared to various benchmark ML and Random Utility models for revealed and stated preference mode choice data as well as a semi-synthetic example. The results highlight both the great predictive performance and the direct interpretability of our proposed approach, allowing for the identification of complex behaviours associated with different alternatives. The smoothed attribute utility functions allow for the calculation of various behavioural indicators such as the Value of Time (VoT) and marginal utilities. Finally, we demonstrate the flexibility of our methodology by showing how the RUMBoost model can be extended to complex model specifications, including attribute interactions, correlation within alternative error terms (Nested Logit model) and heterogeneity within the population (Mixed Logit model).