A note on out-of-sample prediction, marginal effects computations, and temporal testing with random parameters crash-injury severity models

Abstract

Random parameters logit models have become an increasingly popular method to investigate crash-injury severities in recent years. However, there remain potential elements of the approach that need clarification including out-of-sample prediction, the calculation of marginal effects, and temporal instability testing. In this study, four models are considered for comparison: a fixed parameters multinomial logit model; a random parameters logit model; a random parameters logit model with heterogeneity in means; and a random parameters logit model with heterogeneity in means and variances. A full simulation of random parameters is undertaken for out-of-sample injury-severity predictions, and the prediction accuracy of the estimated models was assessed. Results indicate, not surprisingly, that the random parameters logit model with heterogeneity in the means and variances outperformed other models in predictive performance. Following this, two alternative methods for computing marginal effects are considered: one using Monte Carlo simulation and the other using individual estimates of random parameters. The empirical results indicate that both methods produced defensible results since the full distributions of random parameters are considered. Finally, two testing alternatives for temporal instability are evaluated: a global test across all time periods being considered, and a pairwise time-period to time-period comparison. It is shown that the pairwise comparison can provide more detailed insights into possible temporal variability.