“I Have Never Seen a Bad Backtest”: Modeling Reality in Quantitative Investing

Abstract

Backtests often are discounted based on the expected failure between live and simulated (theoretical) trading. These failures are associated with the modeling assumptions that address market conditions. Deviations between live trading and strategy model performance will be biased downward by poor backtesting methodologies that do not account for model uncertainty and trading cost assumptions. However, model reality can be improved through explicitly accounting for all trading costs, endogenizing costs as part of the overall backtesting methodology, and forming systematic backtesting methodologies that account for sources of randomness. This article presents a framework for assessing backtested performance that can be employed by both investors and managers as a checklist for improving model reality, reducing trading cost biases, and accounting for uncertainty.