AI’s predictable memory in financial analysis

Abstract

Look-ahead bias in Large Language Models (LLMs) arises when information that would not have been available at the time of prediction is included in the training data and inflates prediction performance. This paper proposes a practical methodology to quantify look-ahead bias in financial applications. By prompting LLMs to retrieve historical stock returns without context, we construct a proxy to estimate memorization-driven predictability. We show that the bias varies predictably with data frequency, model size, and aggregation level: smaller models and finer data granularity exhibit negligible bias. Our results help researchers navigate the trade-off between statistical power and bias in LLMs.