A novel data-efficient double deep Q-network framework for intelligent financial portfolio management

Abstract

Navigating the complexities of dynamic and uncertain financial markets demands intelligent systems capable of learning profitable strategies amidst risk and volatility. While Deep Q-Networks (DQN) offer a foundation for such systems, they often suffer from overestimation bias, training instability, and poor generalization in noisy financial environments. To address these challenges, this work introduces Portfolio Double Deep Q-Network (PDQN), a novel architecture inspired by recent advancements in reinforcement learning. PDQN enhances portfolio management by integrating Double Q-Learning to reduce overestimation, alongside Leaky ReLU activation, Xavier initialization, Huber loss, and dropout regularization to improve learning stability and generalization. Unlike prior methods that rely on large datasets and heavy computational infrastructure, PDQN achieves competitive—and often superior—performance using substantially less training data and lightweight infrastructure, making it well-suited for real-world, resource-constrained financial applications. Distinct from conventional approaches, PDQN uses separate networks to adapt portfolio decisions across varying market conditions. Empirical results across multiple market years show that PDQN often outperforms baseline strategies, including classic DQN and Buy-and-Hold, across key metrics such as Sharpe ratio, Sterling ratio, and cumulative return. PDQN—like all data-driven models—exhibits room for improvement under highly irregular or extreme financial scenarios. These observations suggest promising directions for future refinement and increased robustness, without detracting from the model's practical effectiveness and competitive edge.