Portfolio management based on a reinforcement learning framework

Abstract

Portfolio management is crucial for investors. We propose a dynamic portfolio management framework based on reinforcement learning using the proximal policy optimization algorithm. The two-part framework includes a feature extraction network and a full connected network. First, the majority of the previous research on portfolio management based on reinforcement learning has been dedicated to discrete action spaces. We propose a potential solution to the problem of a continuous action space with a constraint (i.e., the sum of the portfolio weights is equal to 1). Second, we explore different feature extraction networks (i.e., convolutional neural network [CNN], long short-term memory [LSTM] network, and convolutional LSTM network) combined with our system, and we conduct extensive experiments on the six kinds of assets, including 16 features. The empirical results show that the CNN performs best in the test set. Last, we discuss the effect of the trading frequency on our trading system and find that the monthly trading frequency has a higher Sharpe ratio in the test set than other trading frequencies.