Policy Gradient for Ratio Optimization: A Case Study

Abstract

We consider policy gradient methods for ratio optimization problems by way of an illustrative case study: maximizing the Omega ratio of a financial portfolio. We propose a general framework for ratio optimization in sequential decision-making problems, explore the notion of hidden quasiconcavity in such problems, and propose an actor-critic algorithm for the Omega ratio problem. Our central contribution is to show that the algorithm converges almost surely to (a neighborhood of) a global optimum and to demonstrate its performance in practice.