Reinforcement Learning using Reward Expectations in Scenarios with Aleatoric Uncertainties

In scenarios with aleatoric uncertainties, the reward got by an agent when executing the same action in the same state is random, which can reduce the stability and convergence speed of the reinforcement algorithms. However, in most scenarios, reward functions have regularity, and their expectations are determined, which can be got through models or sample statistics. This paper discusses the distribution relationship between reward functions and value functions in scenarios with aleatoric uncertainties and proves the feasibility of using reward expectations for reinforcement learning. Finally, experiments show that algorithms have better stability and convergence speed when using reward expectations than random rewards.