A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling

Abstract

Although evidence integration to the boundary model has successfully explained a wide range of behavioral and neural data in decision making under uncertainty, how animals learn and optimize the boundary remains unresolved. Here, we propose a model-free reinforcement learning algorithm for perceptual decisions under uncertainty that implements a sequential sampling process with an implicit decision boundary. Our model learns whether to commit to a decision given the available evidence or continue sampling information at a cost. We reproduced the canonical features of perceptual decision making such as dependence of accuracy and reaction time on evidence strength, modulation of speed-accuracy trade-off by payoff regime, and many others. By unifying learning and decision making within the same framework, this model can inspire a new look at the mechanisms of flexibility in context-dependent changes of behavior.