Smart Choices and the Selection Monad

Abstract

Describing systems in terms of choices and their resulting costs and rewards promises to free algorithm designers and programmers from specifying how to make those choices. In implementations, the choices can be realized by optimization or machine-learning methods.We study this approach from a programming-language perspective. We define a small language that supports decision-making abstraction, rewards, and probabilities. We give a globally optimizing operational semantics, and, using the selection monad for decision-making, three denotational semantics with auxiliary monads for reward and probability; the three model various correlations between returned values and expected rewards. We show the two kinds of semantics coincide by proving adequacy theorems; we show that observational equivalence is characterized by semantic equality (at basic types) by proving full abstraction theorems; and we discuss program equations.