Online Learning with Universal Model and Predictor Classes

Abstract

We review and relate some classical and recent results from the theory of online learning based on discrete classes of models or predictors. Among these frameworks, Bayesian methods, MDL, and prediction (or action) with expert advice are studied. We will discuss ways to work with universal base classes corresponding to sets of all programs on some fixed universal Turing machine, resulting in universal induction schemes.