Basic probability theory

Abstract

The discussion provides a refresher of probability theory, in particular with respect to the formulations that build the theoretical language of modern machine learning. Probability theory is the formalism of random numbers, and this chapter outlines what these are and how they are characterized by probability density or probability mass functions. How such functions have traditionally been characterized is covered, and a review of how to work with such mathematical objects such as transforming density functions and how to measure differences between density function is presented. Definitions and basic operations with multiple random variables, including the Bayes law, are covered. The chapter ends with an outline of some important approximation techniques of so-called Monte Carlo methods.