Markov Chains

Abstract

• Markov chains are random, memoryless stochastic processes with a discrete state space. In essence, it is a system which changes states based on given probabilities, andthose probabilities depend only on the current state, not past states. • For example, you’re shopping at three stores. Each hour, you can decide to stay at the store, or go to a new one. If you are at any store, there is a 20% chance you stay at the store, or a 40% chance you change stores.