Markov Chain Basics
A Markov chain models transitions between states where the next state depends only on the current state. The steady state distribution represents the long-run proportion of time spent in each state.
Transition matrix P: P[i][j] = probability of going from state i to state j
N-step matrix: P^n gives n-step transition probabilities
Steady state pi: pi * P = pi, sum(pi) = 1