Discrete-time Markov chain: transition kernel, Chapman–Kolmogorov, irreducibility, recurrence, stationary distribution. Bridge to MCMC and stochastic processes.
graph TD
D1["Def: Markov chain\nP(Xₙ₊₁|X₀,...,Xₙ)=P(Xₙ₊₁|Xₙ)"]
D2["Def: Transition matrix\nP(i,j)=P(X₁=j|X₀=i)"]
D3["Def: Chapman–Kolmogorov\nPⁿ⁺ᵐ=PⁿPᵐ"]
D4["Def: Irreducible\nall states communicate"]
D5["Def: Stationary dist\nπP=π"]
T1["Thm: Chapman–Kolmogorov\nfollows from Markov prop"]
T2["Thm: Perron–Frobenius\nirred aperiodic ⇒ unique π"]
T3["Thm: LLN for MC\n1/n Σf(Xₖ)→π(f) a.s."]
T4["Thm: Recurrence criterion\n∑Pⁿ(i,i)=∞"]
T5["Thm: Foster–Lyapunov\ndrift for stability"]
D1 --> D2
D2 --> D3
D1 --> T1
D2 --> T1
D3 --> T1
D4 --> T2
D5 --> T2
T2 --> T3
D4 --> T4
T4 --> T5
classDef definition fill:#b197fc,color:#fff
classDef theorem fill:#51cf66,color:#fff
class D1,D2,D3,D4,D5 definition
class T1,T2,T3,T4,T5 theorem
Process Statistics
- Nodes: 15
- Edges: 14
- Definitions: 5
- Theorems: 5
Frontier: MCMC (Gibbs, Metropolis–Hastings), geometric ergodicity, non-reversible chains, quasi-stationary distributions. math.PR