1. Random
  2. 0
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. 10
  13. 11
  14. 12
  15. 13
  16. 14
  17. 15
  18. 16
  19. 17

15. Markov Processes

Summary

A Markov process is a random process in which the future is independent of the past, given the present. Thus, Markov processes are the natural stochastic analogs of the deterministic processes described by differential and difference equations. They form one of the most important classes of random processes.

General Theory

  1. Introduction
  2. Potentials and Generators

Discrete-Time Markov Chains

  1. Introduction
  2. Recurrence and Transience
  3. Periodicity
  4. Stationary and Limiting Distributions
  5. Time Reversal

Special Discrete-Time Chains

  1. The Ehrenfest Chains
  2. The Bernoulli-Laplace Chain
  3. Reliability Chains
  4. The Branching Chain
  5. Queuing Chains
  6. Birth Death Chains
  7. Random Walks on Graphs

Continuous-Time Markov Chains

  1. Introduction
  2. Transition Matrices and Generators
  3. Potential Matrices
  4. Stationary and Limiting Distributions
  5. Time Reversal

Special Continuous-Time Chains

  1. Chains Subordinate to the Poisson Process
  2. Birth-Death Chains
  3. Queuing Chains
  4. Branching Chains

Apps

Sources and Resources

Quote