The Ehrenfest chains, named for Paul Ehrenfest, are simple, discrete models for the exchange of gas molecules between two containers. However, they can be formulated as simple ball and urn models; the balls correspond to the molecules and the urns to the two containers. Thus, suppose that we have two urns, labeled 0 and 1, that contain a total of \( m \in \N_+ \) balls. The state of the system at time \( n \in \N \) is the number of balls in urn 1, which we will denote by \( X_n \). Our stochastic process is \( \bs{X} = (X_0, X_1, X_2, \ldots) \) with state space \( S = \{0, 1, \ldots, m\} \). Of course, the number of balls in urn 0 at time \( n \) is \( m - X_n \).
In the basic Ehrenfest model, at each discrete time unit, independently of the past, a ball is selected at random and moved to the other urn.
\( \bs{X} \) is a discrete-time Markov chain on \( S \) with transition probability matrix \( P \) given by \[ P(x, x - 1) = \frac{x}{m}, \; P(x, x + 1) = \frac{m - x}{m}, \quad x \in S \]
We will give a construction of the chain from a more basic process. Let \( V_n \) be the ball selected at time \( n \in \N_+ \). Thus \( \bs{V} = (V_1, V_2, \ldots) \) is a sequence of independent random variables, each uniformly distributed on \( \{1, 2, \ldots, m\} \). Let \( X_0 \in S \) be independent of \( \bs{V} \). (We can start the process any way that we like.) Now define the state process recursively as follows: \[ X_{n+1} = \begin{cases} X_n - 1, & V_{n+1} \le X_n \\ X_n + 1, & V_{n+1} \gt X_n \end{cases}, \quad n \in \N \]
In the Ehrenfest experiment, select the basic model. For selected values of \( m \) and selected values of the initial state, run the chain for 1000 time steps and note the limiting behavior of the proportion of time spent in each state.
Suppose now that we modify the basic Ehrenfest model as follows: at each discrete time, independently of the past, we select a ball at random and a urn at random. We then put the chosen ball in the chosen urn.
\( \bs{X} \) is a discrete-time Markov chain on \( S \) with the transition probability matrix \( Q \) given by \[ Q(x, x - 1) = \frac{x}{2 m}, \; Q(x, x) = \frac{1}{2}, \; Q(x, x + 1) = \frac{m - x}{2 m}, \quad x \in S \]
Again, we can construct the chain from a more basic process. Let \( X_0 \) and \( \bs{V} \) be as in Theorem 1. Let \( U_n \) be the urn selected at time \( n \in \N_+ \). Thus \( \bs{U} = (U_1, U_2, \ldots) \) is a sequence of independent random variables, each uniformly distributed on \( \{0, 1\} \) (so that \( \bs{U} \) is a fair, Bernoulli trials sequence). Also, \( \bs{U} \) is independent of \( \bs{V} \) and \( X_0 \). Now define the state process recursively as follows: \[ X_{n+1} = \begin{cases} X_n - 1, & V_{n+1} \le X_n, \; U_{n+1} = 0 \\ X_n + 1, & V_{n+1} \gt X_n, \; U_{n+1} = 1 \\ X_n, & \text{otherwise} \end{cases}, \quad n \in \N \]
Note that \( Q(x, y) = \frac{1}{2} P(x, y) \) for \( y \in \{x - 1, x + 1\} \).
In the Ehrenfest experiment, select the modified model. For selected values of \( m \) and selected values of the initial state, run the chain for 1000 time steps and note the limiting behavior of the proportion of time spent in each state.
The basic and modified Ehrenfest chains are irreducible and positive recurrent.
The chains are clearly irreducible since every state leads to every other state. It follows that the chains are positive recurrent since the state space \( S \) is finite.
The basic Ehrenfest chain is periodic with period 2. The cyclic classes are the set of even states and the set of odd states. The two-step transition matrix is
\[ P^2(x, x - 2) = \frac{x (x - 1)}{m^2}, \; P^2(x, x) = \frac{x(m - x + 1) + (m - x)(x + 1)}{m^2}, \; P^2(x, x + 2) = \frac{(m - x)(m - x - 1)}{m^2}, \quad x \in S \]The modified Ehrenfest chain is aperiodic.
For the basic and modified Ehrenfest chains, the invariant distribution is the binomial distribution with trial parameter \( m \) and success parameter \( \frac{1}{2} \). So the invariant probability density function \( f \) is given by \[ f(x) = \binom{m}{x} \left( \frac{1}{2} \right)^m, \quad x \in S \]
For the basic chain we have \begin{align*} (f P)(y) & = f(y - 1) P(y - 1, y) + f(y + 1) P(y + 1, y)\\ & = \binom{m}{y - 1} \left(\frac{1}{2}\right)^m \frac{m - y + 1}{m} + \binom{m}{y + 1} \left(\frac{1}{2}\right)^m \frac{y + 1}{m} \\ & = \left(\frac{1}{2}\right)^m \left[\binom{m - 1}{y - 1} + \binom{m - 1}{y}\right] = \left(\frac{1}{2}\right)^m \binom{m}{y} = f(y), \quad y \in S \end{align*} The last step uses a fundamental identity for binomial coefficients. For the modified chain we can use the result for the basic chain: \begin{align*} (f Q)(y) & = f(y - 1) Q(y - 1, y) + f(y) Q(y, y) + f(y + 1)Q(y + 1, y) \\ & = \frac{1}{2} f(y - 1) P(y - 1, y) + \frac{1}{2} f(y + 1) P(y + 1, y) + \frac{1}{2} f(y) = f(y), \quad y \in S \end{align*}
Thus, the invariant distribution corresponds to placing each ball randomly and independently either in urn 0 or in urn 1.
The mean return time to state \( x \in S \) for the basic or modified Ehrenfest chain is \( \mu(x) = 2^m \big/ \binom{m}{x} \).
For the basic Ehrenfest chain, the limiting behavior of the chain is as follows:
For the modified Ehrenfest chain, \( Q^n(x, y) \to \binom{m}{y} \left(\frac{1}{2}\right)^m \) as \( n \to \infty \) for \( x, \, y \in S \).
In the Ehrenfest experiment, the limiting binomial distribution is shown graphically and numerically. For each model and for selected values of \( m \) and selected values of the initial state, run the chain for 1000 time steps and note the limiting behavior of the proportion of time spent in each state. How do the choices of \( m \), the initial state, and the model seem to affect the rate of convergence to the limiting distribution?
The basic and modified Ehrenfest chains are reversible.
Let \( g(x) = \binom{m}{x} \) for \( x \in S \). The crucial observations are \( g(x) P(x, y) = g(y) P(y, x)\) and \(g(x) Q(x, y) = g(y) Q(y, x)\) for all \( x, \, y \in S \). For the basic chain, if \( x \in S \) then \begin{align*} g(x) P(x, x - 1) & = g(x - 1) P(x - 1, x) = \binom{m - 1}{x - 1} \\ g(x) P(x, x + 1) & = g(x + 1) P(x + 1, x) = \binom{m - 1}{x} \end{align*} In all other cases, \( g(x) P(x, y) = g(y) P(y, x) = 0 \). The reversibility condition for the modified chain follows trivially from that of the basic chain since \( Q(x, y) = \frac{1}{2} P(x, y) \) for \( y = x \pm 1 \) (and of course the reversibility condition is trivially satisfied when \( x = y \)). Note that the invariant PDF \( f \) is simply \( g \) normalized. The reversibility condition gives another (and better) proof that \( f \) is invariant.
Run the simulation of the Ehrenfest experiment 10,000 time steps for each model, for selected values of \( m \), and with initial state 0. Note that at first, you can see the arrow of time
. After a long period, however, the direction of time is no longer evident.
Consider the basic Ehrenfest chain with \( m = 5 \) balls, and suppose that \( X_0 \) has the uniform distribution on \( S \).
Consider the modified Ehrenfest chain with \( m = 5 \) balls, and suppose that the chain starts in state 2 (with probability 1).