\(\newcommand{\R}{\mathbb{R}}\) \(\newcommand{\N}{\mathbb{N}}\) \(\newcommand{\E}{\mathbb{E}}\) \(\newcommand{\P}{\mathbb{P}}\) \(\newcommand{\var}{\text{var}}\) \(\newcommand{\sd}{\text{sd}}\) \(\newcommand{\cov}{\text{cov}}\) \(\newcommand{\cor}{\text{cor}}\) \(\newcommand{\skw}{\text{skew}}\) \(\newcommand{\kur}{\text{kurt}}\) \(\newcommand{\sgn}{\,\text{sgn}}\) \(\newcommand{\bs}{\boldsymbol}\)
  1. Random
  2. 4. Special Distributions
  3. Infinitely Divisible Distributions

Infinitely Divisible Distributions

This section discusses a theoretical topic that you may want to skip if you are a new student of probability.

Basic Theory

Infinitely divisible distributions form an important class of distributions on \( \R \) that includes the stable distributions, the compound Poisson distributions, as well as several of the most important special parametric families of distribtions. Basically, the distribution of a real-valued random variable is infinitely divisible if for each \( n \in \N_+ \), the variable can be decomposed into the sum of \( n \) independent copies of another variable. Here is the precise definition.

The distribution of a real-valued random variable \( X \) is infinitely divisible if for every \( n \in \N_+ \), there exists a sequence of independent, identically distributed variables \( (X_1, X_2, \ldots, X_n) \) such that \( X_1 + X_2 + \cdots + X_n \) has the same distribution as \( X \).

If the distribution of \( X \) is stable then the distribution is infinitely divisible.

Details:

Let \( n \in \N_+ \) and let \( (X_1, X_2, \ldots, X_n) \) be a sequence of independent variables, each with the same distribution as \( X \). By the definition of stability, there exists \( a_n \in \R \) and \( b_n \in (0, \infty) \) such that \( \sum_{i=1}^n X_i \) has the same distribution as \( a_n + b_n X \). But then \[ \frac{1}{b_n} \left(\sum_{i=1}^n X_i - a_n\right) = \sum_{i=1}^n \frac{X_i - a_n/n}{b_n} \] has the same distribution as \( X \). But \( \left(\frac{X_i - a_n/n}{b_n}: i \in \{1, 2, \ldots, n\} \right) \) is an IID sequence, and hence the distribution of \( X \) is infinitely divisible.

Suppose now that \( \bs{X} = (X_1, X_2, \ldots) \) is a sequence of independent, identically distributed random variables, and that \( N \) has a Poisson distribution and is independent of \( \bs{X} \). Recall that the distribution of \( \sum_{i=1}^N X_i \) is said to be compound Poisson. Like the stable distributions, the compound Poisson distributions form another important class of infinitely divisible distributions.

Suppose that \( Y \) is a random variable.

  1. If \( Y \) is compound Poisson then \( Y \) is infinitely divisible.
  2. If \( Y \) is infinitely divisible and takes values in \( \N \) then \( Y \) is compound Poisson.
Details:
  1. Suppose that \( Y \) is compound Poisson, so that we can take \( Y = \sum_{i=1}^N X_i\) where \( \bs{X} = (X_1, X_2, \ldots) \) is a sequence of independent, identically distributed random variables with common characteristic function \( \phi \), and where \( N \) is independent of \( \bs{X} \) and has the Poisson distribution with parameter \( \lambda \in (0, \infty) \). The characteristic function \( \chi \) of \( Y \) is given by \( \chi(t) = \exp(\lambda [\phi(t) - 1]) \) for \( t \in \R \). But then for \( n \in \N_+ \), \[ \chi(t) = \left[\exp\left(\frac{\lambda}{n}[\phi(t) - 1]\right)\right]^n, \quad t \in \R \] But \( t \mapsto \exp\left(\frac{\lambda}{n}[\phi(t) - 1]\right) \) is the characteristic function of the compound Poisson distribution corresponding to \( \bs{X} \) but with Poisson parameter \( \lambda / n \). Restated in terms of random variables, \( Y = \sum_{i=1}^n Y_i \) where \( Y_i \) has the compound Poisson distribution corresponding to \( \bs{X} \) with Poisson parameter \( \lambda / n \).
  2. The proof is from An Introduction to Probability Theory and Its Applications by William Feller, and requires some additional notation. Recall that the symbol \( \asymp \) is used to connect functions that are asymptotically the same in the sense that that the ratio converges to 1. Suppose now that \( Y \) takes values in \( \N \) and is infinitely divisible. In this case we can use probability generating functions rather than characteristic functions, so let \( P \) denote the PGF of \( Y \). By definition, \( P(t) = \sum_{k=0}^\infty p_k t^k \) where \( p_k = \P(Y = k) \) for \( k \in \N \). Since \( Y \) is infinitely divisible, \( P^{1/n} \) is also a PGF for every \( n \in \N_+ \), so let \( P^{1/n}(t) = \sum_{k=0}^\infty p_{n k} t^k \) where \( p_{n k} \ge 0 \) for \( k \in \N \) and \( n \in \N_+ \) and \( \sum_{k=0}^\infty p_{n k} = 1 \) for \( n \in \N_+ \). As with all PGFs, the series for \( P(t) \) and for \( P^{1/n}(t) \) converge at least for \( t \in [0, 1] \), and this interval is sufficient for a PGF to completely determine the underlying distribution. For \( n \in \N_+ \), we have \[ \sum_{k=0}^\infty p_k t^k = \left(\sum_{k=0}^\infty p_{n k} t^k\right)^n \] Expanding the series on the right and then equating coefficients of the two series term by term, we see that if \( p_0 = 0 \) then \( p_{n 0} = 0 \) which in turn would imply \( p_1 = \cdots = p_{n-1} = 0 \). Since this is true for all \( n \in \N_+ \), we would have \( P \) identically 0, which is a contradiction. Hence \( p_0 \gt 0 \) and so \( P(t) \gt 0 \) for \( t \in [0, 1] \) and therefore \( \left[P(t) \big/ p_0\right]^{1/n} \to 1 \) as \( n \to \infty \) for \( t \in [0, 1] \). Next recall that \( \ln(1 + x) \asymp x \) as \( x \downarrow 0 \). It follows that for \( t \in [0, 1] \), \[ \ln\left(\left[\frac{P(t)}{p_0}\right]^{1/n}\right) = \ln\left\{1 + \left(\left[\frac{P(t)}{p_0}\right]^{1/n} - 1\right)\right\} \asymp \left[\frac{P(t)}{p_0}\right]^{1/n} - 1 \text{ as } n \to \infty\] As a special case, when \( t = 1 \), we have \(\ln\left[\left(1 / p_0\right)^{1/n}\right] \asymp \left(1 / p_0\right)^{1/n} - 1\) as \( n \to \infty \). Hence using properties of logarithms and a bit of algebra, \[ \frac{\ln[P(t)] - \ln(p_0)}{-\ln(p_0)} = \frac{\ln\left(\left[P(t) \big/ p_0\right]^{1/n}\right)}{\ln\left[\left(1 / p_0\right)^{1/n}\right]} \asymp \frac{P^{1/n}(t) - p_0^{1/n}}{1 - p_0^{1/n}} \text{ as } n \to \infty \] The power series (about 0) for the expression on the right has positive coefficients, and the expression takes the value 1 when \( t = 1 \). Thus, the expression on the right is a PGF for each \( n \in \N_+ \). By the continuity theorem for convergence in distribution, it follows that the left side, which we will denote by \( Q(t) \), is also a PGF. Solving we have \[ P(t) = \exp(\lambda [Q(t) - 1]), \quad t \in [0, 1] \] where \( \lambda = -\ln(p_0) \). This is the PGF of the distribution obtained by compounding the distribution with PGF Q with the Poisson distribution with parameter \( \lambda \).

Special Cases

A number of special distributions are infinitely divisible. Proofs of the results stated below are given in the individual sections.

Stable Distributions

First, the normal distribution, the Cauchy distribution, and the Lévy distribution are stable, so they are infinitely divisible. However, direct arguments give more information, because we can identify the distribution of the component variables.

The normal distribution is infinitely divisible. If \( X \) has the normal distribution with mean \( \mu \in \R \) and standard deviation \( \sigma \in (0, \infty) \), then for \( n \in \N_+ \), \( X \) has the same distribution as \( X_1 + X_2 + \cdots + X_n \) where \( (X_1, X_2, \ldots, X_n) \) are independent, and \( X_i \) has the normal distribution with mean \( \mu/n \) and standard deviation \( \sigma/\sqrt{n} \) for each \( i \in \{1, 2, \ldots, n\} \).

The Cauchy distribution is infinitely divisible. If \( X \) has the Cauchy distribution with location parameter \( a \in \R \) and scale parameter \( b \in (0, \infty) \), then for \( n \in \N_+ \), \( X \) has the same distribution as \( X_1 + X_2 + \cdots + X_n \) where \( (X_1, X_2, \ldots, X_n) \) are independent, and \( X_i \) has the Cauchy distribution with location parameter \( a/n \) and scale parameter \( b/n \) for each \( i \in \{1, 2, \ldots, n\} \).

Other Special Distributions

On the other hand, there are distributions that are infinitely divisible but not stable.

The gamma distribution is infinitely divisible. If \( X \) has the gamma distribution with shape parameter \( k \in (0, \infty) \) and scale parameter \( b \in (0, \infty) \), then for \( n \in \N_+ \), \( X \) has the same distribution as \( X_1 + X_2 + \cdots + X_n \) where \( (X_1, X_2, \ldots, X_n) \) are independent, and \( X_i \) has the gamma distribution with shape parameter \( k / n \) and scale parameter \( b \) for each \( i \in \{1, 2, \ldots, n\} \)

The chi-square distribution is infinitely divisible. If \( X \) has the chi-square distribution with \( k \in (0, \infty) \) degrees of freedom, then for \( n \in \N_+ \), \( X \) has the same distribution as \( X_1 + X_2 + \cdots + X_n \) where \( (X_1, X_2, \ldots, X_n) \) are independent, and \( X_i \) has the chi-square distribution with \( k / n \) degrees of freedom for each \( i \in \{1, 2, \ldots, n\} \).

The Poisson distribution distribution is infinitely divisible. If \( X \) has the Poisson distribution with rate parameter \( \lambda \in (0, \infty) \), then for \( n \in \N_+ \), \( X \) has the same distribution as \( X_1 + X_2 + \cdots + X_n \) where \( (X_1, X_2, \ldots, X_n) \) are independent, and \( X_i \) has the Poisson distribution with rate parameter \( \lambda/n \) for each \( i \in \{1, 2, \ldots, n\} \).

The general negative binomial distribution on \( \N \) is infinitely divisible. If \( X \) has the negative binomial distribution on \( \N \) with parameters \( k \in (0, \infty) \) and \( p \in (0, 1) \), then for \( n \in \N_+ \), \( X \) has the same distribution as \( X_1 + X_2 + \cdots + X_n \) were \( (X_1, X_2, \ldots, X_n) \) are independent, and \( X_i \) has the negative binomial distribution on \( \N \) with parameters \( k / n \) and \( p \) for each \( i \in \{1, 2, \ldots, n\} \).

Since the Poisson distribution and the negative binomial distributions are distributions on \( \N \), it follows from the characterization in that these distributions must be compound Poisson. Of course it is completely trivial that the Poisson distribution is compound Poisson, but it's far from obvious that the negative binomial distribution has this property. It turns out that the negative binomial distribution can be obtained by compounding the logarithmic series distribution with the Poisson distribution.

The Wald distribution is infinitely divisible. If \( X \) has the Wald distribution with shape parameter \( \lambda \in (0, \infty) \) and mean \( \mu \in (0, \infty) \), then for \( n \in \N_+ \), \( X \) has the same distribution as \( X_1 + X_2 + \cdots + X_n \) where \( (X_1, X_2, \ldots, X_n) \) are independent, and \( X_i \) has the Wald distribution with shape parameter \( \lambda / n^2 \) and mean \( \mu / n \) for each \( i \in \{1, 2, \ldots, n\} \).