\(\newcommand{\P}{\mathbb{P}}\) \(\newcommand{\E}{\mathbb{E}}\) \(\newcommand{\var}{\text{var}}\) \(\newcommand{\skw}{\text{skew}}\) \(\newcommand{\N}{\mathbb{N}}\) \( \newcommand{\Li}{\text{Li}} \)
  1. Random
  2. 4. Special Distributions
  3. The Zeta Distribution

The Zeta Distribution

The zeta distribution is used to model the size or ranks of certain types of objects randomly chosen from certain types of populations. Typical examples include the frequency of occurrence of a word randomly chosen from a text, or the population rank of a city randomly chosen from a country. The zeta distribution is also known as the Zipf distribution, in honor of the American linguist George Zipf.

Basic Theory

The Zeta Function

The Riemann zeta function \(\zeta\), named after Bernhard Riemann, is defined as follows: \[ \zeta(a) = \sum_{n=1}^\infty \frac{1}{n^a}, \quad a \gt 1 \]

You might recall from calculus that the series in the zeta function converges for \(a \gt 1\) and diverges for \(a \le 1\).

Graph of \( \zeta \) on the interval \( (1, 10] \)
Graph of the zeta function

The zeta function satifies the following properties:

  1. \(\zeta\) is decreasing.
  2. \(\zeta\) is concave upward.
  3. \(\zeta(a) \downarrow 1\) as \(a \uparrow \infty\)
  4. \(\zeta(a) \uparrow \infty\) as \(a \downarrow 1\)

The zeta function is transcendental, and most of its values must be approximated. However, \(\zeta(a)\) can be given explicitly for even integer values of \(a\); in particular, \(\zeta(2) = \frac{\pi^2}{6}\) and \(\zeta(4) = \frac{\pi^4}{90}\).

The Probability Density Function

Random variable \( N \) has the zeta distribution with shape parameter \( a \in (1, \infty) \) if \( N \) has a discrete distribution on \( \N_+ \) with probability density function given by. \[ f(n) = \frac{1}{\zeta(a) n^a}, \quad n \in \N_+ \]

  1. \( f \) is decreasing with mode \( n = 1 \).
  2. When smoothed, \( f \) is concave upward.
Proof:

Clearly \( f \) is a valid PDF, since by definition, \( \zeta(a) \) is the normalizing constant for the function \( n \mapsto \frac{1}{n^a} \) on \( \N_+ \). Part (a) is clear. For part (b), note that the function \( x \mapsto x^{-a} \) on \( [1, \infty) \) has a positive second derivative.

Open the special distribution simulator and select the zeta distribution. Vary the shape parameter and note the shape of the probability density function. For selected values of the parameter, run the simulation 1000 times and compare the empirical density function to the probability density function.

The distribution function and quantile function do not have simple closed forms, except in terms of other special functions.

Open the special distribution calculator and select the zeta distribution. Vary the parameter and note the shape of the distribution and probability density functions. For selected values of the parameter, compute the median and the first and third quartiles.

Moments

Suppose again that \( N \) has the zeta distribution with shape parameter \( a \in (1, \infty) \). The moments of \( X \) can be expressed easily in terms of the zeta function.

If \( k \ge a - 1 \), \( \E(X) = \infty \). If \( k \lt a - 1 \), \[E\left(N^k\right) = \frac{\zeta(a - k)}{\zeta(a)}\]

Proof:

Note that \[ \E\left(N^k\right) = \sum_{n=1}^\infty n^k \frac{1}{\zeta(a) n^a} = \frac{1}{\zeta(a)} \sum_{n=1}^\infty \frac{1}{n^{a - k}}\] If \( a - k \le 1 \), the last sum diverges to \( \infty \). If \( a - k \gt 1 \), the sum converges to \( \zeta(a - k) \)

In particular, the mean, variance, and skewness of \(N\) are as follows:

  1. If \( 1 \lt a \le 2 \), \( \E(N) = \infty \). If \( a \gt 2 \), \[\E(N) = \frac{\zeta(a - 1)}{\zeta(a)}\]
  2. If \( 1 \lt a \le 2 \), \( \var(N) \) does not exist. If \( 2 \lt a \le 3 \), \( \var(N) = \infty \). If \( a \gt 3 \), \[\var(N) = \frac{\zeta(a - 2)}{\zeta(a)} - \left(\frac{\zeta(a - 1)}{\zeta(a)}\right)^2\]
  3. If \( 1 \lt a \le 3 \), \( \skw(N) \) does not exist. If \( 3 \lt a \le 4 \), \( \skw(N) = \infty \). If \( a \gt 4 \), \[ \skw(N) = \frac{\zeta(a - 3) \zeta^2(a) - 3 \zeta(a - 1) \zeta(a - 2) \zeta(a) + 2 \zeta^3(a - 1)}{\zeta^2(a)[\zeta(a - 2) \zeta(a) - \zeta^2(a - 1)]^{3/2}} \]

Open the special distribution simulator and select the zeta distribution. Vary the parameter and note the shape and location of the mean \( \pm \) standard deviation bar. For selected values of the parameter, run the simulation 1000 times and compare the empirical mean and standard deviation to the distribution mean and standard deviation.

The probability generating function of \( N \) can be expressed in terms of the polylogarithm function \( \Li \) that was introduced in the section on the exponential-logarithmic distribution.

\( N \) has probability generating function \( P \) given by \[ P(t) = \E\left(t^N\right) = \frac{\Li_a(t)}{\zeta(a)}, \quad t \in (-1, 1) \]

Proof:

Note that \[ \E\left(t^N\right) = \sum_{n=1}^\infty t^n \frac{1}{n^a \zeta(a)} = \frac{1}{\zeta(a)} \sum_{n=1}^\infty \frac{t^n}{n^a} \] The last sum is the definition of \( \Li_a(t) \).

Related Distributions

In an algebraic sense, the zeta distribution is a discrete version of the Pareto distribution. Recall that if \( a \gt 1 \), the Pareto distribution with shape parameter \( a - 1 \) is a continuous distribution on \( [1, \infty) \) with probability density function \[ f(x) = \frac{a - 1}{x^a}, \quad x \in [1, \infty) \]

Naturally, the limits of the zeta distribution with respect to the shape parameter \( a \) are of interest.

The zeta distribution with shape parameter \( a \in (1, \infty) \) converges to point mass at 1 as \( a \to \infty \).

Proof:

For the PDF \( f \) above, note that \( f(1) = \zeta(a) \to 1 \) as \( a \to \infty \) and for \( n \in \{2, 3, \ldots\} \), \( f(n) = 1 \big/ n^a \zeta(a) \to 0 \) as \( a \to \infty \)

Finally, the zeta distribution is a member of the family of general exponential distributions.

Suppose that \(N\) has the zeta distribution with parameter \(a\). Then the distribution is a one-parameter exponential family with natural parameter \(a\) and natural statistic \(-\ln(N)\).

Proof:

This follows from the definition of the general exponential distribution, since the zeta PDF above can be written in the form \[ f(n) = \frac{1}{\zeta(a)} \exp[-a \ln(n)], \quad n \in \N_+ \]

Computational Exercises

Let \(N\) denote the frequency of occurrence of a word chosen at random from a certain text, and suppose that \(X\) has the zeta distribution with parameter \(a = 2\). Find \(\P(X \gt 4)\).

Answer:

\(\P(N \gt 4) = 1 - \frac{49}{6 \pi^2} \approx 0.1725\)

Suppose that \(N\) has the zeta distribution with parameter \(a = 6\). Approximate each of the following:

  1. \(\E(N)\)
  2. \(\var(N)\)
Answer:
  1. \(\E(N) = 1.109\)
  2. \(\var(N) = 0.025\)