The Exponential Distribution

Recall that in the basic model of the Poisson process, we have points that occur randomly in continuous time, modeled by

[0, \infty)

. The sequence of inter-arrival times is

X = (X_{1}, X_{2}, \dots)

. The strong renewal assumption states that at each arrival time and at each fixed time, the process must probabilistically restart, independent of the past. The first part of that assumption implies that

X

is a sequence of independent, identically distributed variables. The second part of the assumption implies that if the first arrival has not occurred by time

s

, then the time remaining until the arrival occurs must have the same distribution as the first arrival time itself. This is known as the memoryless property and can be stated in terms of a general random variable as follows:

Suppose that $X$ takes values in $[0, \infty)$ . Then $X$ has the memoryless property if the conditional distribution of $X - s$ given $X > s$ is the same as the distribution of $X$ for every $s \in [0, \infty)$ . Equivalently, $P (X > t + s ∣ X > s) = P (X > t), s, t \in [0, \infty)$

The memoryless property determines the distribution of

X

up to a positive parameter, as we will see now.

Suppose that $X$ takes values in $[0, \infty)$ and satisfies the memoryless property. Then $X$ has a continuous distribution and there exists $r \in (0, \infty)$ such that the distribution function $F$ of $X$ is $F (t) = 1 - e^{- r t}, t \in [0, \infty)$

Details:

Let $F^{c} = 1 - F$ denote the denote the right-tail distribution function of $X$ (also known as the reliability function), so that $F^{c} (t) = P (X > t)$ for $t \geq 0$ . From the definition of conditional probability, the memoryless property is equivalent to the law of exponents: $F^{c} (t + s) = F^{c} (s) F^{c} (t), s, t \in [0, \infty)$ Let $a = F^{c} (1)$ . Implicit in the memoryless property is $P (X > t) > 0$ for $t \in [0, \infty)$ , so $a > 0$ . If $n \in N_{+}$ then $F^{c} (n) = F^{c} (\sum_{i = 1}^{n} 1) = \prod_{i = 1}^{n} F^{c} (1) = {[F^{c} (1)]}^{n} = a^{n}$ Next, if $n \in N_{+}$ then $a = F^{c} (1) = F^{c} (\frac{n}{n}) = F^{c} (\sum_{i = 1}^{n} \frac{1}{n}) = \prod_{i = 1}^{n} F^{c} (\frac{1}{n}) = {[F^{c} (\frac{1}{n})]}^{n}$ so $F^{c} (\frac{1}{n}) = a^{1 / n}$ . Now suppose that $m \in N$ and $n \in N_{+}$ . Then $F^{c} (\frac{m}{n}) = F^{c} (\sum_{i = 1}^{m} \frac{1}{n}) = \prod_{i = 1}^{m} F^{c} (\frac{1}{n}) = {[F^{c} (\frac{1}{n})]}^{m} = a^{m / n}$ Thus we have $F^{c} (q) = a^{q}$ for rational $q \in [0, \infty)$ . For $t \in [0, \infty)$ , there exists a sequence of rational numbers $(q_{1}, q_{2}, \dots)$ with $q_{n} ↓ t$ as $n ↑ \infty$ . We have $F^{c} (q_{n}) = a^{q_{n}}$ for each $n \in N_{+}$ . But $F^{c}$ is continuous from the right, so taking limits gives $a^{t} = F^{c} (t)$ . Now let $r = - \ln (a)$ . Then $F^{c} (t) = e^{- r t}$ for $t \in [0, \infty)$ .

The probability density function of $X$ is $f (t) = r e^{- r t}, t \in [0, \infty)$

$f$ is decreasing on $[0, \infty)$ .
$f$ is concave upward on $[0, \infty)$ .
$f (t) \to 0$ as $t \to \infty$ .

Details:

The probability density function follows from [2] since $f = F^{'}$ . The properties in parts (a)–(c) are simple.

The probability distribution defined by the distribution function in [2] or equivalently the probability density function in [3] is the exponential distribution with rate parameter

r

. The reciprocal

\frac{1}{r}

is the scale parameter (as will be justified in [14]). Note that the mode of the distribution is 0, regardless of the parameter

r

, not very helpful as a measure of center.

In the gamma experiment, set $n = 1$ so that the simulated random variable has an exponential distribution. Vary $r$ with the scrollbar and watch how the shape of the probability density function changes. For selected values of $r$ , run the experiment 1000 times and compare the empirical density function to the probability density function.

The quantile function of $X$ is $F^{- 1} (p) = \frac{- \ln (1 - p)}{r}, p \in [0, 1)$

The median of $X$ is $\frac{1}{r} \ln (2) \approx 0.6931 \frac{1}{r}$
The first quartile of $X$ is $\frac{1}{r} [\ln (4) - \ln (3)] \approx 0.2877 \frac{1}{r}$
The third quartile $X$ is $\frac{1}{r} \ln (4) \approx 1.3863 \frac{1}{r}$
The interquartile range is $\frac{1}{r} \ln (3) \approx 1.0986 \frac{1}{r}$

Details:

The formula for $F^{- 1}$ follows from [2] by solving $p = F^{- 1} (t)$ for $t$ in terms of $p$ .

In the quantile app, select the exponential distribution. Vary the scale parameter (which is $1 / r$ ) and note the shape of the distribution/quantile function. For selected values of the parameter, compute a few values of the distribution function and the quantile function.

A process of random points in time is a Poisson process with rate $r \in (0, \infty)$ if and only the interarrvial times are independent, and each has the exponential distribution with rate $r$ .

Constant Failure Rate

Suppose now that

X

has a continuous distribution on

[0, \infty)

and is interpreted as the lifetime of a device. If

F

denotes the distribution function of

X

, then

F^{c} = 1 - F

is the reliability function of

X

. If

f

denotes the probability density function of

X

then the failure rate function

h

is given by

h (t) = \frac{f (t)}{F^{c} (t)}, t \in [0, \infty)

The property of constant failure rate, like the memoryless property, characterizes the exponential distribution.

Random variable $X$ has constant failure rate $r \in (0, \infty)$ if and only if $X$ has the exponential distribution with parameter $r$ .

Details:

If $X$ has the exponential distribution with rate $r > 0$ , then from [2], the reliability function is $F^{c} (t) = e^{- r t}$ and from [3] the probability density function is $f (t) = r e^{- r t}$ , so trivially $X$ has constant rate $r$ . For the converse, recall that in general, the distribution of a lifetime variable $X$ is determined by the failure rate function $h$ . Specifically, if $F^{c} = 1 - F$ denotes the reliability function, then $(F^{c})^{'} = - f$ , so $- h = (F^{c})^{'} / F^{c}$ . Integrating and then taking exponentials gives $F^{c} (t) = \exp (- \int_{0}^{t} h (s) d s), t \in [0, \infty)$ In particular, if $h (t) = r$ for $t \in [0, \infty)$ , then $F^{c} (t) = e^{- r t}$ for $t \in [0, \infty)$ .

The memoryless and constant failure rate properties are the most famous characterizations of the exponential distribution, but are by no means the only ones. Indeed, entire books have been written on characterizations of this distribution.

Moments

Suppose again that

X

has the exponential distribution with rate parameter

r > 0

. Naturaly, we want to know the the mean, variance, and various other moments of

X

If $n \in N$ then $E (X^{n}) = n! / r^{n}$ .

Details:

By the change of variables theorem for expected value, $E (X^{n}) = \int_{0}^{\infty} t^{n} r e^{- r t} d t$ Integrating by parts gives $E (X^{n}) = \frac{n}{r} E (X^{n - 1})$ for $n \in N +$ . Of course $E (X^{0}) = 1$ so the result now follows by induction.

More generally,

E (X^{a}) = Γ (a + 1) / r^{a}

for every

a \in [0, \infty)

, where

Γ

is the gamma function.

In particular.

$E (X) = \frac{1}{r}$
$var (X) = \frac{1}{r^{2}}$
$skew (X) = 2$
$kurt (X) = 9$

Details:

These results follow from [9] and the compuational formulas for variance, skewness, and kurtosis.

In the context of the Poisson process, the parameter

r

is known as the rate of the process. On average, there are

1 / r

time units between arrivals, so the arrivals come at an average rate of

r

per unit time. The Poisson process is completely determined by the sequence of inter-arrival times, and hence is completely determined by the rate

r

Note also that the mean and standard deviation are equal for an exponential distribution, and that the median is always smaller than the mean. Recall also that skewness and kurtosis are standardized measures, and so do not depend on the parameter

r

(which is the reciprocal of the scale parameter). The excess kurtosis is

kurt (X) - 3 = 6

The moment generating function of $X$ is $M (s) = E (e^{s X}) = \frac{r}{r - s}, s \in (- \infty, r)$

Details:

By the change of variables theorem $M (s) = \int_{0}^{\infty} e^{s t} r e^{- r t} d t = \int_{0}^{\infty} r e^{(s - r) t} d t$ The integral evaluates to $\frac{r}{r - s}$ if $s < r$ and to $\infty$ if $s \geq r$ .

In the gamma experiment, set $n = 1$ so that the simulated random variable has an exponential distribution. Vary $r$ with the scrollbar and watch how the mean $\pm$ standard deviation bar changes. For various values of $r$ , run the experiment 1000 times and compare the empirical mean and standard deviation to the distribution mean and standard deviation, respectively.

Additional Properties

The exponential distribution has a number of interesting and important mathematical properties. First, and not surprisingly, it's a member of the general exponential family.

Suppose that $X$ has the exponential distribution with rate parameter $r \in (0, \infty)$ . Then $X$ has a one parameter general exponential distribution, with natural parameter $- r$ and natural statistic $X$ .

Details:

This follows directly from the form of the PDF in [3] and the definition of the general exponential family.

The Scaling Property

As suggested earlier, the exponential distribution is a scale family, and

1 / r

is the scale parameter. Hence the distribution is trivially closed under scale transformations.

Suppose that $X$ has the exponential distribution with rate parameter $r > 0$ and that $c > 0$ . Then $c X$ has the exponential distribution with rate parameter $r / c$ .

Details:

For $t \geq 0$ , $P (c X > t) = P (X > t / c) = e^{- r (t / c)} = e^{- (r / c) t}$ .

Recall that multiplying a random variable by a positive constant frequently corresponds to a change of units (minutes into hours for a lifetime variable, for example). Thus, the exponential distribution is preserved under such changes of units. In the context of the Poisson process, this has to be the case, since the memoryless property, which led to the exponential distribution in the first place, clearly does not depend on the time units.

In fact, the exponential distribution with rate parameter 1 is referred to as the standard exponential distribution. From [14], if

Z

has the standard exponential distribution and

r > 0

, then

X = \frac{1}{r} Z

has the exponential distribution with rate parameter

r

. Conversely, if

X

has the exponential distribution with rate

r > 0

then

Z = r X

has the standard exponential distribution.

Similarly, the Poisson process with rate parameter 1 is referred to as the standard Poisson process. If

Z_{i}

is the

i

th inter-arrival time for the standard Poisson process for

i \in N_{+}

, then letting

X_{i} = \frac{1}{r} Z_{i}

for

i \in N_{+}

gives the inter-arrival times for the Poisson process with rate

r

. Conversely if

X_{i}

is the

i

th inter-arrival time of the Poisson process with rate

r > 0

for

i \in N_{+}

, then

Z_{i} = r X_{i}

for

i \in N_{+}

gives the inter-arrival times for the standard Poisson process.

Relation to the Geometric Distribution

In many respects, the geometric distribution is a discrete version of the exponential distribution. In particular, recall that the geometric distribution on

N_{+}

is the only distribution on

N_{+}

with the memoryless and constant rate properties. So it is not surprising that the two distributions are also connected through various transformations and limits.

Suppose that $X$ has the exponential distribution with rate parameter $r > 0$ . Then

$⌊ X ⌋$ has the geometric distributions on $N$ with success parameter $1 - e^{- r}$ .
$⌈ X ⌉$ has the geometric distributions on $N_{+}$ with success parameter $1 - e^{- r}$ .

Details:

Let $F$ denote the distribution function of $X$ in [2]

For $n \in N$ note that $P (⌊ X ⌋ = n) = P (n \leq X < n + 1) = F (n + 1) - F (n)$ . Substituting into and simplifying gives $P (⌊ X ⌋ = n) = (e^{- r})^{n} (1 - e^{- r})$ .
For $n \in N_{+}$ note that $P (⌈ X ⌉ = n) = P (n - 1 < X \leq n) = F (n) - F (n - 1)$ . Substituting and simplifying gives $P (⌈ X ⌉ = n) = (e^{- r})^{n - 1} (1 - e^{- r})$ .

Our next discussion generalizes part (a) of [15] and also introduces the truncated exponential distribution. In a sense, this discussion is a continuous version of the alternating coin tossing problem studied in the section on the geometric distribution.

Suppose that $X$ has the exponential distribution with rate parameter $r \in (0, \infty)$ and let $t \in (0, \infty)$ . Define $N = max {n \in N : n t < X}$ and $Y = X - N t$ so that $X = N t + Y$ . Then

$N$ and $Y$ are independent.
$N$ has the geometric distribution on $N$ with success parameter $1 - e^{- r t}$
$Y$ has a continuous distribution on $[0, t]$ with probability density function $g$ given by $g (s) = \frac{r e^{- r s}}{1 - e^{- r t}}, s \in [0, t]$

Details:

Let $n \in N$ and $s \in [0, t]$ . Then $P (N = n, Y \leq s) = P (n t < X \leq n t + s) = e^{- r n t} - e^{- r (n t + s)} = e^{- r n t} (1 - e^{- r s})$ The independence of $Y$ and $N$ now follows from the factroing theorem since the last expression is a product of a function of $n$ only and a function of $s$ only. But more explicitly we can rewrite the expression as $P (N = n, Y \leq s) = {(e^{- r t})}^{n} (1 - e^{- r t}) \frac{1 - e^{- r s}}{1 - e^{- r t}}, n \in N, s \in [0, t]$ The function $n \mapsto {(e^{- r t})}^{n} (1 - e^{- r t})$ is the PDF of the geometric distribution on $N$ with success parameter $1 - e^{- r t}$ while the function $s \mapsto (1 - e^{- r s}) / (1 - e^{- r t})$ is tje cumulative distribution function for a continuous distribution on $[0, t]$ with PDF given in part (b).

In the context of [16], the distribution of $Y$ is the same as the conditional distribution of $X$ given $X \leq t$ , and this is the exponential distribution truncated at $t$ .

Details:

If $f$ denotes the PDF of $X$ then the conditional PDF of $X$ given $X \leq t$ is given by $f (s) / P (X \leq t)$ for $s \in [0, t]$ , and of course this is the function $g$ in [16].

It's not difficult to find the mean, variance, and moment generating function of the truncated exponential distribution directly from the density function, but the relationship between

X

Y

and

N

, and the independence of

Y

and

N

, provides another clever method.

Suppose that $Y$ has the exponential distribution with parameter $r \in (0, \infty)$ truncated at $t \in (0, \infty)$ as in [16]. Then

The mean of $Y$ is $E (Y) = \frac{1}{r} - t \frac{e^{- r t}}{1 - e^{- r t}}$
The variance of $Y$ is $var (Y) = \frac{1}{r^{2}} - t^{2} \frac{e^{- r t}}{{(1 - e^{- r t})}^{2}}$
The moment generating function of $Y$ is given by $E (e^{s Y}) = \frac{r}{r - s} \frac{1 - e^{- (r - s) t}}{1 - e^{- r t}}, s < t$

Details:

The crucial facts are that $X = t N + Y$ and that $N$ and $Y$ are independent.

$E (X) = t E (N) + E (Y)$ . But $E (X) = 1 / r$ and $E (N) = e^{- r t} / (1 - e^{- r t})$ .
$var (X) = t^{2} var (N) + var (Y)$ . But $var (X) = 1 / r^{2}$ and $var (N) = e^{- r t} / (1 - e^{- r t})^{2}$ .
$E (e^{s X}) = E (e^{s t N}) E (e^{s Y})$ . But $E (e^{s X}) = r / (r - s)$ for $s < r$ and $E (e^{s t N}) = (1 - e^{- r t}) / (1 - e^{- r t} e^{r s})$ for $s < r$

Suppose again that $Y$ has the exponential distribution with parameter $r \in (0, \infty)$ truncated at $t \in (0, \infty)$ as in [16]. Then

The distribution of $Y$ converges to the exponential distribution with parameter $r$ as $t ↑ \infty$ .
The distribution of $Y$ converges to the uniform distribution on $[0, t]$ as $r ↓ 0$ .

Details:

Part (a) is obvious from [16] or [17]. Part (b) also follows from [16] or from part (c) of [18] and L'Hospital's rule.

The following connection between the exponential and geometric distributions is interesting by itself, but will also be very important in the section on splitting Poisson processes. In words, a random, geometrically distributed sum of independent, identically distributed exponential variables is itself exponential.

Suppose that $X = (X_{1}, X_{2}, \dots)$ is a sequence of independent variables, each with the exponential distribution with rate $r$ . Suppose that $U$ has the geometric distribution on $N_{+}$ with success parameter $p$ and is independent of $X$ . Then $Y = \sum_{i = 1}^{U} X_{i}$ has the exponential distribution with rate $r p$ .

Details:

Recall that the moment generating function of $Y$ is $P \circ M$ where $M$ is the common moment generating function of the terms in the sum, and $P$ is the probability generating function of the number of terms $U$ . But $M (s) = r / (r - s)$ for $s < r$ and $P (s) = p s / [1 - (1 - p) s]$ for $s < 1 / (1 - p)$ . Thus, $(P \circ M) (s) = \frac{p r / (r - s)}{1 - (1 - p) r / (r - s)} = \frac{p r}{p r - s}, s < p r$ It follows that $Y$ has the exponential distribution with parameter $p r$

The next result explores the connection between the Bernoulli trials process and the Poisson process that was begun in the introduction.

For $n \in N_{+}$ , suppose that $U_{n}$ has the geometric distribution on $N_{+}$ with success parameter $p_{n} \in (0, 1)$ , where $n p_{n} \to r \in (0, \infty$ \) as $n \to \infty$ . Then the distribution of $U_{n} / n$ converges to the exponential distribution with parameter $r$ as $n \to \infty$ .

Details:

Let $F_{n}$ denote the CDF of $U_{n} / n$ . Then for $x \in [0, \infty)$ $F_{n} (x) = P (\frac{U_{n}}{n} \leq x) = P (U_{n} \leq n x) = P (U_{n} \leq ⌊ n x ⌋) = 1 - {(1 - p_{n})}^{⌊ n x ⌋}$ But by a famous limit from calculus, ${(1 - p_{n})}^{n} = {(1 - \frac{n p_{n}}{n})}^{n} \to e^{- r}$ as $n \to \infty$ , and hence ${(1 - p_{n})}^{n x} \to e^{- r x}$ as $n \to \infty$ . But by definition, $⌊ n x ⌋ \leq n x < ⌊ n x ⌋ + 1$ or equivalently, $n x - 1 < ⌊ n x ⌋ \leq n x$ so it follows that ${(1 - p_{n})}^{⌊ n x ⌋} \to e^{- r x}$ as $n \to \infty$ . Hence $F_{n} (x) \to 1 - e^{- r x}$ as $n \to \infty$ , which is the CDF of the exponential distribution.

To understand this result more clearly, suppose that we have a sequence of Bernoulli trials processes. In process

n \in N_{+}

, we run the trials at a rate of

n

per unit time, with probability of success

p_{n}

. Thus, the actual time of the first success in process

n

U_{n} / n

. The last result shows that if

n p_{n} \to r > 0

n \to \infty

, then the sequence of Bernoulli trials processes converges to the Poisson process with rate parameter

r

n \to \infty

. We will return to this point in subsequent sections.

Orderings and Order Statistics

Suppose that $X$ and $Y$ have exponential distributions with parameters $a$ and $b$ , respectively, and are independent. Then $P (X < Y) = \frac{a}{a + b}$

Details:

This result can be proved in a straightforward way by integrating the joint PDF of $(X, Y)$ over ${(x, y) : 0 < x < y < \infty}$ . A more elegant proof uses conditioning and the moment generating function in [11]: $P (Y > X) = E [P (Y > X ∣ X)] = E (e^{- b X}) = \frac{a}{a + b}$

Suppose that $X$ and $Y$ are independent variables with values in $[0, \infty)$ and that $Y$ has the exponential distribution with rate parameter $r > 0$ . Then $X$ and $Y - X$ are conditionally independent given $X < Y$ , and the conditional distribution of $Y - X$ is also exponential with parameter $r$ .

Details:

Suppose that $A \subseteq [0, \infty)$ (measurable of course) and $t \geq 0$ . Then $P (X \in A, Y - X \geq t ∣ X < Y) = \frac{P (X \in A, Y - X \geq t)}{P (X < Y)}$ But conditioning on $X$ we can write the numerator as $P (X \in A, Y - X > t) = E [P (X \in A, Y - X > t ∣ X)] = E [P (Y > X + t ∣ X), X \in A] = E [e^{- r (t + X)}, X \in A] = e^{- r t} E (e^{- r X}, X \in A)$ Similarly, conditioning on $X$ gives $P (X < Y) = E (e^{- r X})$ . Thus $P (X \in A, Y - X > t ∣ X < Y) = e^{- r t} \frac{E (e^{- r X}, X \in A)}{E (e^{- r X})}$ Letting $A = [0, \infty)$ we have $P (Y > t) = e^{- r t}$ so given $X < Y$ , the variable $Y - X$ has the exponential distribution with parameter $r$ . Letting $t = 0$ , we see that given $X < Y$ , variable $X$ has the distribution $A \mapsto \frac{E (e^{- r X}, X \in A)}{E (e^{- r X})}$ Finally, because of the factoring, $X$ and $Y - X$ are conditionally independent given $X < Y$ .

For our next discussion, suppose that

X = (X_{1}, X_{2}, \dots, X_{n})

is a sequence of independent random variables, and that

X_{i}

has the exponential distribution with rate parameter

r_{i} > 0

for each

i \in {1, 2, \dots, n}

Let $U = min {X_{1}, X_{2}, \dots, X_{n}}$ . Then $U$ has the exponential distribution with parameter $\sum_{i = 1}^{n} r_{i}$ .

Details:

Recall that in general, ${U > t} = {X_{1} > t, X_{2} > t, \dots, X_{n} > t}$ and therefore by independence, $F^{c} (t) = F_{1}^{c} (t) F_{2}^{c} (t) \dots F_{n}^{c} (t)$ for $t \geq 0$ , where $F^{c}$ is the reliability function of $U$ and $F_{i}^{c}$ is the reliability function of $X_{i}$ for each $i$ . When $X_{i}$ has the exponential distribution with rate $r_{i}$ for each $i$ , we have $F^{c} (t) = \exp [- (\sum_{i = 1}^{n} r_{i}) t]$ for $t \geq 0$ .

In the context of reliability, if a series system has independent components, each with an exponentially distributed lifetime, then the lifetime of the system is also exponentially distributed, and the failure rate of the system is the sum of the component failure rates. In the context of random processes, if we have

n

independent Poisson process, then the new process obtained by combining the random points in time is also Poisson, and the rate of the new process is the sum of the rates of the individual processes (we will return to this point latter).

Let $V = max {X_{1}, X_{2}, \dots, X_{n}}$ . Then $V$ has distribution function $F$ given by $F (t) = \prod_{i = 1}^{n} (1 - e^{- r_{i} t}), t \in [0, \infty)$

Details:

Recall that in general, ${V \leq t} = {X_{1} \leq t, X_{2} \leq t, \dots, X_{n} \leq t}$ and therefore by independence, $F (t) = F_{1} (t) F_{2} (t) \dots F_{n} (t)$ for $t \geq 0$ , where $F$ is the distribution function of $V$ and $F_{i}$ is the distribution function of $X_{i}$ for each $i$ .

Consider the special case where

r_{i} = r \in (0, \infty)

for each

i \in N_{+}

. In statistical terms,

X

is a random sample of size

n

from the exponential distribution with parameter

r

. From [24], the minimum

U

has the exponential distribution with rate

n r

while by [25], the maximum

V

has distribution function

F (t) = {(1 - e^{- r t})}^{n}

for

t \in [0, \infty)

. Recall that

U

and

V

are the first and last order statistics, respectively.

In the order statistic experiment, select the exponential distribution.

Set $k = 1$ (this gives the minimum $U$ ). Vary $n$ with the scrollbar and note the shape of the probability density function. For selected values of $n$ , run the simulation 1000 times and compare the empirical density function to the true probability density function.
Vary $n$ with the scrollbar, set $k = n$ each time (this gives the maximum $V$ ), and note the shape of the probability density function. For selected values of $n$ , run the simulation 1000 times and compare the empirical density function to the true probability density function.

Curiously, the distribution of the maximum of independent, identically distributed exponential variables is also the distribution of the sum of independent exponential variables, with rates that grow linearly with the index.

Suppose that $r_{i} = i r$ for each $i \in {1, 2, \dots, n}$ where $r \in (0, \infty)$ . Then $Y = \sum_{i = 1}^{n} X_{i}$ has distribution function $F$ given by $F (t) = (1 - e^{- r t})^{n}, t \in [0, \infty)$

Details:

By assumption, $X_{k}$ has PDF $f_{k}$ given by $f_{k} (t) = k r e^{- k r t}$ for $t \in [0, \infty)$ . We want to show that $Y_{n} = \sum_{i = 1}^{n} X_{i}$ has PDF $g_{n}$ given by $g_{n} (t) = n r e^{- r t} (1 - e^{- r t})^{n - 1}, t \in [0, \infty)$ The PDF of a sum of independent variables is the convolution of the individual PDFs, so we want to show that $f_{1} * f_{2} * \dots * f_{n} = g_{n}, n \in N_{+}$ The proof is by induction on $n$ . Trivially $f_{1} = g_{1}$ , so suppose the result holds for a given $n \in N_{+}$ . Then $\begin{aligned} g_{n} * f_{n + 1} (t) & = \int_{0}^{t} g_{n} (s) f_{n + 1} (t - s) d s = \int_{0}^{t} n r e^{- r s} (1 - e^{- r s})^{n - 1} (n + 1) r e^{- r (n + 1) (t - s)} d s \\ = r (n + 1) e^{- r (n + 1) t} \int_{0}^{t} n (1 - e^{- r s})^{n - 1} r e^{r n s} d s \end{aligned}$ Now substitute $u = e^{r s}$ so that $d u = r e^{r s} d s$ or equivalently $r d s = d u / u$ . After some algebra, $\begin{aligned} g_{n} * f_{n + 1} (t) & = r (n + 1) e^{- r (n + 1) t} \int_{1}^{e^{r t}} n (u - 1)^{n - 1} d u \\ = r (n + 1) e^{- r (n + 1) t} (e^{r t} - 1)^{n} = r (n + 1) e^{- r t} (1 - e^{- r t})^{n} = g_{n + 1} (t) \end{aligned}$

This result has an application to the Yule process, named for George Yule. The Yule process, which has some parallels with the Poisson process, is studied in the chapter on Markov processes. We can now generalize [22]:

For $i \in {1, 2, \dots, n}$ , $P (X_{i} < X_{j} for all j \neq i) = \frac{r_{i}}{\sum_{j = 1}^{n} r_{j}}$

Details:

First, note that $X_{i} < X_{j}$ for all $i \neq j$ if and only if $X_{i} < min {X_{j} : j \neq i}$ . But the minimum on the right is independent of $X_{i}$ and by [24], has the exponential distribution with parameter $\sum_{j \neq i} r_{j}$ . The result now follows from [22].

Suppose that for each

i

X_{i}

is the time until an event of interest occurs (the arrival of a customer, the failure of a device, etc.) and that these times are independent and exponentially distributed. Then the first time

U

that one of the events occurs is also exponentially distributed, and the probability that the first event to occur is event

i

is proportional to the rate

r_{i}

The probability of a total ordering is $P (X_{1} < X_{2} < \dots < X_{n}) = \prod_{i = 1}^{n} \frac{r_{i}}{\sum_{j = i}^{n} r_{j}}$

Details:

Let $A = {X_{1} < X_{j} for all j \in {2, 3, \dots, n}}$ . then $P (X_{1} < X_{2} < \dots < X_{n}) = P (A, X_{2} < X_{3} < \dots < X_{n}) = P (A) P (X_{2} < X_{3} < \dots < X_{n} ∣ A)$ But $P (A) = \frac{r_{1}}{\sum_{i = 1}^{n} r_{i}}$ from the previous result, and ${X_{2} < X_{3} < \dots < X_{n}}$ is independent of $A$ . Thus we have $P (X_{1} < X_{2} < \dots < X_{n}) = \frac{r_{1}}{\sum_{i = 1}^{n} r_{i}} P (X_{2} < X_{3} < \dots < X_{n})$ so the result follows by induction.

Of course, the probabilities of other orderings can be computed by permuting the parameters appropriately in [29].

Results [24] and [28] are very important in the theory of continuous-time Markov chains. But for that application and others, it's convenient to extend the exponential distribution to two degenerate cases: point mass at 0 and point mass at

\infty

(so the first is the distribution of a random variable that takes the value 0 with probability 1, and the second the distribution of a random variable that takes the value

\infty

with probability 1). In terms of the rate parameter

r

and the distribution function

F

, point mass at 0 corresponds to

r = \infty

so that

F (t) = 1

for

0 < t < \infty

. Point mass at

\infty

corresponds to

r = 0

so that

F (t) = 0

for

0 < t < \infty

. The memoryless property, as expressed in terms of the reliability function

F^{c}

, still holds for these degenerate cases on

(0, \infty)

F^{c} (s) F^{c} (t) = F^{c} (s + t), s, t \in (0, \infty)

We also need to extend some of results above for a finite number of variables to a countably infinite number of variables. So for the remainder of this discussion, suppose that

{X_{i} : i \in I}

is a countable collection of independent random variables, and that

X_{i}

has the exponential distribution with parameter

r_{i} \in (0, \infty)

for each

i \in I

Let $U = inf {X_{i} : i \in I}$ . Then $U$ has the exponential distribution with parameter $\sum_{i \in I} r_{i}$

Details:

The proof is almost the same as in [24]. Note that ${U \geq t} = {X_{i} \geq t for all i \in I}$ and so $P (U \geq t) = \prod_{i \in I} P (X_{i} \geq t) = \prod_{i \in I} e^{- r_{i} t} = \exp [- (\sum_{i \in I} r_{i}) t]$ If $\sum_{i \in I} r_{i} < \infty$ then $U$ has a proper exponential distribution with the sum as the parameter. If $\sum_{i \in I} r_{i} = \infty$ then $P (U \geq t) = 0$ for all $t \in (0, \infty)$ so $P (U = 0) = 1$ .

For $i \in N_{+}$ , $P (X_{i} < X_{j} for all j \in I - {i}) = \frac{r_{i}}{\sum_{j \in I} r_{j}}$

Details:

First note that since the variables have continuous distributions and $I$ is countable, $P (X_{i} < X_{j} for all j \in I - {i}) = P (X_{i} \leq X_{j} for all j \in I - {i})$ Next note that $X_{i} \leq X_{j}$ for all $j \in I - {i}$ if and only if $X_{i} \leq U_{i}$ where $U_{i} = inf {X_{j} : j \in I - {i}}$ . But $U_{i}$ is independent of $X_{i}$ and, by [29], has the exponential distribution with parameter $s_{i} = \sum_{j \in I - {i}} r_{j}$ . If $s_{i} = \infty$ , then $U_{i}$ is 0 with probability 1, and so $P (X_{i} \leq U_{i}) = 0 = r_{i} / s_{i}$ . If $s_{i} < \infty$ , then $X_{i}$ and $U_{i}$ have proper exponential distributions, and so the result now follows from [22].

We need one last result in this setting: a condition that ensures that the sum of an infinite collection of exponential variables is finite with probability one.

Let $Y = \sum_{i \in I} X_{i}$ and $μ = \sum_{i \in I} 1 / r_{i}$ . Then $μ = E (Y)$ and $P (Y < \infty) = 1$ if and only if $μ < \infty$ .

Details:

The result is trivial if $I$ is finite, so assume that $I = N_{+}$ . Recall that $E (X_{i}) = 1 / r_{i}$ and hence $μ = E (Y)$ . Trivially if $μ < \infty$ then $P (Y < \infty) = 1$ . Conversely, suppose that $P (Y < \infty) = 1$ . Then $P (e^{- Y} > 0) = 1$ and hence $E (e^{- Y}) > 0$ . Using independence and the moment generating function in [11], $E (e^{- Y}) = E (\prod_{i = 1}^{\infty} e^{- X_{i}}) = \prod_{i = 1}^{\infty} E (e^{- X_{i}}) = \prod_{i = 1}^{\infty} \frac{r_{i}}{r_{i} + 1} > 0$ Next recall that if $p_{i} \in (0, 1)$ for $i \in N_{+}$ then $\prod_{i = 1}^{\infty} p_{i} > 0 if and only if \sum_{i = 1}^{\infty} (1 - p_{i}) < \infty$ Hence it follows that $\sum_{i = 1}^{\infty} (1 - \frac{r_{i}}{r_{i} + 1}) = \sum_{i = 1}^{\infty} \frac{1}{r_{i} + 1} < \infty$ In particular, this means that $1 / (r_{i} + 1) \to 0$ as $i \to \infty$ and hence $r_{i} \to \infty$ as $i \to \infty$ . But then $\frac{1 / (r_{i} + 1)}{1 / r_{i}} = \frac{r_{i}}{r_{i} + 1} \to 1 as i \to \infty$ By the comparison test for infinite series, it follows that $μ = \sum_{i = 1}^{\infty} \frac{1}{r_{i}} < \infty$

Computational Exercises

Show directly that the exponential probability density function is a valid probability density function.

Details:

Clearly $f (t) = r e^{- r t} > 0$ for $t \in [0, \infty)$ . Simple integration that $\int_{0}^{\infty} r e^{- r t} d t = 1$

Suppose that the length of a telephone call (in minutes) is exponentially distributed with rate parameter $r = 0.2$ . Find each of the following:

The probability that the call lasts between 2 and 7 minutes.
The median, the first and third quartiles, and the interquartile range of the call length.

Details:

Let $X$ denote the call length.

$P (2 < X < 7) = 0.4237$
$q_{1} = 1.4384$ , $q_{2} = 3.4657$ , $q_{3} = 6.9315$ , $q_{3} - q_{1} = 5.4931$

Suppose that the lifetime of a certain electronic component (in hours) is exponentially distributed with rate parameter $r = 0.001$ . Find each of the following:

The probability that the component lasts at least 2000 hours.
The median, the first and third quartiles, and the interquartile range of the lifetime.

Details:

Let $T$ denote the lifetime

$P (T \geq 2000) = 0.1353$
$q_{1} = 287.682$ , $q_{2} = 693.147$ , $q_{3} = 1386.294$ , $q_{3} - q_{1} = 1098.612$

Suppose that the time between requests to a web server (in seconds) is exponentially distributed with rate parameter $r = 2$ . Find each of the following:

The mean and standard deviation of the time between requests.
The probability that the time between requests is less that 0.5 seconds.
The median, the first and third quartiles, and the interquartile range of the time between requests.

Details:

Let $T$ denote the time between requests.

$E (T) = 0.5$ , $sd (T) = 0.5$
$P (T < 0.5) = 0.6321$
$q_{1} = 0.1438$ , $q_{2} = 0.3466$ , $q_{3} = 0.6931$ , $q_{3} - q_{1} = 0.5493$

Suppose that the lifetime $X$ of a fuse (in 100 hour units) is exponentially distributed with $P (X > 10) = 0.8$ . Find each of the following:

The rate parameter.
The mean and standard deviation.
The median, the first and third quartiles, and the interquartile range of the lifetime.

Details:

Let $X$ denote the lifetime.

$r = 0.02231$
$E (X) = 44.814$ , $sd (X) = 44.814$
$q_{1} = 12.8922$ , $q_{2} = 31.0628$ , $q_{3} = 62.1257$ , $q_{3} - q_{1} = 49.2334$

The position $X$ of the first defect on a digital tape (in cm) has the exponential distribution with mean 100. Find each of the following:

The rate parameter.
The probability that $X < 200$ given $X > 150$ .
The standard deviation.
The median, the first and third quartiles, and the interquartile range of the position.

Details:

Let $X$ denote the position of the first defect.

$r = 0.01$
$P (X < 200 ∣ X > 150) = 0.3935$
$sd (X) = 100$
$q_{1} = 28.7682$ , $q_{2} = 69.3147$ , $q_{3} = 138.6294$ , $q_{3} - q_{1} = 109.6812$

Suppose that $X, Y, Z$ are independent, exponentially distributed random variables with respective parameters $a, b, c \in (0, \infty)$ . Find the probability of each of the 6 orderings of the variables.

Details:

$P (X < Y < Z) = \frac{a}{a + b + c} \frac{b}{b + c}$
$P (X < Z < Y) = \frac{a}{a + b + c} \frac{c}{b + c}$
$P (Y < X < Z) = \frac{b}{a + b + c} \frac{a}{a + c}$
$P (Y < Z < X) = \frac{b}{a + b + c} \frac{c}{a + c}$
$P (Z < X < Y) = \frac{c}{a + b + c} \frac{a}{a + b}$
$P (Z < Y < X) = \frac{c}{a + b + c} \frac{b}{a + b}$

2. The Exponential Distribution

Basic Theory

The Memoryless Property

Constant Failure Rate

Moments

Additional Properties

The Scaling Property

Relation to the Geometric Distribution

Orderings and Order Statistics

Computational Exercises