Stochastic Processes

This section requires measure theory, so you may need to review the chapter on foundations, particularly the sections on topology, measurable spaces, and positive measures. First, recall that a set

E

almost always comes with a

σ

-algebra

E

of admissible subsets, so that

(E, E)

is a measurable space. Usually in fact,

E

has a topology and

E

is the corresponding Borel

σ

-algebra, that is, the

σ

-algebra generated by the topology. If

E

is countable, we almost always take

E

to be the collection of all subsets of

E

, and in this case

(E, E)

is a discrete space. The other common case is when

E

is a measurable subset of

R^{n}

for some

n \in N

, in which case

E

is the collection of measurable subsets of

E

. If

(E_{1}, E_{1}), (E_{2}, E_{2}), \dots, (E_{n}, E_{n})

are measurable spaces for some

n \in N_{+}

, then the Cartesian product

E_{1} \times E_{2} \times \dots \times E_{n}

is given the product

σ

-algebra

E_{1} \times E_{2} \times \dots \times E_{n}

. As a special case, the Cartesian power

E^{n}

is given the corresponding power

σ

-algebra

E^{n}

With these preliminary remarks out of the way, suppose that

(Ω, F, P)

is a probability space so that

Ω

is the set of outcomes,

F

the

σ

-algebra of events, and

P

is the probability measure on the sample space

(Ω, F)

. Suppose also that

(S, S)

and

(T, T)

are measurable spaces. Here is our main definition:

A random process or stochastic process on $(Ω, F, P)$ with state space $(S, S)$ and index set $T$ is a collection of random variables $X = {X_{t} : t \in T}$ such that $X_{t}$ takes values in $S$ for each $t \in T$ .

Sometimes it's notationally convenient to write

X (t)

instead of

X_{t}

for

t \in T

. Often

T = N

T = [0, \infty)

and the elements of

T

are interpreted as points in time (discrete time in the first case and continuous time in the second). So then

X_{t} \in S

is the state of the random process at time

t \in T

, and the index space

(T, T)

becomes the time space.

Since

X_{t}

is itself a function from

Ω

into

S

, it follows that ultimately, a stochastic process is a function from

Ω \times T

into

S

. Stated another way,

t \mapsto X_{t}

is a random function on the probability space

(Ω, F, P)

. To make this precise, recall that

S^{T}

is the notation sometimes used for the collection of functions from

T

into

S

. Recall also that a natural

σ

-algebra used for

S^{T}

is the one generated by sets of the form

{f \in S^{T} : f (t) \in A_{t} for all t \in T}, where A_{t} \in S for every t \in T and A_{t} = S for all but finitely many t \in T

This

σ

-algebra, denoted

S^{T}

, generalizes the ordinary power

σ

-algebra

S^{n}

mentioned in the opening paragraph and will be important in the discussion of existence below.

Suppose that $X = {X_{t} : t \in T}$ is a stochastic process on the probability space $(Ω, F, P)$ with state space $(S, S)$ and index set $T$ . Then the mapping that takes $ω$ into the function $t \mapsto X_{t} (ω)$ is measurable with respect to $(Ω, F)$ and $(S^{T}, S^{T})$ .

Details:

Recall that a mapping with values in $S^{T}$ is measurable if and only if each of its coordinate functions is measurable. In the present context that means that we must show that the function $X_{t}$ is measurable with respect to $(Ω, F)$ and $(S, S)$ for each $t \in T$ . But of course, that follows from the very meaning of the term random variable.

For

ω \in Ω

, the function

t \mapsto X_{t} (ω)

is known as a sample path of the process. So

S^{T}

, the set of functions from

T

into

S

, can be thought of as a set of outcomes of the stochastic process

X

, a point we will return to in our discussion of existence below.

As noted in the proof [2],

X_{t}

is a measurable function from

Ω

into

S

for each

t \in T

, by the very meaning of the term random variable. But it does not follow in general that

(ω, t) \mapsto X_{t} (ω)

is measurable as a function from

Ω \times T

into

S

. In fact, the

σ

-algebra on

T

has played no role in our discussion so far. Informally, a statement about

X_{t}

for a fixed

t \in T

or even a statement about

X_{t}

for countably many

t \in T

defines an event. But it does not follow that a statement about

X_{t}

for uncountably many

t \in T

defines an event. We often want to make such statements, so the following definition is inevitable:

A stochastic process $X = {X_{t} : t \in T}$ defined on the probability space $(Ω, F, P)$ and with index space $(T, T)$ and state space $(S, S)$ is measurable if $(ω, t) \mapsto X_{t} (ω)$ is a measurable function from $Ω \times T$ into $S$ .

Every stochastic process indexed by a countable set

T

is measurable, so the definition is only important when

T

is uncountable, and in particular for

T = [0, \infty)

Equivalent Processes

Our next goal is to study different ways that two stochastic processes, with the same state and index spaces, can be equivalent, so you may need to review equivalence relations. We will assume that the diagonal

D = {(x, x) : x \in S} \in S^{2}

, an assumption that almost always holds in applications, and in particular for the discrete and Euclidean spaces that are most important to us. Sufficient conditions are that

S

have a sub

σ

-algebra that is countably generated and contains all of the singleton sets, properties that hold for the Borel

σ

-algebra when the topology on

S

is locally compact, Hausdorff, and has a countable base.

First, we often feel that we understand a random process

X = {X_{t} : t \in T}

well if we know the finite dimensional distributions, that is, if we know the distribution of

(X_{t_{1}}, X_{t_{2}}, \dots, X_{t_{n}})

for every choice of

n \in N_{+}

and

(t_{1}, t_{2}, \dots, t_{n}) \in T^{n}

. Thus, we can compute

P [(X_{t_{1}}, X_{t_{2}}, \dots, X_{t_{n}}) \in A]

for every

n \in N_{+}

(t_{1}, t_{2}, \dots, t_{n}) \in T^{n}

, and

A \in S^{n}

. Using various rules of probability, we can compute the probabilities of many events involving infinitely many values of the index parameter

t

as well. With this idea in mind, we have the following definition:

Random processes $X = {X_{t} : t \in T}$ and $Y = {Y_{t} : t \in T}$ with state space $(S, S)$ and index set $T$ are equivalent in distribution if they have the same finite dimensional distributions. This defines an equivalence relation on the collection of stochastic processes with this state space and index set. That is, if $X$ , $Y$ , and $Z$ are such processes then

$X$ is equivalent in distribution to $X$ (the reflexive property)
If $X$ is equivalent in distribution to $Y$ then $Y$ is equivalent in distribution to $X$ (the symmetric property)
If $X$ is equivalent in distribution to $Y$ and $Y$ is equivalent in distribution to $Z$ then $X$ is equivalent in distribution to $Z$ (the transitive property)

Note that since only the finite-dimensional distributions of the processes

X

and

Y

are involved in the definition, the processes need not be defined on the same probability space. Thus, equivalence in distribution partitions the collection of all random processes with a given state space and index set into mutually disjoint equivalence classes. But of course, we already know that two random variables can have the same distribution but be very different as variables (that is, as functions on the sample space). Clearly, the same statement applies to random processes.

Suppose that $X$ is a sequence of independent indicator random variables with $P (X_{n} = 1) = \frac{1}{2}$ for each $n \in N_{+}$ . Let $Y_{n} = 1 - X_{n}$ for $n \in N_{+}$ . Then $Y = (Y_{1}, Y_{2}, \dots)$ is equivalent in distribution to $X$ but $P (X_{n} \neq Y_{n} for every n \in N_{+}) = 1$

Details:

The state set is ${0, 1}$ and $Y_{n} = 1$ if and only if $X_{n} = 0$ . Hence $Y$ is also a sequence of independent indicator variables with $P (Y_{n} = 1) = \frac{1}{2}$ for each $n \in N_{+}$ , and so $X$ and $Y$ are equivalent in distribution.

In example [5],

X

and

Y

are sequences of Bernoulli trials with success parameter

p = \frac{1}{2}

Motivated by this example, let's look at another, stronger way that random processes can be equivalent. First recall that random variables

X

and

Y

(Ω, F, P)

, with values in

S

, are equivalent if

P (X = Y) = 1

Suppose that $X = {X_{t} : t \in T}$ and $Y = {Y_{t} : t \in T}$ are stochastic processes defined on the same probability space $(Ω, F, P)$ and both with state space $(S, S)$ and index set $T$ . Then $Y$ is a versions of $X$ if $Y_{t}$ is equivalent to $X_{t}$ (so that $P (X_{t} = Y_{t}) = 1$ ) for every $t \in T$ . This defines an equivalence relation on the collection of stochastic processes on the same probability space and with the same state space and index set. That is, if $X$ , $Y$ , and $Z$ are such processes then

$X$ is a version of $X$ (the reflexive property)
If $X$ is a version of $Y$ then $Y$ is ia version of $X$ (the symmetric property)
If $X$ is a version of $Y$ and $Y$ is of $Z$ then $X$ is a version of $Z$ (the transitive property)

Details:

Note that $(X_{t}, Y_{t})$ is a random variable with values in $S^{2}$ (and so the function $ω \mapsto (X_{t} (ω), Y_{t} (ω))$ is measurable). The event ${X_{t} = Y_{t}}$ is the inverse image of the diagonal $D \in S^{2}$ under this mapping, and so the definition makes sense.

So the version of relation partitions the collection of stochastic processes on a given probability space and with a given state space and index set into mutually disjoint equivalence classes.

Suppose again that $X = {X_{t} : t \in T}$ and $Y = {Y_{t} : t \in T}$ are random processes on $(Ω, F, P)$ with state space $(S, S)$ and index set $T$ . If $Y$ is a version of $X$ then $Y$ and $X$ are equivalent in distribution.

Details:

Suppose that $(t_{1}, t_{2}, \dots, t_{n}) \in T^{n}$ and that $A \in S^{n}$ . Recall that the intersection of a finite (or even countably infinite) collection of events with probability 1 still has probability 1. Hence $\begin{aligned} P [(X_{t_{1}}, X_{t_{2}}, \dots, X_{t_{n}}) \in A] & = P [(X_{t_{1}}, X_{t_{2}}, \dots, X_{t_{n}}) \in A, X_{t_{1}} = Y_{t_{1}}, X_{t_{2}} = Y_{t_{2}}, \dots, X_{t_{n}} = Y_{t_{n}}] \\ = P [(Y_{t_{1}}, Y_{t_{2}}, \dots, Y_{t_{n}}) \in A, X_{t_{1}} = Y_{t_{1}}, X_{t_{2}} = Y_{t_{2}}, \dots, X_{t_{n}} = Y_{t_{n}}] = P [(Y_{t_{1}}, Y_{t_{2}}, \dots, Y_{t_{n}}) \in A] \end{aligned}$

As noted in the proof, a countable intersection of events with probability 1 still has probability 1. Hence if

T

is countable and random processes

X

is a version of

Y

then

P (X_{t} = Y_{t} for all t \in T) = 1

X

and

Y

really are essentially the same random process. But when

T

is uncountable the result in the displayed equation may not be true, and

X

and

Y

may be very different as random functions on

T

. Here is a simple example:

Suppose that $Ω = T = [0, \infty)$ , $F = T$ is the $σ$ -algebra of Borel measurable subsets of $[0, \infty)$ , and $P$ is any continuous probability measure on $(Ω, F)$ . Let $S = {0, 1}$ (with all subsets measurable, of course). For $t \in T$ and $ω \in Ω$ , define $X_{t} (ω) = 1_{t} (ω)$ and $Y_{t} (ω) = 0$ . Then $X = {X_{t} : t \in T}$ is a version of $Y = {Y_{t} : t \in T}$ , but $P (X_{t} = Y_{t} for all t \in T} = 0$ .

Details:

For $t \in [0, \infty)$ , $P (X_{t} \neq Y_{t}) = P {t} = 0$ since $P$ is a continuous measure. But ${ω \in Ω : X_{t} (ω) = Y_{t} (ω) for all t \in T} = \emptyset$ .

Suppose that $X = {X_{t} : t \in T}$ and $Y = {Y_{t} : t \in T}$ are measurable random processes on the probability space $(Ω, F, P)$ and with state space $(S, S)$ and index space $(T, T)$ . Then $X$ is indistinguishable from $Y$ if $P (X_{t} = Y_{t} for all t \in T) = 1$ . This defines an equivalence relation on the collection of measurable stochastic processes defined on the same probability space and with the same state and index spaces. That is, if $X$ , $Y$ , and $Z$ are such processes then

$X$ is indistinguishable from $X$ (the reflexive property)
If $X$ is indistinguishable from $Y$ then $Y$ is indistinguishable from $X$ (the symmetric property)
If $X$ is indistinguishable from $Y$ and $Y$ is indistinguishable from $Z$ then $X$ is indistinguishable from $Z$ (the transitive property)

Details:

The measurability requirement for the stochastic processes is needed to ensure that ${X_{t} = Y_{t} for all t \in T}$ is a valid event. To see this, note that $(ω, t) \mapsto (X_{t} (ω), Y_{t} (ω))$ is measurable, as a function from $Ω \times T$ into $S^{2}$ . As before, let $D = {(x, x) : x \in S}$ denote the diagonal. Then $D^{c} \in S^{2}$ and the inverse image of $D^{c}$ under our mapping is ${(ω, t) \in Ω \times T : X_{t} (ω) \neq Y_{t} (ω)} \in F \times T$ The projection of this set onto $Ω$ ${ω \in Ω : X_{t} (ω) \neq Y_{t} (ω) for some t \in T} \in F$ since the projection of a measurable set in the product space is also measurable. Hence the complementary event ${ω \in Ω : X_{t} (ω) = Y_{t} (ω) for all t \in T} \in F$

So the indistinguishable from relation partitions the collection of measurable stochastic processes on a given probability space and with given state space and index space into mutually disjoint equivalence classes. Trivially, if

X

is indistinguishable from

Y

, then

X

is a version of

Y

. As noted above, when

T

is countable, the converse is also true, but not, as example [8] shows, when

T

is uncountable. So to summarize, indistinguishable from implies version of implies equivalent in distribution, but none of the converse implications hold in general.

The Kolmogorov Construction

In applications, a stochastic process is often modeled by giving various distributional properties that the process should satisfy. So the basic existence problem is to construct a process that has these properties. More specifically, how can we construct random processes with specified finite dimensional distributions? Let's start with the simplest case, one that we have seen several times before, and build up from there. Our simplest case is to construct a single random variable with a specified distribution.

Suppose that $(S, S, P)$ is a probability space. Then there exists a random variable $X$ on a probability space $(Ω, F, P)$ such that $X$ takes values in $S$ and has distribution $P$ .

Details:

The proof is utterly trivial. Let $(Ω, F, P) = (S, S, P)$ and define $X : Ω \to S$ by $X (ω) = ω$ , so that $X$ is the identity function. Then ${X \in A} = A$ and so $P (X \in A) = P (A)$ for $A \in S$ .

In spite of its triviality the last result contains the seeds of everything else we will do in this discussion. Next, let's see how to construct a sequence of independent random variables with specified distributions.

Suppose that $P_{i}$ is a probability measure on the measurable space $(S, S)$ for $i \in N_{+}$ . Then there exists an independent sequence of random variables $(X_{1}, X_{2}, \dots)$ on a probability space $(Ω, F, P)$ such that $X_{i}$ takes values in $S$ and has distribution $P_{i}$ for $i \in N_{+}$ .

Details:

Let $Ω = S^{\infty} = S \times S \times \dots$ . Next let $F = S^{\infty}$ , the corresponding product $σ$ -algebra. Recall that this is the $σ$ -algebra generated by sets of the form $A_{1} \times A_{2} \times \dots where A_{i} \in S for each i \in I and A_{i} = S for all but finitely many i \in I$ Finally, let $P = P_{1} \times P_{2} \times \dots$ , the corresponding product measure on $(Ω, F)$ . Recall that this is the unique probability measure that satisfies $P (A_{1} \times A_{2} \times \dots) = P_{1} (A_{1}) P_{2} (A_{2}) \dots$ where $A_{1} \times A_{2} \times \dots$ is a set of the type in the first displayed equation. Now define $X_{i}$ on $Ω$ by $X_{i} (ω_{1}, ω_{2}, \dots) = ω_{i}$ , for $i \in N_{+}$ , so that $X_{i}$ is simply the coordinate function for index $i$ . If $A_{1} \times A_{2} \times \dots$ is a set of the type in the first displayed equation then ${X_{1} \in A_{1}, X_{2} \in A_{2}, \dots} = A_{1} \times A_{2} \times \dots$ and so by the definition of the product measure, $P (X_{1} \in A_{1}, X_{2} \in A_{2}, \dots) = P_{1} (A_{1}) P_{2} (A_{2}) \dots$ It follows that $(X_{1}, X_{2}, \dots)$ is a sequence of independent variables and that $X_{i}$ has distribution $P_{i}$ for $i \in N$ .

If you looked at the proof of the last two results you might notice that the last result can be viewed as a special case of the one before, since

X = (X_{1}, X_{2}, \dots)

is simply the identity function on

Ω = S^{\infty}

. The important step is the existence of the product measure

P

(Ω, F)

The full generalization of these results is known as the Kolmogorov existence theorem (named for Andrei Kolmogorov). We start with the state space

(S, S)

and the index set

T

. The theorem states that if we specify the finite dimensional distributions in a consistent way, then there exists a stochastic process defined on a suitable probability space that has the given finite dimensional distributions. The consistency condition is a bit clunky to state in full generality, but the basic idea is very easy to understand. Suppose that

s

and

t

are distinct elements in

T

and that we specify the distribution (probability measure)

P_{s}

X_{s}

P_{t}

X_{t}

P_{s, t}

(X_{s}, X_{t})

, and

P_{t, s}

(X_{t}, X_{s})

. Then clearly we must specify these so that

P_{s} (A) = P_{s, t} (A \times S), P_{t} (B) = P_{s, t} (S \times B)

For all

A, B \in S

. Clearly we also must have

P_{s, t} (C) = P_{t, s} (C^{'})

for all measurable

C \in S^{2}

, where

C^{'} = {(y, x) : (x, y) \in C}

To state the consistency conditions in general, we need some notation. For

n \in N_{+}

, let

T^{(n)} \subset T^{n}

denote the set of

n

-tuples of distinct elements of

T

, and let

T = ⋃_{n = 1}^{\infty} T^{(n)}

denote the set of all finite sequences of distinct elements of

T

. If

n \in N_{+}

t = (t_{1}, t_{2}, \dots, t_{n}) \in T^{(n)}

and

π

is a permutation of

{1, 2, \dots, n}

, let

t π

denote the element of

T^{(n)}

with coordinates

(t π)_{i} = t_{π (i)}

. That is, we permute the coordinates of

t

according to

π

. If

C \in S^{n}

, let

π C = {(x_{1}, x_{2}, \dots, x_{n}) \in S^{n} : (x_{π (1)}, x_{π (2)}, \dots, x_{π (n)}) \in C} \in S^{n}

finally, if

n > 1

, let

t_{-}

denote the vector

(t_{1}, t_{2}, \dots, t_{n - 1}) \in T^{(n - 1)}

Now suppose that

P_{t}

is a probability measure on

(S^{n}, S^{n})

for each

n \in N_{+}

and

t \in T^{(n)}

. The idea, of course, is that we want the collection

P = {P_{t} : t \in T}

to be the finite dimensional distributions of a random process with index set

T

and state space

(S, S)

. Here is the critical definition:

The collection of probability distributions $P$ relative to $T$ and $(S, S)$ is consistent if

$P_{t π} (C) = P_{t} (π C)$ for every $n \in N_{+}$ , $t \in T^{(n)}$ , permutation $π$ of ${1, 2, \dots, n}$ , and measurable $C \subseteq S^{n}$ .
$P_{t_{-}} (C) = P_{t} (C \times S)$ for every $n > 1$ , $t \in T^{(n)}$ , and measurable $C \subseteq S^{n - 1}$

With the proper definition of consistence, we can state the fundamental theorem.

Kolmogorov Existence Theorem. If $P$ is a consistent collection of probability distributions relative to the index set $T$ and the state space $(S, S)$ , then there exists a probability space $(Ω, F, P)$ and a stochastic process $X = {X_{t} : t \in T}$ on this probability space such that $P$ is the collection of finite dimensional distribution of $X$ .

Details:

Let $Ω = S^{T}$ , the set of functions from $T$ to $S$ . Such functions are the outcomes of the stochastic process. Let $F = S^{T}$ , the product $σ$ -algebra, generated by sets of the form $B = {ω \in Ω : ω (t) \in A_{t} for all t \in T}$ where $A_{t} \in S$ for all $t \in T$ and $A_{t} = S$ for all but finitely many $t \in T$ . We know how our desired probability measure $P$ should work on the sets that generate $F$ . Specifically, suppose that $B$ is a set of the type in the displayed equation, and $A_{t} = S$ except for $t = (t_{1}, t_{2}, \dots, t_{n}) \in T^{(n)}$ . Then we want $P (B) = P_{t} (A_{t_{1}} \times A_{t_{2}} \times \dots \times A_{t_{n}})$ Basic existence and uniqueness theorems, and the consistency of $P$ , guarantee that $P$ can be extended to a probability measure on all of $F$ . Finally, for $t \in T$ we define $X_{t} : Ω \to S$ by $X_{t} (ω) = ω (t)$ for $ω \in Ω$ , so that $X_{t}$ is simply the coordinate function of index $t$ . Thus, we have a stochastic process $X = {X_{t} : t \in T}$ with state space $(S, S)$ , defined on the probability space $(Ω, F, P)$ , with $P$ as the collection of finite dimensional distributions.

Note that except for the more complicated notation, the construction is very similar to the one for a sequence of independent variables in [11]. Again,

X

is essentially the identity function on

Ω = S^{T}

. The important and more difficult part is the construction of the probability measure

P

(Ω, F)

Applications

Our last discussion is a summary of the stochastic processes that are studied in this text. All are classics and are immensely important in applications.

Random processes are associated with Bernoulli trials, include

The sequence of Bernoulli variables
The sequence of binomial variables
The sequence of geometric variables
The sequence of negative binomial variables
The simple random walk

Details:

The Bernoulli trials sequence in (a) is a sequence of independent, identically distributed indicator random variables, and so can be constructed as in [11]. The random processes in (b)–(e) are constructed from the Bernoulli trials sequence.

Random process associated with the Poisson model, include

The sequence of inter-arrival times
The sequence of arrival times
The counting process on $[0, \infty)$ , both in the homogenous case and the non-homogeneous case.
The compound Poisson process.
The counting process on a general measure space

Details:

The random process in (a) is a sequence of independent random variable with a common exponential distribution, and so can be constructed as in [11]. The processes in (b) and (c) can be constructed from the sequence in (a).

8. Stochastic Processes

Introduction

Equivalent Processes

The Kolmogorov Construction

Applications