Our starting point is a positive measure space \( (S, \ms S, \mu) \) so that \( S \) is a set, \( \ms S \) is a \( \sigma \)-algebra of subsets of \( S \), and \( \mu \) is a positive measure on \( (S, \ms S) \).
Special positive measure spaces:
In the context of (a), often our measure space is a subspace \((S, \ms S, \lambda^n)\) so that \( S \in \ms R^n \) with \(\lambda_n(S) \gt 0\) and \(\ms S = \{A \in \ms R^n: A \subseteq S\}\). We have defined the integral of certain measurable functions \( f: S \to \R \) with respect to \( \mu \) and we have studied properties of the integral. In this section, we will study vector spaces of functions that are defined in terms of certain integrability conditions. These function spaces are of fundamental importance in all areas of analysis, including probability. In particular, the results of this section will reappear in the form of spaces of random variables.
Consider a statement on the elements of \(S \), for example an equation or an inequality with \( x \in S \) as a free variable. (Technically such a statement is a predicate on \( S \).) For \( A \in \ms S \), we say that the statement holds on \( A \) if it is true for every \( x \in A \). We say that the statement holds almost everywhere on \( A \) (with respect to \( \mu \)) if there exists \( B \in \ms S \) with \( B \subseteq A \) such that the statement holds on \( B \) and \( \mu(A \setminus B) = 0 \).
Measurable functions \( f, \, g: S \to \R \) are equivalent (with respect to \(\mu\)) if \( f = g \) almost everywhere on \( S \), in which case we write \( f \equiv g \).
The relation \( \equiv \) is an equivalenc relation on the collection of measurable functions from \( S \) to \( \R \). That is, if \( f, \, g, \, h: S \to \R \) are measurable then
Thus, equivalent functions are indistinguishable from the point of view of the measure \( \mu \). As with any equivalence relation, \( \equiv \) partitions the underlying set (in this case the collection of real-valued measurable functions on \( S \)) into equivalence classes of mutually equivalent elements. As we will see, we often view these equivalence classes as the basic objects of study. Our next task is to define measures of the size
of a function; these will become norms in our spaces.
Suppose that \( f: S \to \R \) is measurable. For \( p \in (0, \infty) \) we define \[ \|f\|_p = \left(\int_S \left|f\right|^p \, d\mu\right)^{1/p} \] We also define \( \|f\|_\infty = \inf\left\{b \in [0, \infty]: |f| \le b \text{ almost everywhere on } S \right\}\).
Since \( \left|f\right|^p \) is a nonnegative, measurable function for \( p \in (0, \infty) \), \( \int_S \left|f\right|^p \, d\mu \) exists in \( [0, \infty] \), and hence so does \( \|f\|_p \). Clearly \( \|f\|_\infty \) also exists in \( [0, \infty] \) and is known as the essential supremum of \( f \). A number \( b \in [0, \infty] \) such that \( |f| \le b \) almost everywhere on \( S \) is an essential bound of \( f \) and so, appropriately enough, the essential supremum of \( f \) is the infimum of the essential bounds of \( f \). Thus, we have defined \( \|f\|_p \) for all \( p \in (0, \infty] \). The definition for \( p = \infty \) is special, but we will see that it's the appropriate one.
For \( p \in (0, \infty] \), let \( L^p \) denote the collection of measurable functions \( f: S \to \R \) such \( \|f\|_p \lt \infty \)
So for \( p \in (0, \infty) \), \( f \in L^p \) if and only if \( |f|^p \) is absolutely integrable. The symbol \( L \) is in honor of Henri Lebesgue, who first developed the theory. If we want to indicate the dependence on the underlying measure space, we write \( L^p(S, \ms S, \mu) \). Of course, \( L^1 \) is simply the collection of functions that are absolutely integrable with respect to \( \mu \). Our goal is to study the spaces \( L^p \) for \( p \in (0, \infty] \). We start with some simple properties.
Suppose that \( f: S \to \R \) is measurable. Then for \( p \in (0, \infty] \),
Suppose that \( f: S \to \R \) is measurable and \( c \in \R \). Then \( \|c f\|_p = |c| \|f\|_p \) for \( p \in (0, \infty] \).
Again, when \( p \in (0, \infty) \), this result follow easily from properties of the integral that we already have: \[ \int_S \left|c f\right|^p \, d\mu = \left|c\right|^p \int_S \left|f\right|^p \, d\mu \] Taking the \( p \)th root of both sides gives the result. For \( p = \infty \), the result is trivially true if \( c = 0 \). For \( c \ne 0 \), note that \( b \in [0, \infty] \) is an essential bound of \( |f| \) if and only if \( |c| b \) is an essential bound if \( |c f | \).
In particular, if \( f \in L^p \) and \( c \in \R \) then \( c f \in L^p \).
Certain pairs of our function spaces turn out to be dual or complimentary to one another in a sense. To understand this, we need the following definition.
Indices \( p, \, q \in (1, \infty) \) are said to be conjugate if \( 1/p + 1/q = 1 \). In addition, \( 1 \) and \( \infty \) are conjugate indices.
For justification of the last case, note that if \( p \in (1, \infty) \), then the index conjugate to \( p \) is \[ q = \frac{1}{1 - 1/p} \] and \( q \uparrow \infty \) as \( p \downarrow 1 \). Note that \( p = q = 2 \) are conjugate indices, and this is the only case where the indices are the same. Ultimately, the importance of conjugate indices stems from the following inequality:
If \( x, \, y \in (0, \infty) \) and if \( p, \, q \in (1, \infty) \) are conjugate indices, then \[ x y \le \frac{1}{p} x^p + \frac{1}{q} y^q \] Moreover, equality occurs if and only if \( x^p = y^q \).
Proof 1. From properties of the natural logarithm function, \[ \ln(x y) = \ln(x) + \ln(y) = \frac{1}{p}\ln\left(x^p\right) + \frac{1}{q} \ln\left(y^q\right) \] But the natural logarithm function is concave and \( 1/p + 1/q = 1 \) so \[ \ln(x y) = \frac{1}{p}\ln\left(x^p\right) + \frac{1}{q}\ln\left(y^q\right) \le \ln\left(\frac{1}{p} x^p + \frac{1}{q} y^q\right) \] Taking exponentials we have \[ x y \le \frac{1}{p} x^p + \frac{1}{q} y^q \]
Proof 2. Fix \( y \in (0, \infty) \) and define \( f: (0, \infty) \to \R\) by \[ f(x) = \frac{1}{p} x^p + \frac{1}{q} y^q - x y, \quad x \in (0, \infty) \] Then \( f^\prime(x) = x^{p-1} - y\) and \( f^{\prime\prime}(x) = (p - 1) x^{p-2} \) for \( x \in (0, \infty) \). Hence \( f \) has a single critical point at \( x = y^{1/(p-1)} = y^{q/p} \) and \( f^{\prime\prime}(x) \gt 0 \) for \( x \in (0, \infty) \). It follows that the minimum value of \( f \) on \( (0, \infty) \) occurs at \( y^{q/p} \) and \(f\left(y^{q/p}\right) = 0\). Hence \( f(x) \ge 0 \) for \( x \in (0, \infty) \) with equality only at \( x = y^{q/p} \) (that is, \( x^p = y^q \)).
our next major result is Hölder's inequality, named for Otto Hölder, which clearly indicates the importance of conjugate indices.
Suppose that \( f, \, g: S \to \R \) are measurable and that \( p \) and \( q \) are conjugate indices. Then \[ \|f g\|_1 \le \|f\|_p \|g\|_q \]
The result is obvious if \( \|f\|_p = \infty \) or \( \|g\|_q = \infty \), so suppose that \( f \in L^p \) and \( g \in L^q \). For our first case, suppose that \( p = 1 \) and \( q = \infty \). Note that \( \left|g\right| \le \|g\|_\infty \) almost everywhere on \( S \). Hence \[ \int_S \left|fg\right| \, d\mu = \int_S \left|f\right| \left|g\right| \, d\mu \le \|g\|_\infty \int_S \left|f\right| \, d\mu = \|f\|_1 \|g\|_\infty \] For the second case, suppose \( p, \, q \in (1, \infty) \). By part (b) of the positive property , the result holds if \( \|f\|_p = 0 \) or \( \|g\|_q = 0 \), so assume that \( \|f\|_p \gt 0 \) and \( \|g\|_q \gt 0 \). By the additivity of the integral over disjoint domains, we can restrict the integrals to the set \( \{x \in S: f(x) \ne 0, g(x) \ne 0\} \), or simply assume that \( f \ne 0 \) and \( g \ne 0 \) on \( S \). From , \[ \left|f g\right| \le \frac{1}{p} \left|f\right|^p + \frac{1}{q} \left|g\right|^q \] Suppose first that \( \|f\|_p = \|g\|_q = 1 \). From the increasing and linearity properties of the integral, \[ \int_S \left|f g\right| \, d\mu \le \frac{1}{p} \int_S \left|f\right|^p \, d\mu + \frac{1}{q} \int_S \left|g\right|^q \, d\mu = \frac{1}{p} + \frac{1}{q} = 1 \] For the general case where \( \|f\|_p \gt 0 \) and \( \|g\|_q \gt 0 \), let \( f_1 = f \big/ \|f\|_p \) and \( g_1 = g \big/ \|g\|_q \). Then \( \left\|f_1\right\|_p = \left\|g_1\right\|_q = 1 \) so \( \left\|f_1 g_1 \right\|_1 \le 1 \). So by , \[ \left\|f_1 g_1\right\|_1 = \frac{\|f g\|_1}{\|f\|_p \|g\|_q} \le 1 \]
In particular, if \( f \in L^p \) and \( g \in L^q \) then \( f g \in L^1 \). The most important special case of Hölder's inequality is when \( p = q = 2 \), in which case we have the Cauchy-Schwartz inequality, named for Augustin Louis Cauchy and Karl Hermann Schwarz:
Suppose again that \(f, \, g: S \to \R\) are measurable. Then \[ \|f g\|_1 \le \|f\|_2 \|g\|_2 \]
Our next major result is Minkowski's inequality, named for Hermann Minkowski. This inequality will help show that \( L^p \) is a vector space and that \( \| \cdot \|_p \) is a norm (up to equivalence) when \( p \ge 1 \).
Suppose that \( f, \, g: S \to \R \) are measurable and that \( p \in [1, \infty] \). Then \[ \|f + g\|_p \le \|f\|_p + \|g\|_p \]
Again, the result is trivial if \( \|f\|_p = \infty \) or \( \|g\|_p = \infty \), so assume that \( f, \, g \in L^p \). When \( p = 1 \), the result is the simple triangle inequality for the integral: \[ \|f + g\|_1 = \int_S \left|f + g\right| \, d\mu \le \int_S \left(\left|f\right| + \left|g\right|\right) \, d\mu = \int_S \left|f\right| \, d\mu + \int_S \left|g\right| \, d\mu = \|f\|_1 + \|g\|_1 \] For the case \( p = \infty \), note that if \( a \in [0, \infty] \) is an essential bound for \( f \) and \( b \in [0, \infty] \) is an essential bound for \( g \) then \( a + b \) is an essential bound for \( f + g \). Hence \( \|f + g\|_\infty \le \|f\|_\infty + \|g\|_\infty \). For the last case, suppose that \( p \in (1, \infty) \) and let \( q \) be the index conjugate to \( p \). Then \[ \left|f + g\right|^p = \left|f + g\right|^{p-1} \left|f + g\right| \le \left|f + g\right|^{p-1}\left(\left|f\right| + \left|g\right|\right) = \left|f + g\right|^{p-1}\left|f\right| + \left|f+g\right|^{p-1} \left|g\right| \] Integrating over \( S \) and using the additive and increasing properties of the integral gives \[ \|f + g\|_p^p \le \int_S \left|f + g\right|^{p-1} \left|f\right| \, d\mu + \int_S \left|f + g\right|^{p-1} \left|g\right| \, d\mu\] But by Höder's inequality , \[ \int_S \left|f + g\right|^{p-1} \left|f\right| \, d\mu \le \|\left|f+g\right|^{p-1}\|_q \|f\|_p, \; \int_S \left|f + g\right|^{p-1} \left|g\right| \, d\mu \le \|\left|f+g\right|^{p-1}\|_q \|g\|_p\] Combining this with the previous inequality we have \[ \|f+g\|_p^p \le \|\left|f + g\right|^{p-1}\|_q \left(\|f\|_p + \|g\|_p\right) \] But \( (p - 1) q = p \) and \( 1/q = (p - 1) / p \) so \[ \|\left|f+g\right|^{p-1}\|_q = \left(\int_S \left|f + g\right|^{(p-1)q} \, d\mu \right)^{1/q} = \left(\int_S \left|f + g\right|^p \, d\mu\right)^{(p - 1)/p} = \|f + g\|_p^{p-1}\] Hence we have \[ \|f+g\|_p^p \le \|f + g\|_p^{p-1} \left(\|f\|_p + \|g\|_p\right) \] and therefore \( \|f + g\|_p \le \|f\|_p + \|g\|_p \).
We can now discuss various vector spaces of functions. First, we know from our previous work with measure spaces, that the set \( \ms V \) of all measurable functions \( f: S \to \R \) is a vector space under our standard (pointwise) definitions of sum and scalar multiple. The spaces we are studying in this section are subspaces:
\( L^p \) is a subspace of \( \ms V \) for every \( p \in [1, \infty] \).
However, we usually want to identify functions that are equal almost everywhere on \( S \) (with respect to \( \mu \)). Recalling the equivalence relation \( \equiv \) , here are the definitions:
Let \( [f] \) denote the equivalence class of \( f \in \ms V \) under the equivalence relation \( \equiv \), and let \( \ms U = \left\{[f]: f \in \ms V\right\} \). For \( f, \, g \in \ms V \) and \( c \in \R \) we define
Then \( \ms U \) is a vector space.
we know from our previous work that these definitions are consistent in the sense that they do not depend on the particular representatives of the equivalence classes. That is if \( f_1 \equiv f \) and \( g_1 \equiv g \) then \( f_1 + g_1 \equiv f + g \) and \( c f_1 \equiv c f \). That \( \ms U \) is a vector space then follows from the fact that \( \ms V \) is a vector space.
Now we can define the Lebesgue vector spaces precisely.
For \( p \in [1, \infty] \), let \( \ms L^p = \left\{[f]: f \in L^p\right\} \). For \( f \in \ms V \) define \( \left\|[f]\right\|_p = \|f\|_p \). Then \( \ms L^p \) is a subspace of \( \ms U \) and \( \| \cdot \|_p \) is a norm on \( \ms L^p \). That is, for \( f, g \in L^p \) and \( c \in \R \)
That \( \ms L^p \) is a subspace of \( \ms U \) follows immediately from the fact that \( L^p \) is a subspace of \( \ms V \). The fact that \( \| \cdot \| \) is a norm on \( \ms L^p \) also follows from our previous work.
We have stated these results precisely, but on the other hand, we don't want to be overly pedantic. It's more natural and intuitive to simply work with the space \( \ms V \) and the subspaces \( L^p \) for \( p \in [1, \infty] \), and just remember that functions that are equal almost everywhere on \( S \) are regarded as the same vector. This will be our point of view for the rest of this section, and so in particular we assume that \(p \in [1, \infty]\).
Every norm on a vector space naturally leads to a metric. That is, we measure the distance between vectors as the norm of their difference. Stated in terms of the norm \( \| \cdot \|_p \), here are the properties of the metric on \( L^p \) for \(p \in [1, \infty]\).
Suppose that \( f, \, g, \, h \in L^p \). Then
Once we have a metric, we naturally have a criterion for convergence.
Suppose that \( f_n \in L^p \) for \( n \in \N_+ \) and \( f \in L^p \). Then by definition, \( f_n \to f \) as \( n \to \infty \) in \( L^p \) if and only if \( \|f_n - f\|_p \to 0 \) as \( n \to \infty \).
Limits are unique, up to equivalence. (That is, limits are unique in \( \ms L^p \).)
Suppose again that \( f_n \in L^p \) for \( n \in \N_+ \). Recall that this sequence is said to be a Cauchy sequence if for every \( \epsilon \gt 0 \) there exists \( N \in \N_+ \) such that if \( m, \, n \gt N \) then \( \|f_n - f_m\|_p \lt \epsilon \). Needless to say, the Cauchy criterion is named for our ubiquitous friend Augustin Cauchy. A metric space in which every Cauchy sequence converges (to an element of the space) is said to be complete. Intuitively, one expects a Cauchy sequence to converge, so a complete space is literally one that is not missing any elements that should be there. A complete, normed vector space is called a Banach space, after the Polish mathematician Stefan Banach. Banach spaces are of fundamental importance in analysis, in large part because of the following result:
\( L^p \) is a Banach space for every \( p \in [1, \infty] \).
The norm \( \| \cdot \|_2 \) is special because it corresponds to an inner product.
For \( f, \, g \in L^2 \), define \[ \langle f, g \rangle = \int_S f g \, d\mu \]
Note that the integral is well defined by the Cauchy-Schwarz inequality . As with all of our other definitions, this one is consistent with the equivalence relation. That is, if \( f \equiv f_1 \) and \( g \equiv g_1 \) then \( f g \equiv f_1 g_1 \) so \( \int_S f g \, d\mu = \int_S f_1 g_1 \, d\mu \) and hence \( \langle f, g \rangle = \langle f_1, g_1 \rangle \). Note also that \( \langle f, f \rangle = \|f\|_2^2 \) for \( f \in L^2 \), so this definition generates the 2-norm.
\( L^2 \) is an inner product space. That is, if \( f, \, g, \, h \in L^2 \) and \( c \in \R \) then
Part (a) is a restatement of the positive property of the norm \( \| \cdot \|_2 \). Part (b) is obvious and parts (c) and (d) follow from the linearity of the integral.
From parts (c) and (d), the inner product is linear in the first argument, with the second argument fixed. By the symmetric property (b), it follows that the inner product is also linear in the second argument with the first argument fixed. That is, the inner product is bi-linear. A complete. inner product space is known as a Hilbert space, named for the German mathematician David Hilbert. Thus, the following result follows immediately from the previous two.
\( L^2 \) is a Hilbert space.
All inner product spaces lead naturally to the concept of orthogonality; \( L^2 \) is no exception.
Functions \( f, \, g \in L^2 \) are orthogonal if \( \langle f, g \rangle = 0 \), in which case we write \( f \perp g \). Equivalently \( f \perp g \) if \[ \int_S f g \, d\mu = 0 \]
Of course, all of the basic theorems of general inner product spaces hold in \( L^2 \). For example, the following result is the Pythagorean theorem, named of course for Phythagoras.
If \( f, \, g \in L^2 \) and \( f \perp g \) then \( \|f + g\|_2^2 = \|f\|_2^2 + \|g\|_2^2 \).
The proof just uses basic properties of inner product in . No special properties of \( L^2 \) are used. If \( f, \, g \in L^2 \) and \( f \perp g \) then \[ \|f + g\|^2 = \langle f + g, f + g \rangle = \langle f, f \rangle + 2 \langle f, g \rangle + \langle g, g \rangle = \| f \|^2 + \| g \|^2 \]
Recall again that the measure space \( (S, \ms S, \#) \) is discrete if \( S \) is countable, \( \ms S = \ms P(S)\) is the \( \sigma \)-algebra of all subsets of \( S \), and of course, \( \# \) is counting measure. In this case, recall that integrals are sums. The exposition will look more familiar if we use the notation of sequences rather than functions. Thus, let \( x: S \to \R \), and denote the value of \( x \) at \( i \in S \) by \( x_i \) rather than \( x(i) \). For \( p \in [1, \infty) \), the \( p \)-norm is \[ \|x\|_p = \left(\sum_{i \in S} \left|x\right|_i^p\right)^{1/p} \] On the other hand, \( \|x\|_\infty = \sup\{|x_i|: i \in S\} \). The only null set for \( \# \) is \( \emptyset \), so the equivalence relation \( \equiv \) is simply equality, and so the spaces \( L^p \) and \( \ms L^p \) are the same. For \( p \in [1, \infty) \), \( x \in L^p \) if and only if \[ \sum_{i \in S} \left|x\right|_i^p \lt \infty \] When \( p \in \N_+ \) (as is often the case), this condition means that \( \sum_{i \in S} x_i^p \) is absolutely convergent. On the other hand, \( x \in L^\infty \) if and only if \( x \) is bounded. When \( S = \N_+ \), the space \( L^p \) is often denoted \( l^p \). The inner produce on \( L^2 \) is \[ \langle x, y \rangle = \sum_{i \in S} x_i y_i, \quad x, \, y \in L^2 \] When \( S = \{1, 2, \ldots, n\} \), \( L^2 \) is simply the vector space \( \R^n \) with the usual addition, scalar multiplication, inner product, and norm that we study in elementary linear algebra. Orthogonal vector are perpendicular in the usual sense.
Suppose that \( (S, \ms S, \P) \) is a probability space so that \( S \) is the set of outcomes of a random experiment, \( \ms S \) is the \( \sigma \)-algebra of events, and \( \P \) is a probability measure on the sample space \( (S, \ms S) \). A measurable function \( X: S \to \R \) is a real-valued random variable. For \( p \in [1, \infty) \), the integral \( \int_S \left|x\right|^p \, d\P \) is the expected value of \( \left|X\right|^p \), and is denoted \(\E\left(\left|X\right|^p\right) \). So in this case, \( L^p \) is the collection of real-valued random variables \( X \) with \( \E\left(\left|X\right|^p\right) \lt \infty \). We will study these spaces in more detail in the chapter on expected value.