This text is a study of concepts in classical reliability theory generalized to very abstract settings. Concepts involving the reliability function, the failure rate function, a type of random walk, and constant rate distributions that ultimately depend only on the order relation \(\le\) are generalized to a set and a binary relation (referred to as a graph in this text). Other concepts that involve memoryless and exponential distributions that depend on the shift operator \(+\) are generalized to a semigropup (a set with an associative binary operator). Many of the classical results generalize to these abstract settings with only minimal additional assumptions. The Introduction reviews the classical theory and gives an overview of abstract theory.
Much of the underlying mathematics in the general theory is relatively simple and well known, corresponding to basic results in measure theory, linear algebra, functional analysis, and graph theory. But the application of the mathematics to the particular topics in probability theory presented here is not well known, to the best of my knowledge. But it should be. The theory is elegant and the applications interesting and diverse, even if they sometimes have little relation to the classical reliability theory that served as motivation.
My hope is that this text will be interesting and useful to students and researchers who study the interplay between probability and algebraic structures. The text includes some simple exercises and also includes a few problems that are interesting to me, but whose solutions I do not know. This text is currently a work in progress, and may well contain mistakes, hopefully mostly minor but pehaps some that are serious. I am grateful for comments and corrections.
The prerequisites for this text are basic topics in the set theory, combinatorics, linear algebra, measure theory and integration, probabiity theory, and certain stochastic processes. The companion web-book Random has all of these topics (and many more) written in a similar style, and with consistent notation and terminology. The following chapters include the topics that are the basic prerequisites:
In particular, the chapter on Foundations include basic facts about sets, functions and relations, measure spaces and integration, and topology
The following notation is used for special sets:
The following subsections review a few definitions and properties that will be most important in this text.
A measurable space \((S, \ms S)\) consists of a set \(S\) and a collection \(\ms S\) of subsets of \(S\) satisfying the following properties:
The collection \(\ms S\) is a \(\sigma\)-algebra and the sets in \(\ms S\) are measurable.
To avoid measurable spaces that are too sparce, we generally assume that \(\{x\} \in \ms S\) for \(x \in S\), and hence \(C \in \ms S\) for countable \(C \subseteq S\).
The intersection of a collection of \(\sigma\)-algebras on a set \(S\) is again a \(\sigma\)-algebra on \(S\). If \(\ms A\) is a collection of subsets of \(S\) then \[\sigma(\ms A) = \bigcap\{\ms S: \ms S \text{ is a } \sigma \text{-algebra of subsets of } S \text{ and } \ms A \subseteq \ms S\}\] is the \(\sigma\)-algebra generated by \(\ms A\).
So \(\sigma(\ms A)\) is the smallest \(\sigma\)-algebra containing \(\ms A\).
A measure space \((S, \ms S)\) has a measurable diagonal if \(\{(x, x): x \in S\} \in \ms S^2\).
In the language and notation of Chapter 1, measurable diagonal means that the basic graph \((S, =)\) is measurable. Spaces with measurable diagonals have a number of very nice properties. The most important one is given next.
Suppose that \((S, \ms S)\) and \((T, \ms T)\) are measurable spaces and that \((T, \ms T)\) has a measurable diagonal. If \(f: S \to T\) is measurable, then the graph of \(f\) is measurable: \[\{(x, f(x)): x \in S\} \in \ms S \times \ms T\]
For a proof, see the paper by Dravecký
A measure space \((S, \ms S, \mu)\) is \(\sigma\)-finite, if there exists a countable collection \(\{A_i: i \in I\}\) such that \(\bigcup_{i \in I} A_i = S\) and \(\mu(A_i) \lt \infty\) for \(i \in I\).
We generally assume that measure spaces are \(\sigma\)-finite. This assumption is needed for product measure spaces and in particular for Fubini's theorem, which we use often, and for the existence of density functions.
Suppose that \((S, \ms S, \mu)\) and \((T, \ms T, \nu)\) are \(\sigma\)-finite measure spaces.
The product structure extends to a finite collection of measure spaces, of course. In particular, if \(n \in \N_+\) and \((S, \ms S, \mu)\) is a \(\sigma\)-finite measure space, then \((S^n, \ms S^n, \mu^n)\) is the power measure space of order \(n\).
We try to avoid topological assumptions unnecessarily. Our point of view is that there is no need to invoke topolgy unless there is a real topological concept at play—continuity or convergence, for example. It seems a bit remarkable (to me at least) that most of the basic theory in this text flows from the basic assumptions of product measurablility and \(\sigma\)-finite measures.
If \(S\) is a countable set and \(\ms S\) the collection of all subsets of \(S\) then \((S, \ms S)\) is a discrete measurable space. Counting measure \(\#\) is the standard reference measure for such spaces, so \((S, \ms S, \#)\) is a discrete measure space.
We use the terms integrable in the weak sense and absolutely integrable in the strong sense.
Suppose that \((S, \ms S, \mu)\) is a \(\sigma\)-finite measure space and that \(f: S \to \R\) is measurable. Let \(f^+\) and \(f^-\) denote the postive and negative parts of \(f\).
So if \(f\) is integrable then \(\int_S f(x) \, d\mu(x)\) exists in \(\R \cup \{-\infty, \infty\}\) and if \(f\) is absolutely integrable then \(\int_S f(x) \, d\mu(x)\) exists in \(\R\).
Suppose again that \((S, \ms S, \mu)\) is a \(\sigma\)-finite measure space and \(k \in [1, \infty)\). If \(f: S \to \R\) is measurable, define the norm of order \(k\) by \[\|f\|_k = \left[\int_S |f|^k \, d\mu(x)\right]^{1/k}\] \(\ms L^k(S, \ms S, \mu)\) denotes the usual Banach space of measurable functions \(f: S \to \R\) with norm \(\|f\|_k \lt \infty\).
If the measure space is understood we abbreviate the notation to \(\ms L^k\).
Recall that a topological space \((S, \ms U)\) consists of a set \(S\) and a collection of subsets \(\ms U\) of \(S\) satisfying the following properties:
The collection \(\ms U\) is the topology and sets in \(\ms U\) are open. The complement of an open set is closed.
So the open sets are preserved under arbitrary unions and finite interesections. Topology is intimately conncectd with measure theory:
If \((S, \ms U)\) is a topological space then \(\ms S = \sigma(\ms U)\) is the Borel \(\sigma\)-algebra on \(S\).
Suppose that \((S, \ms U)\) is a topological space. A collection of subsets \(\ms B \subseteq \ms U\) is a base for the topology if every set in \(\ms U\) can be written as a union of sets in \(\ms B\).
Bases are useful for constructing topologies with a given collection of open sets.
A collection of subsets \(\ms B\) of \(S\) is a base for a topology on \(S\) if and only if the following conditions are satisfied.
Suppose that \((S, \ms U)\) and \((T, \ms V)\) are topological spaces. Then \(\{U \times V: U \in \ms U, V \in \ms V\}\) is a base for a topology on \(S \times T\), called the product topology and denoted \(\ms U \times \ms V\).
Once again, the product topology \(\ms U \times \ms V\) is not to be confused with an ordinary Cartesian product.
Suppose that \((S, \ms U)\) is a topological space. Recall that \(A \subseteq S\) is a neighborhood of \(x \in S\) if there exists an open set \(U \in \ms U\) with \(x \in U \subseteq A\).
A topological space that is locally compact, Hausdorff, and has a countable base is an LCCB space.
An LCCB space has a number of nice properties. The space is metrizable so that there exists a metric that generates the topology. In turn, this means that continuity and other convergence properties can be formulated in terms of convergent sequences. With the Borel \(\sigma\)-algebra, the space has a measurable diagonal so our fundamental assumptions are consistent. With a topological space, we can impose additional conditions on a measure:
Suppose that \(S\) has an LCCB topology and that \(\ms S\) is the corresponding Borel \(\sigma\) algebra. A measure \(\mu\) on \((S, \ms S)\) is regualr if for \(A \in \ms S\) with \(\mu(A) \lt \infty\),
If \(S\) is countable and we give \(S\) the discrete topology, so that every subset is open (and closed), then the Borel \(\sigma\)-algebra \(\ms S\) is the collection of all subsets of \(S\). So \((S, \ms S)\) is a discrete measurable space, as above. Counting measure is trivially regular. Besides discrete spaces, here is another basic class of spaces:
For \(n \in \N_+\), the \(n\)-dimensional Euclidean measurable space is \((\R^n, \ms R^n)\) where \(\ms R^n\) is the Borel \(\sigma\)-algebra of subsets of \(\R^n\) under the standard Euclidean topology. The standard reference measure is \(n\)-dimensional Lebesge measure \(\lambda^n\) so \((\R^n, \ms R^n, \lambda^n)\) is the \(n\)-dimensional Euclidean measure space.
Of course, \(\R^n\) with the standard Euclidean topolgy is an LCCB space for \(n \in \N_+\), and Lebesgue measure \(\lambda^n\) is regular. When \(n = 1\) we drop the superscript. Often our interest is in a subspce \((S, \ms S, \mu)\) where \(S \in \ms R^n\) for some \(n \in N_+\) with \(\lambda^n(S) \gt 0\), \(\ms S = \{A \in \ms R^n: A \subseteq S\}\) and \(\mu\) is \(\lambda^n\) restricted to \(\ms S\).
A topological space \((S, \ms U)\) is Hausdorff if and only if the diagonal \(D = \{(x, x): x \in S\}\) is closed in \(S^2\) (with the product topology).
This is a well-known result, but we give the proof since it's so simple. Note first that if \(U, \, V \subseteq S\) then \(U\) and \(V\) are disjoint if and only if \(U \times V\) and \(D\) are disjoint. Suppose that the space is Hausdorff and let \((x, y) \in D^c\) so that \(x \ne y\). Then there exist disjoint open sets \(U, \, V \subseteq S\) with \(x \in U\) and \(y \in V\). Hence \((x, y) \in U \times V \subseteq D^c\). Since \(U \times V\) is open in \(S^2\) with the product topology, it follows that \(D^c\) is open and hence that \(D\) is closed. Conversely, suppose that \(D\) is closed in \(S^2\) with the product topology. Let \(x, \, y \in S \) with \(x \ne y\), so that \((x, y) \in D^c\). The set \(D^c\) is open in the product topology and sets of the form \(U \times V\) with \(U, \, V\) open in \(S\) form a basis for this topology. Hence there exist \(U, \, V\) open in \(S\) with \((x, y) \in U \times V \subseteq D^c\). Hence \(x \in U\), \(y \in V\) and \(U\), \(V\) are disjoint and so the space is Hausdorff.
So for a Hausdorff space, the diagonal is in the Borel \(\sigma\)-algebra corresponding to the product topolgy. But this may not be the product of the Borel \(\sigma\)-algebras. Here is the missing piece:
Suppose that \((S, \ms U)\) is a separable, metrizable topological space. Then \(\sigma\left(\ms U^2\right) = [\sigma(\ms U)]^2\).
So if follows that in a topological space that is separable and metrizable, the diagonal is measurable. In particular, an LCCB space has a measurable diagonal.
We use the term random variable in the general sense.
If \((\Omega, \ms F, \P)\) is a probability space and \((S, \ms S)\) another measurable space, then a measurable function \(X: \Omega \to S\) is a random variable in \(S\). The probability distribution of \(X\) is the probability measure \(P\) on \((S, \ms S)\) defined by \[ P(A) = \P(X \in A), \quad A \in \ms S \]
Here is another useful expression that we will use frequently:
Suppose that \(X\) is a random variable in \(S\). Then \(\bs{X} = (X_1, X_2, \ldots)\) is a sequence of independent copies of \(X\) if \(\bs{X}\) is an indpendent sequence and \(X_i\) has the same distribution as \(X\) for \(i \in \N_+\).
The text is divided into chapters, and each chapter in turn is divided into sections. Each section is a separate web page. Definitions, mathematical statements, exercises, examples, and open problems are indicated by a die icon and are numbered consecutively in each section. Definitions are indicated by a green die, mathematical statements by a blue die, and simulation exercises by a red die. For mathematical statements, I do not bother to distinguish between theorems, propositions, lemmas, or corollaries. Proofs (or partial proofs) of statements, and solutions or answers to exercises, are referred to as details. These are initially hidden, but the details for a mathematical item can be expanded or contracted by clicking on the small triangle that accompanies the item. My hope is that this feature will make it easier to browse the exposition without getting too involved in the details, and will encourage the reader to think about the proof of a statement or try an exercise before looking at the details in the text. All of the details in a section can be expaned or contracted by clicking on the plus or minus icons at the top and bottom of a page. Links to topics in other sections are clearly indicated by referencing the section or chapter number. You may wish to open a page of this type in a separate browser tab to avoid the delay in rendering the mathematics that occurs when you go back and forth between pages. Links to topics in external sites always open in separate browser tabs.
The apps in this project are designed to demonstrate the mathematical theory in a dynamic, interactive way. A standard Graphical User Interface (GUI) is used, with command buttons, scroll bars and list boxes. The app output is displayed numerically and graphically in a set of coordinated tables and graphs. A consistent color-coding is used. Graphical objects that depend only on the distributions or parameters are shown in blue, while graphical objects that depend on data are shown in red. Most app objects have tool tips, small pop-up boxes that explain the object. Rest the cursor on an object to display the tool tip.
Apps that are simulations of random processes all have a standard toolbar with the following basic buttons and controls:
The stop frequency is selected from the second list box in the main toolbar. The stop frequency is the number of runs before the simulation stops in run mode. In most apps you can select a stop frequency of 10, 100, 1000, or 10000. In some apps, other stop rules are provided. In addition, most apps have one or more parameters that can be varied, usually with a scrollbar. The controls for varying parameters appear on toolbar.