In this chapter, we study several general families of probability distributions and a number of special parametric families of distributions. Unlike the other expository chapters in this text, the sections are not linearly ordered so this chapter serves primarily as a reference. You may want to study these topics as the need arises.
First, we need to discuss what makes a probability distribution special in the first place. In some cases, a distribution may be important because it is connected with other special distributions in interesting ways (via transformations, limits, conditioning, etc.). In some cases, a parametric family may be important because it can be used to model a wide variety of random phenomena. This may be the case because of fundamental underlying principles, or simply because the family has a rich collection of probability density functions with a small number of parameters (usually 3 or less). As a general philosophical principle, we try to model a random process with as few parameters as possible; this is sometimes referred to as the principle of parsimony of parameters. In turn, this is a special case of Ockham's razor, named in honor of William of Ockham, the principle that states that one should use the simplest model that adequately describes a given phenomenon. Parsimony is important because often the parameters are not known and must be estimated.
In many cases, a special parametric family of distributions will have one or more distinguished standard members, corresponding to specified values of some of the parameters. Usually the standard distributions will be mathematically simplest, and often other members of the family can be constructed from the standard distributions by simple transformations on the underlying standard random variable.
An incredible variety of special distributions have been studied over the years, and new ones are constantly being added to the literature. To truly deserve the adjective special, a distribution should have a certain level of mathematical elegance and economy, and should arise in interesting and diverse applications.
These general families can be thought of as distributions parameterized by functions, sequences, or other distributions. Location-scale families are fundamental and often correspond to a change of units. General exponential families are important in inferential statistics. Stable and infinitely divisible distributions are more theoretical topics. Power series distributions include many of the most famous special discrete distributions.
The normal distribution is of fundamental importance in probability, mathematical statistics, and stochastic processes.
These distributions are important for inferential statistics.
Each of these distributions can be obtained from independent normally distributed variables by simple transformations.
The beta is the most important family of distributions with a bounded support interval.
Uniform distributions are basic, and can be defined in a number of settings, from simple to abstract.
These are distributions with bounded support that are based on simple curves such as triangles, circles and polynomial graphs.
These are other continuous distributions with support \( \R \).
These are other continuous distribution with support on a subset of \( [0, \infty) \).
These are simple discrete distributions.
There are several other parametric families of distributions that are studied elsewhere in this text, because the natural home for these distributions are various random processes.
Beauty is the first test: there is no permanent place in the world for ugly mathematics.—GH Hardy, A Mathematician's Apology.