A random sample is a sequence of independent, identically distributed (IID) random variables. The term random sample is ubiquitous in mathematical statistics while the abbreviation IID is just as common in basic probability, and thus this chapter can be viewed as a bridge between the two subjects. Much of basic probability and mathematical statistics deals with random variables constructed from random samples—the sample mean, sample variance, sample covariance, and order statistics are particularly important examples. Moreover, the two fundamental theorems of probability, the law of large numbers and the central limit theorem, concern the convergence of the sample mean.
Several topics in this chapter are explored from two points of view—first from a descriptive point of view (without the assumption that the data are generated from an underlying probability distribution) and the second from a probabilistic point of view (in which the variables are random variables). These topics include the sample mean, the sample variance, order statistics, and sample covariance, correlation and regression.
There are three kinds of lies: lies, damned lies, and statistics.—Attributed to Benjamin Disraeli in The Autobiography of Mark Twain by Samuel Clemens.