The normal distribution is perhaps the most important distribution in the study of mathematical statistics, in part because of the central limit theorem. As a consequence of this theorem, a measured quantity that is subject to numerous small, random errors will have, at least approximately, a normal distribution. Such variables are ubiquitous in statistical experiments, in subjects varying from the physical and biological sciences to the social sciences.
So in this section, we assume that is a random sample from the normal distribution with mean and standard deviation. Our goal is to construct confidence intervals for and individually, and then more generally, confidence sets for . These are among of the most important special cases of set estimation. A parallel section on Tests in the Normal Model is in Chapter 8 on hypothesis testing. First we need to review some basic facts that will be critical for our analysis.
From our study of point estimation, recall that is an unbiased and consistent estimator of while is an unbiased and consistent estimator of . From these basic statistics we can construct the pivot variables that will be used to construct our interval estimates. Recall the following special properties of the normal distribution:
It follows that each of these random variables is a pivot variable for since the distributions do not depend on the parameters, but the variables themselves functionally depend on one or both parameters. Pivot variables and will be used to construct interval estimates of while will be used to construct interval estimates of . To construct our estimates, we will need quantiles of these standard distributions. The quantiles can be computed using the quantile app or from most mathematical and statistical software packages. Here is the notation we will use:
Let and .
denotes the quantile of order for the standard normal distribution.
denotes the quantile of order for the student distribution with degrees of freedom.
denotes the quantile of order for the chi-square distribution with degrees of freedom
Since the standard normal and student distributions are symmetric about 0, it follows that and for and . On the other hand, the chi-square distribution is not symmetric.
Confidence Intervals for with Known
For our first discussion, we assume that the distribution mean is unknown but the standard deviation is known. This is not always an artificial assumption. There are often situations where is stable over time, and hence is at least approximately known, while changes because of different treatments. Examples are given in the computational exercises below. The pivot variable leads to confidence intervals for .
For ,
is a confidence interval for .
is a confidence lower bound for
is a confidence upper bound for
Details:
Since has the standard normal distribution, each of the following events has probability by definition of the quantiles:
In each case, solving the inequality for gives the result.
These are the standard interval estimates for when is known. The two-sided confidence interval in (a) is symmetric about the sample mean , and as the proof shows, corresponds to equal probability in each tail of the distribution of the pivot variable . But of course, this is not the only two-sided confidence interval; we can divide the probability anyway we want between the left and right tails of the distribution of .
For every , a confidence interval for is
gives the symmetric, equal-tail confidence interval.
gives the interval with the confidence upper bound.
gives the interval with the confidence lower bound.
Details:
From the normal distribution of and the definition of the quantile function,
The result then follows by solving for in the inequality.
In terms of the distribution of the pivot variable , as the proof shows, the two-sided confidence interval above corresponds to in the right tail and in the left tail. Next, let's study the length of this confidence interval.
For , the (deterministic) length of the two-sided confidence interval above is
is a decreasing function of , and as and as .
is a decreasing function of , and as .
is an increasing function of , and as and as .
As a function of , decreases and then increases, with minimum at the point of symmetry .
The last result shows again that there is a tradeoff between the confidence level and the length of the confidence interval. If and are fixed, we can decrease , and hence tighten our estimate, only at the expense of decreasing our confidence in the estimate. Conversely, we can increase our confidence in the estimate only at the expense of increasing the length of the interval. In terms of , the best of the two-sided confidence intervals (and the one that is almost always used) is symmetric, equal-tail interval with :
Use the mean estimation experiment to explore the procedure. Select the normal distribution and select normal pivot. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times. As the simulation runs, note that the confidence interval successfully captures the mean if and only if the value of the pivot variable is between the quantiles. Note the size and location of the confidence intervals and compare the proportion of successful intervals to the theoretical confidence level.
For the standard confidence intervals, let denote the distance between the sample mean and an endpoint. That is,
where for the two-sided interval and for the upper or lower confidence interval. The number is the margin of error of the estimate.
Note that is deterministic, and the length of the standard two-sided interval is . In many cases, the first step in the design of the experiment is to determine the sample size needed to estimate with a given margin of error and a given confidence level.
The sample size needed to estimate with confidence and margin of error is
Details:
This follows by solving for in the definition of above, and then rounding up to the next integer.
Note that varies directly with and with and inversely with . This last fact implies a law of diminishing return in reducing the margin of error. For example, if we want to reduce a given margin of error by a factor of , we must increase the sample size by a factor of 4.
Confidence Intervals for with Unknown
For our next discussion, we assume that the distribution mean and standard deviation are unknown, the usual situation. In this case, we can use the pivot variable, rather than the pivot variable, to construct confidence intervals for .
For ,
is a confidence interval for .
is a lower bound for
is a upper bound for
Details:
Since has the distribution with degees of freedom, each of the following events has probability , by definition of the quantiles:
In each case, solving for in the inequality gives the result.
These are the standard interval estimates of with unknown. The two-sided confidence interval in (a) is symmetric about the sample mean and corresponds to equal probability in each tail of the distribution of the pivot variable . As before, this is not the only confidence interval; we can divide between the left and right tails any way that we want.
For every , a confidence interval for is
gives the symmetric, equal-tail confidence interval.
gives the interval with the confidence upper bound.
gives the interval with the confidence lower bound.
Details:
Since has the student distribution with degrees of freedom, it follows from the definition of the quantiles that
The result then follows by solving for in the inequality.
The two-sided confidence interval above corresponds to in the right tail and in the left tail of the distribution of the pivot variable . Next, let's study the length of this confidence interal.
For , the (random) length of the two-sided confidence interval above is
is a decreasing function of , and as and as .
As a function of , decreases and then increases, with minimum at the point of symmetry .
Details:
Parts (a) and (b) follow from properties of the student quantile function . Parts (c) and (d) follow from the fact that has a chi distribution with degrees of freedom.
Once again, there is a tradeoff between the confidence level and the length of the confidence interval. If and are fixed, we can decrease , and hence tighten our estimate, only at the expense of decreasing our confidence in the estimate. Conversely, we can increase our confidence in the estimate only at the expense of increasing the length of the interval. In terms of , the best of the two-sided confidence intervals (and the one that is almost always used) is symmetric, equal-tail interval with . Finally, note that it does not really make sense to consider as a function of , since is a statistic rather than an algebraic variable. Similarly, it does not make sense to consider as a function of , since changing means new data and hence a new value of .
Use the mean estimation experiment to explore the procedure. Select the normal distribution and the pivot. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times. As the simulation runs, note that the confidence interval successfully captures the mean if and only if the value of the pivot variable is between the quantiles. Note the size and location of the confidence intervals and compare the proportion of successful intervals to the theoretical confidence level.
Confidence Intervals for
Next we will construct confidence intervals for using the pivot variable given in [2]
For ,
is a confidence interval for
is a confidence lower bound for
is a confidence upper bound for .
Details:
Since has the chi-square distribution with degrees of freedom, each of the following events has probability by definition of the quantiles:
In each case, solving for in the inequality give the result.
These are the standard interval estimates for . The two-sided interval in (a) is the equal-tail interval, corresponding to probability in each tail of the distribution of the pivot variable . Note however that this interval is not symmetric about the sample variance . Once again, we can partition the probability between the left and right tails of the distribution of any way that we like.
For every , a confidence interval for is
gives the equal-tail confidence interval.
gives the interval with the upper bound
gives the interval with the lower bound.
In terms of the distribution of the pivot variable , the confidence interval above corresponds to in the right tail and in the left tail. Once again, let's look at the length of the general two-sided confidence interval. The length is random, but is a multiple of the sample variance . Hence we can compute the expected value and variance of the length.
For , the (random) length of the two-sided confidence interval in the last theorem is
To construct an optimal two-sided confidence interval, it would be natural to find that minimizes the expected length. This is a complicated problem, but it turns out that for large , the equal-tail interval with is close to optimal. Of course, taking square roots of the endpoints of any of the confidence intervals for gives confidence intervals for the distribution standard deviation .
Use variance estimation experiment to explore the procedure. Select the normal distribution. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 time. As the simulation runs, note that the confidence interval successfully captures the standard deviation if and only if the value of the pivot variable is between the quantiles. Note the size and location of the confidence intervals and compare the proportion of successful intervals to the theoretical confidence level.
Confidence Sets for
In the discussion above, we constructed confidence intervals for and for separately (again, usually both parameters are unknown). In our next discussion, we will consider confidence sets for the parameter point . These sets will be subsets of the underlying parameter space .
Confidence Sets Constructed from the Pivot Variables
Each of the pivot variables , , and can be used to construct confidence sets for . In isolation, each will produce an unbounded confidence set, not surprising since, we are using a single pivot variable to estimate two parameters. We consider the normal pivot variable first.
For any , a level confidence set for is
The confidence set is a cone in the parameter space, with vertex at and boundary lines of slopes and
Details:
From the normal distribution of and the definition of the quantile function,
The result then follows by solving for in the inequality.
The confidence cone is shown in the graph below. (Note, however, that both slopes might be negative or both positive.)
The confidence set based on the normal pivot variable
The pivot variable leads to the following result:
For every , a level confidence set for is
Details:
From the student distribution of and the definition of the quanitle function,
The result now follows from solving for in the inequality.
The confidence set based on the pivot variable
By design, this confidence set gives no information about . Finally, the pivot variable leads to the following result:
For every , a level confidence set for is
Details:
From the chi-square distribution of and the definition of the quantile function,
The result then follows by solving for in the inequalilty.
The confidence set based on the pivot variable
By design, this confidence set gives no information about .
Intersections
We can now form intersections of some of the confidence sets constructed above to obtain bounded confidence sets for . We will use the fact that the sample mean and the sample variance are independent, one of the most important special properties of a normal sample. We will also need the result from the introduction on the intersection of confidence interals. In the following theorems, suppose that with .
The set is a conservative confidence sets for .
The confidence set
The set is a confidence set for .
The confidence set
It is interesting to note that the confidence set is a product set as a subset of the parameter space, but is not a product set as a subset of the sample space. By contrast, the confidence set is not a product set as a subset of the parameter space, but is a product set as a subset of the sample space.
Exercises
Robustness
The main assumption that we made was that the underlying sampling distribution is normal. Of course, in real statistical problems, we are unlikely to know much about the sampling distribution, let alone whether or not it is normal. When a statistical procedure works reasonably well, even when the underlying assumptions are violated, the procedure is said to be robust. In this subsection, we will explore the robustness of the estimation procedures for and .
Suppose in fact that the underlying distribution is not normal. When the sample size is relatively large, the distribution of the sample mean will still be approximately normal by the central limit theorem. Thus, our interval estimates of may still be approximately valid.
Use the simulation of the mean estimation experiment to explore the procedure. Select the gamma distribution and select student pivot. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times. Note the size and location of the confidence intervals and compare the proportion of successful intervals to the theoretical confidence level.
How large needs to be for the interval estimation procedures of to work well depends, of course, on the underlying distribution; the more this distribution deviates from normality, the larger must be. Fortunately, convergence to normality in the central limit theorem is rapid and hence, as you observed in the exercises, we can get away with relatively small sample sizes (30 or more) in most cases.
In general, the interval estimation procedures for are not robust; there is no analog of the central limit theorem to save us from deviations from normality.
In variance estimation experiment, select the gamma distribution. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times. Note the size and location of the confidence intervals and compare the proportion of successful intervals to the theoretical confidence level.
In variance estimation experiment, select the uniform distribution. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times. Note the size and location of the confidence intervals and compare the proportion of successful intervals to the theoretical confidence level.
Computational Exercises
In the following exercises, use the equal-tailed construction for two-sided confidence intervals, unless otherwise instructed.
The length of a certain machined part is supposed to be 10 centimeters but due to imperfections in the manufacturing process, the actual length is a normally distributed with mean and variance . The variance is due to inherent factors in the process, which remain fairly stable over time. From historical data, it is known that . On the other hand, may be set by adjusting various parameters in the process and hence may change to an unknown value fairly frequently. A sample of 100 parts has mean 10.2.
Construct the 95% confidence interval for .
Construct the 95% confidence upper bound for .
Construct the 95% confidence lower bound for .
Details:
10.25
10.15
Suppose that the weight of a bag of potato chips (in grams) is a normally distributed random variable with mean and standard deviation , both unknown. A sample of 75 bags has mean 250 and standard deviation 10.
Construct the 90% confidence interval for .
Construct the 90% confidence interval for .
Construct a conservative 90% confidence rectangle for .
Details:
At a telemarketing firm, the length of a telephone solicitation (in seconds) is a normally distributed random variable with mean and standard deviation , both unknown. A sample of 50 calls has mean length 300 and standard deviation 60.
Construct the 95% confidence upper bound for .
Construct the 95% confidence lower bound for .
Details:
314.3.
51.6.
At a certain farm the weight of a peach (in ounces) at harvest time is a normally distributed random variable with standard deviation 0.5. How many peaches must be sampled to estimate the mean weight with a margin of error and with 95% confidence.
Details:
25
The hourly salary for a certain type of construction work is a normally distributed random variable with standard deviation $1.25 and unknown mean . How many workers must be sampled to construct a 95% confidence lower bound for with margin of error $0.25?
Details:
68
Data Analysis Exercises
In Michelson's data, assume that the measured speed of light has a normal distribution with mean and standard deviation , both unknown.
Construct the 95% confidence interval for . Is the true value of the speed of light in this interval?
Construct the 95% confidence interval for .
Explore, in an informal graphical way, the assumption that the underlying distribution is normal.
Details:
. No, the true value is not in the interval.
In Cavendish's data, assume that the measured density of the earth has a normal distribution with mean and standard deviation , both unknown.
Construct the 95% confidence interval for . Is the true value of the density of the earth in this interval?
Construct the 95% confidence interval for .
Explore, in an informal graphical way, the assumption that the underlying distribution is normal.
Details:
. Yes, the true value is in the interval.
In Short's data, assume that the measured parallax of the sun has a normal distribution with mean and standard deviation , both unknown.
Construct the 95% confidence interval for . Is the true value of the parallax of the sun in this interval?
Construct the 95% confidence interval for .
Explore, in an informal graphical way, the assumption that the underlying distribution is normal.
Details:
. Yes, the true value is in the interval.
Suppose that the length of an iris petal of a given type (Setosa, Verginica, or Versicolor) is normally distributed. Use Fisher's iris data to construct 90% two-sided confidence intervals for each of the following parameters.