In this section we will study a distribution, and some relatives, that have special importance in statistics. In particular, the chi-square distribution will arise in the study of the sample variance when the underlying distribution is normal and in goodness of fit tests.
Contents
The Chi-Square Distribution
Distribution Functions
Recall that the gamma function is the special function defined by
For , the gamma distribution with shape parameter and scale parameter 2 is called the chi-square distribution with degrees of freedom. The probability density function is given by
So the chi-square distribution is a continuous distribution on . For reasons that will be clear later, is usually a positive integer, although technically this is not a mathematical requirement. When is a positive integer, the gamma function in the normalizing constant can be be given explicitly.
If then
- if is even.
- if is odd.
The chi-square distribution has a rich collection of shapes.
The chi-square probability density function with degrees of freedom satisfies the following properties:
- If , is decreasing with as .
- If , is decreasing with .
- If , increases and then decreases with mode at .
- If , is concave downward.
- If , is concave downward and then upward, with inflection point at
- If then is concave upward then downward and then upward again, with inflection points at
Open the special distribution simulator and select the chi-square distribution. Keep the app open while you read this section. First, vary with the scrollbar and note the shape of the probability density function. For selected values of , run the simulation 1000 times and compare the empirical density function to the true probability density function.
The distribution function and the quantile function do not have simple, closed-form representations for most values of the parameter. However, the distribution function can be given in terms of the complete and incomplete gamma functions.
Suppose that has the chi-square distribution with degrees of freedom. The distribution function of is given by
Approximate values of the distribution and quantile functions can be obtained from most mathematical and statistical software packages.
In the quantile app, select the chi-square distribution. Vary the parameter and note the shape of the probability density, distribution, and quantile functions. In each of the following cases, find the median, the first and third quartiles, and the interquartile range.
Moments
The mean, variance, moments, and moment generating function of the chi-square distribution can be obtained easily from general results for the gamma distribution.
If has the chi-square distribution with degrees of freedom then
-
In the special distribution simulator, select the chi-square distribution. Vary with the scrollbar and note the size and location of the mean standard deviation bar. For selected values of , run the simulation 1000 times and compare the empirical moments to the distribution moments.
The skewness and kurtosis of the chi-square distribution are given next.
If has the chi-square distribution with degrees of freedom, then
Note that and as . In particular, the excess kurtosis as .
In the special distribution simulator, select the chi-square distribution. Increase with the scrollbar and note the shape of the probability density function in light of the results on skewness and kurtosis in [9]. For selected values of , run the simulation 1000 times and compare the empirical density function to the true probability density function.
The next result gives the general moments of the chi-square distribution.
If has the chi-square distribution with degrees of freedom, then for ,
In particular, if then
Note also if .
If has the chi-square distribution with degrees of freedom, then has moment generating function
Related Distributions
The chi-square distribution is connected to a number of other special distributions. Of course, the most important relationship is the definition—the chi-square distribution with degrees of freedom is a special case of the gamma distribution, corresponding to shape parameter and scale parameter 2. On the other hand, any gamma distributed variable can be re-scaled into a variable with a chi-square distribution.
If has the gamma distribution with shape parameter and scale parameter then has the chi-square distribution with degrees of freedom.
Details:
Since the gamma distribution is a scale family, has a gamma distribution with shape parameter and scale parameter . Hence has the chi-square distribution with degrees of freedom.
The chi-square distribution with 2 degrees of freedom is the exponential distribution with scale parameter 2.
Details:
The chi-square distribution with 2 degrees of freedom is the gamma distribution with shape parameter 1 and scale parameter 2, which we already know is the exponential distribution with scale parameter 2.
If has the standard normal distribution then has the chi-square distribution with 1 degree of freedom.
Details:
As usual, let and denote the PDF and CDF of the standard normal distribution, respectivley Then for ,
Differentiating with respect to gives the density function of :
which we recognize as the chi-square PDF with 1 degree of freedom.
Recall that if we add independent gamma variables with a common scale parameter, the resulting random variable also has a gamma distribution, with the common scale parameter and with shape parameter that is the sum of the shape parameters of the terms. Specializing to the chi-square distribution, we have the following important result:
If has the chi-square distribution with degrees of freedom, has the chi-square distribution with degrees of freedom, and and are independent, then has the chi-square distribution with degrees of freedom.
Theorems [15] and [16] lead to the following theorem, which is fundamentally important in statistics.
Suppose that and that is a sequence of independent standard normal variables. Then the sum of the squares
has the chi-square distribution with degrees of freedom:
Theorem [17] is the reason that the chi-square distribution deserves a name of its own, and the reason that the degrees of freedom parameter is usually a positive integer. Sums of squares of independent normal variables occur frequently in statistics.
From the central limit theorem and previous results for the gamma distribution, it follows that if is large, the chi-square distribution with degrees of freedom can be approximated by the normal distribution with mean and variance . Here is the precise statement:
If has the chi-square distribution with degrees of freedom, then the distribution of the standard score
converges to the standard normal distribution as .
In the special distribution simulator, select the chi-square distribution. Start with and increase . Note the shape of the probability density function in light of the previous theorem. For selected values of , run the experiment 1000 times and compare the empirical density function to the true density function.
Like the gamma distribution, the chi-square distribution is infinitely divisible:
Suppose that has the chi-square distribution with degrees of freedom. For , has the same distribution as , where is a sequence of independent random variables, each with the chi-square distribution with degrees of freedom.
Also like the gamma distribution, the chi-square distribution is a member of the general exponential family of distributions:
Suppose that has the chi-square distribution with with degrees of freedom. Then the distribution of is a one-parameter exponential family with natural parameter , and natural statistic .
Details:
This follows from the definition of the general exponential family. The PDF can be written as
The Chi Distribution
The chi distribution, appropriately enough, is the distribution of the square root of a variable with the chi-square distribution
Suppose that has the chi-square distribution with degrees of freedom. Then has the chi distribution with degrees of freedom.
So like the chi-square distribution, the chi distribution is a continuous distribution on .
Distribution Functions
The distribution function of the chi distribution with degrees of freedom is given by
Details:
Suppose that has the chi distribution with degrees of freedom so that has the chi-square distribution with degrees of freedom. For ,
where is the chi-square distribution function with degrees of freedom in [5].
The probability density function of the chi distribution with degrees of freedom is given by
Details:
Suppose again that has the chi distribution with degrees of freedom so that has the chi-square distribution with degrees of freedom. The transformation maps one-to-one onto . The inverse transformation is with . Hence by the standard change of variables formula,
where is the chi-square PDF with degrees of freedom in [1].
The chi probability density function also has a variety of shapes.
The chi probability density function with degrees of freedom satisfies the following properties:
- If , is decreasing with as .
- If , is decreasing with as .
- If , increases and then decreases with mode
- If , is concave upward.
- If , is concave downward and then upward with inflection point at
- If , is concave upward then downward then upward again with inflection points at
Moments
The raw moments of the chi distribution are easy to comput in terms of the gamma function.
Suppose that has the chi distribution with degrees of freedom. Then
Details:
By definition
The change of variables , so that and gives (after simplification)
The last integral is .
Curiously, the second moment is simply the degrees of freedom parameter.
Suppose again that has the chi distribution with degrees of freedom. Then
Details:
For part (b), using the fundamental identity of the gamma function we have
The other parts follow from direct substitution.
Related Distributions
The fundamental relationship of course is the one between the chi distribution and the chi-square distribution given in definition [22]. In turn, this leads to a fundamental relationship between the chi distribution and the normal distribution.
Suppose that and that is a sequence of independent variables, each with the standard normal distribution. Then
has the chi distribution with degrees of freedom.
Note that the random variable in the last result is the standard Euclidean norm of , thought of as a random vector in . Note also that the chi distribution with 1 degree of freedom is the distribution of , the absolute value of a standard normal variable, which is known as the standard half-normal distribution. The chi distribution with 2 degrees of freedom is the Rayleigh distribution, and the chi-square distribution with 3 dgrees of freedom is the Maxwell distribution.
The Non-Central Chi-Square Distribution
Much of the importance of the chi-square distribution stems from the fact that it is the distribution that governs the sum of squares of independent, standard normal variables. A natural generalization, and one that is important in statistical applications, is to consider the distribution of a sum of squares of independent normal variables, each with variance 1 but with different means.
Suppose that and that is a sequence of independent variables, where has the normal distribution with mean and variance 1 for . The distribution of is the non-central chi-square distribution with degrees of freedom and non-centrality parameter .
Note that the degrees of freedom is a positive integer while the non-centrality parameter , but we will soon generalize the degrees of freedom.
Distribution Functions
Like the chi-square and chi distributions, the non-central chi-square distribution is a continuous distribution on . The probability density function and distribution function do not have simple, closed expressions, but there is a fascinating connection to the Poisson distribution. To set up the notation, let and denote the probability density and distribution functions of the chi-square distribution with degrees of freedom. Suppose that has the non-central chi-square distribution with degrees of freedom and non-centrality parameter . The following fundamental theorem gives the probability density function of as an infinite series, and shows that the distribution does in fact depend only on and .
The probability density function of is given by
Details:
Suppose that is a sequence of independent random variables, where has the normal distribution with mean and variance 1, and where . So by definition, has the non-central chi-square distribution with degrees of freedom and non-centrality parameter . The random vector has a multivariate normal distribution with mean vector and variance-covariance matrix (the identity matrix). The (joint) PDF of is symmetric about : for . Because of this symmetry, the distribution of depends on only through the parameter . It follows that has the same distribution as where are independent, has the normal distribution with mean and variance 1, and are standard normal.
The distribution of is found by the usual change of variables methods. Let and denote the standard normal PDF and CDF, respectively, so that has CDF given by for . Thus,
Taking derivatives, the PDF of is given by
But for , so substituting and simplifying gives
Next, recall that the Taylor series for the hyperbolic cosine function is
which leads to
After a bit more algebra, we get the representation in the theorem, with . That is,
Or in functional form, .
To complete the proof, we know that has the chi-square distribution with degrees of freedom, and hence has PDF , and is independent of . Therefore the distribution of is
where denotes convolution as usual, and where we have used the fundamental result on the sum of independent chi-square variables in [17].
The function on is the probability density function of the Poisson distribution with parameter . So it follows that if has the Poisson distribution with parameter and the conditional distribution of given is chi-square with parameter , then has the distribution discussed here—non-central chi-square with degrees of freedom and non-centrality parameter . Moreover, it's clear that is a valid probability density function for any , so we can generalize our definition a bit.
For and , the distribution with probability density function in [30] is the non-central chi-square distribution with degrees of freedom and non-centrality parameter .
The distribution function is given by
Details:
This follows immediately from [30], since and .
Moments
In this discussion, we assume again that has the non-central chi-square distribution with degrees of freedom and non-centrality parameter .
The moment generating function of is given by
Details:
We will use the fundamental relationship in [30]. Thus, suppose that has the Poisson distribution with parameter , and that given , has the chi-square distribution with degrees of freedom. Conditioning and using [12] gives
The last expected value is the probability generating function of , evaluated at . Hence
The mean and variance of are
Details:
These results can be obtained by taking derivatives of the MGF, but the derivation using the connection with the Poisson distribution is more interesting. So suppose again that has the Poisson distribution with parameter and that the conditional distribution of given is chi-square with degrees of freedom. Conditioning and using the means and variances of the chi-square and Poisson distributions, we have
The skewness and kurtosis of are
Note that as or as . Note also that the excess kurtosis is . So (the kurtosis of the normal distribution) as or as .
Related Distributions
Trivially of course, the ordinary chi-square distribution is a special case of the non-central chi-square distribution, with non-centrality parameter 0. The most important relation is the orignal definition [29]. The non-central chi-square distribution with degrees of freedom and non-centrality parameter is the distribution of the sum of the squares of independent normal variables with variance 1 and whose means satisfy . The next most important relation is the one that arose in the probability density function and was so useful for computing moments. We state this one again for emphasis.
Suppose that has the Poisson distribution with parameter , where , and that the conditional distribution of given is chi-square with degrees of freedom, where . Then the (unconditional) distribution of is non-central chi-square with degree of freedom and non-centrality parameter .
Details:
For , let denote the chi-square PDF with degrees of freedom. Then from the assumptions, the PDF of is given by
which is the PDF of the non-central chi-square distribution with degrees of freedom and non-centrality parameter , derived above.
As the asymptotic results for the skewness and kurtosis suggest, there is also a central limit theorem.
Suppose that has the non-central chi-square distribution with degrees of freedom and non-centrality parameter . Then the distribution of the standard score
converges to the standard normal distribution as or as .
Computational Exercises
Suppose that a missile is fired at a target at the origin of a plane coordinate system, with units in meters. The missile lands at where and are independent and each has the normal distribution with mean 0 and variance 100. The missile will destroy the target if it lands within 20 meters of the target. Find the probability of this event.
Details:
Let denote the distance from the missile to the target.
Suppose that has the chi-square distribution with degrees of freedom. For each of the following, compute the true value using the quantile app and then compute the normal approximation. Compare the results.
- The 75th percentile of .
Details:
- ,
- ,