Introduction to the Central Limit Theorem

The Central Limit Theorem (CLT) is one of the most profound and practically useful results in all of statistics. It bridges the gap between probability theory and real-world inference, allowing us to draw conclusions about entire populations using only sample data , even when the shape of the population distribution is completely unknown.

Central Limit Theorem: For a population with mean $μ$ and standard deviation $σ$ , if repeated random samples of size $n$ are taken, then for sufficiently large $n$ , the distribution of the sample mean $\overset{ˉ}{X}$ approaches a normal distribution:
$\overset{ˉ}{X} \sim N (μ, \frac{σ ^{2}}{n})$
This holds regardless of the shape of the underlying population distribution.

The power of this theorem cannot be overstated: whether your population is skewed, uniform, bimodal, or any other shape, the distribution of sample means will tend toward the bell curve as sample size increases.

Analogy

Imagine rolling a single die , the outcomes are uniformly distributed (each face equally likely). Now imagine rolling 50 dice and recording their average. If you repeated this experiment thousands of times, the distribution of those averages would look remarkably bell-shaped, even though a single die is anything but normal. That's the CLT in action.

Linear Combinations of Normal Random Variables

Before fully appreciating the CLT, it helps to understand a key property of normal distributions that underpins it.

Note

Key Property: Any linear combination of independent normal random variables is itself exactly normally distributed , this is a precise result, not an approximation.

More precisely, if $X_{1}, X_{2}, \dots, X_{n}$ are independent normal random variables with $X_{i} \sim N (μ_{i}, σ_{i}^{2})$ , and we form:
$Y = a_{1} X_{1} + a_{2} X_{2} + \dots + a_{n} X_{n}$
then $Y$ is also normally distributed with:

Mean: $μ_{Y} = a_{1} μ_{1} + a_{2} μ_{2} + \dots + a_{n} μ_{n}$
Variance: $σ_{Y}^{2} = a_{1}^{2} σ_{1}^{2} + a_{2}^{2} σ_{2}^{2} + \dots + a_{n}^{2} σ_{n}^{2}$

Notice that the variances are always added (never subtracted), even when a coefficient is negative. This is because variance measures spread, which cannot cancel out.

Example

Let $X_{1} \sim N (10, 4)$ and $X_{2} \sim N (6, 9)$ be independent. Find the distribution of $Y = 2 X_{1} - 3 X_{2}$ .

Step 1 , Mean:
$μ_{Y} = 2 (10) - 3 (6) = 20 - 18 = 2$

Step 2 , Variance (note: coefficients are squared):
$σ_{Y}^{2} = (2)^{2} (4) + (- 3)^{2} (9) = 4 (4) + 9 (9) = 16 + 81 = 97$

Result: $Y \sim N (2, 97)$

Warning

When combining variances, always use $a_{i}^{2} σ_{i}^{2}$ , the coefficient is squared. A negative coefficient like $- 3$ becomes $(- 3)^{2} = 9$ when applied to the variance. Forgetting to square the coefficient is a very common error.

This property connects directly to the CLT. The sample mean $\overset{ˉ}{X} = \frac{1}{n} (X_{1} + X_{2} + \dots + X_{n})$ is itself a linear combination of the $X_{i}$ . This means:

If the population is normal, $\overset{ˉ}{X}$ is exactly normally distributed for any sample size $n$ , no approximation needed.
If the population is non-normal, the $X_{i}$ are not normal variables, so this linear combination property does not directly apply. The CLT steps in as an approximation result, telling us that $\overset{ˉ}{X}$ is approximately normal for large enough $n$ .

This is why we study linear combinations first: they explain the exact case, and the CLT extends the conclusion (approximately) to the general case.

Free preview

10 more sections in this topic

Introduction to the Central Limit Theorem

Analogy

Linear Combinations of Normal Random Variables

Before fully appreciating the CLT, it helps to understand a key property of normal distributions that underpins it.

Note

Key Property: Any linear combination of independent normal random variables is itself exactly normally distributed , this is a precise result, not an approximation.

Mean: $μ_{Y} = a_{1} μ_{1} + a_{2} μ_{2} + \dots + a_{n} μ_{n}$
Variance: $σ_{Y}^{2} = a_{1}^{2} σ_{1}^{2} + a_{2}^{2} σ_{2}^{2} + \dots + a_{n}^{2} σ_{n}^{2}$

Notice that the variances are always added (never subtracted), even when a coefficient is negative. This is because variance measures spread, which cannot cancel out.

Example

Let $X_{1} \sim N (10, 4)$ and $X_{2} \sim N (6, 9)$ be independent. Find the distribution of $Y = 2 X_{1} - 3 X_{2}$ .

Step 1 , Mean:
$μ_{Y} = 2 (10) - 3 (6) = 20 - 18 = 2$

Step 2 , Variance (note: coefficients are squared):
$σ_{Y}^{2} = (2)^{2} (4) + (- 3)^{2} (9) = 4 (4) + 9 (9) = 16 + 81 = 97$

Result: $Y \sim N (2, 97)$

Warning

This property connects directly to the CLT. The sample mean $\overset{ˉ}{X} = \frac{1}{n} (X_{1} + X_{2} + \dots + X_{n})$ is itself a linear combination of the $X_{i}$ . This means:

If the population is normal, $\overset{ˉ}{X}$ is exactly normally distributed for any sample size $n$ , no approximation needed.
If the population is non-normal, the $X_{i}$ are not normal variables, so this linear combination property does not directly apply. The CLT steps in as an approximation result, telling us that $\overset{ˉ}{X}$ is approximately normal for large enough $n$ .

This is why we study linear combinations first: they explain the exact case, and the CLT extends the conclusion (approximately) to the general case.

Free preview

AHL 4.15—Central limit theorem

Introduction to the Central Limit Theorem

Linear Combinations of Normal Random Variables

10 more sections in this topic

AHL 4.15—Central limit theorem

Introduction to the Central Limit Theorem

Linear Combinations of Normal Random Variables

10 more sections in this topic