Math Atom · M-C03

Variance and standard deviation

8 minMath AtomProbability

By the end you will be able to

  1. 1Explain spread around the mean, and why deviations are squared before averaging.
  2. 2Calculate the variance as the probability-weighted average squared deviation, and the standard deviation as its square root.
  3. 3Use the scaling rules: Var(aX + b) = a²Var(X) and sd(aX) = |a| sd(X).
  4. 4Say when to divide by N (a full list of outcomes) and when by n − 1 (estimating from data).

Why you need it

Two investments can both average 1% a month. One barely moves; the other swings from −4% to +6%. The average cannot tell them apart. The standard deviation can, and in finance it has its own name: volatility.

These Core Path lessons use it:

It builds on the mean from M-C02: Expectation, and leads to M-C06: The normal distribution, whose bell curve is set by a mean and a standard deviation.

The idea

How far from the average?

Here are six months of two illustrative return series. Both average 1% a month. But one is a gentle ride and the other a rollercoaster.

Steadymean 1% · sd 0.41%
-4%0%4%123456
Bumpymean 1% · sd 3.83%
-4%0%4%123456

Six months each, in per cent. Dashed line: the mean. Each stick is a deviation from the mean. Illustrative figures.

The difference is how far each month lands from the average: the sticks. Short sticks, low spread. Long sticks, high spread. To put one number on spread, we want a kind of average stick length. Each stick is a deviation, and the way we average them gives the variance and the standard deviation: about 0.41% for the steady series and 3.83% for the bumpy one.

The rules

We call the mean μ\mu (the Greek letter “mu”), as in M-C02: Expectation, and use the outcomes xix_i and probabilities pip_i from there.

Scaling. What happens to the spread if you double everything, or add a fixed amount?

The rules, for reference

Var⁡(X)=σ2=∑ipi (xi−μ)2σ=Var⁡(X)\operatorname{Var}(X) = \sigma^2 = \sum_i p_i\,(x_i - \mu)^2 \qquad \sigma = \sqrt{\operatorname{Var}(X)}
Var⁡(aX+b)=a2Var⁡(X)sd⁡(aX+b)=∣a∣ sd⁡(X)\operatorname{Var}(aX + b) = a^2\operatorname{Var}(X) \qquad \operatorname{sd}(aX + b) = |a|\,\operatorname{sd}(X)

A list of N equally likely values (the population formula): σ2=1N∑(xi−μ)2\sigma^2 = \tfrac1N\sum (x_i-\mu)^2. Estimating from a sample of n observations: divide by n − 1 instead. See M-D01: Estimating mean and volatility.

Another route to the same number

Expanding the square gives a shortcut that is handy with a calculator: the mean of the squares minus the square of the mean.

Var⁡(X)=E[X2]−μ2die: 15.1667−3.52=2.9167\operatorname{Var}(X) = \mathbb{E}[X^2] - \mu^2 \qquad \text{die: } 15.1667 - 3.5^2 = 2.9167

Try it

Five points, each equally likely. Drag them and watch the squares. Things to try: load “One outlier” and see how much of the variance one far-away point supplies. Move a point twice as far from the mean: its square gets four times bigger. Then set a to 2, or to −2, and compare the spread.

Load:

Drag a point along the line, or type its value (0 to 20).

05101520mean 11
Each coloured square has a side equal to one point's distance from the mean, so its area is that point's squared deviation. The variance is the average area. The dashed square has exactly that area: its side is the standard deviation σ.
Point xx − mean(x − mean)²Share
−3918%
−248%
−112%
000%
63672%
Total050

Standard deviation σ

3.162

Mean μ11
Variance σ² (÷ 5)10
Biggest share72%

Now scale every point by a and shift it by b. The spread follows a; b only slides the picture.

Scale a

1436mean 22
Var⁡(2X+0)=40=(2)2×10sd⁡=6.325=2×3.162\operatorname{Var}(2X + 0) = 40 = (2)^2 \times 10 \qquad \operatorname{sd} = 6.325 = 2 \times 3.162

Worked examples

1. Variance of a two-outcome return

Step 1 of 4
  1. 1

    The set-up

    Next year an investment returns 10% with probability 0.6, or −5% with probability 0.4. Work in per cent throughout.

2. Five months of data: N or n − 1?

Step 1 of 4
  1. 1

    The set-up

    A fund returned 2%, −1%, 3%, 0%, 1% in five months. First treat these five as the whole list, each equally likely.

3. From per cent to euros

Step 1 of 4
  1. 1

    The set-up

    A holding worth €200,000 has a daily return X with a standard deviation of 1.5%. The day’s profit or loss in euros is €200,000 × X. How spread out is it?

Common mistakes

Each line below is a tempting slip. Spot what is wrong, then check.

Reaction

Check what stuck

Question 1 · objective 1

Why are the deviations squared before they are averaged?

Question 2 · objective 2

X is 0 or 10, each with probability 0.5. What is Var(X)?

Question 3 · objective 3

Var(X) = 4. What is Var(3X + 5)?

Question 4 · objective 3

A return X has a standard deviation of 2%. What is the standard deviation of −5X, in %?

%

Question 5 · objective 4

You have 60 monthly returns of a fund and want to estimate how volatile the fund is. What do you divide the sum of squared deviations by?

Glossary

Deviation
How far an outcome is from the mean: x − μ. Positive above the mean, negative below.
Variance
The probability-weighted average of the squared deviations from the mean: Var(X) = Σ pᵢ(xᵢ − μ)², also written σ². Its units are the data's units squared.
Standard deviation
The square root of the variance, σ. In the same units as the data; read it as a typical distance from the mean.
Population formula
Treating a list as the whole set of equally likely outcomes, so each has probability 1/N and you divide by N.
Sample (n − 1) formula
Treating a list as a few observations from a bigger process. To estimate that process's spread you divide by n − 1.
Volatility
In finance, the standard deviation of an asset's returns. Estimated from data with n − 1.