Matrix inverse and linear systems · RiskAtoms Academy

By the end you will be able to

1Write a set of linear equations as one matrix equation Ax = b.
2Invert a 2 × 2 matrix with the determinant formula, check that A⁻¹A = I, and know when no inverse exists.
3Solve 2 × 2 and 3 × 3 systems, by the inverse and by elimination, and check the answer.
4Recognise a singular or nearly singular system and explain why its answers are fragile.

Why you need it

Many optimisation problems end the same way: a set of linear equations, one for each unknown, that must all hold at once. Solving them is the last step, and knowing when the answer can be trusted is part of it.

These Core Path lessons use it:

L3.1.07: Mean-variance optimisation
The optimal weights come out of a linear system with the covariance matrix: in matrix form, its inverse times a vector.
L3.1.15: Estimation error, shrinkage and Black-Litterman
A nearly singular covariance matrix magnifies small estimation errors into large swings in the solved weights.

You need rearranging from M-A02: Rearranging equations and the matrix product from M-E02: Matrix multiplication and quadratic forms.

The idea

Where two lines cross

An equation such as 2x + y = 5 has many solutions: every point on a line. A second equation is a second line. A system of both asks for the points on both lines at once.

Two equations, two lines

Pick a case

Case

2x + y = 5

x + 3y = 10

det A = ad − bc5

Solutionx = 1, y = 3

The lines cross once: exactly one solution.

That is the whole story for two unknowns: one crossing, none, or the same line twice. With three unknowns each equation is a flat plane, and three planes usually meet at one point. The algebra that tells the cases apart is the same in any size: the determinant and the inverse.

The rules

A system as one equation. Any set of linear equations can be written as a matrix times the vector of unknowns.

The inverse. For 2 × 2 there is a short formula; for anything bigger, a computer does it.

Elimination. The hand method for 3 × 3: clear the unknowns one column at a time, then substitute back.

When the answer is fragile. A singular matrix has no inverse. A nearly singular one has an inverse, but a fragile one.

The rules, for reference

A\mathbf{x} = \mathbf{b}, \qquad A^{-1}A = AA^{-1} = I, \qquad \mathbf{x} = A^{-1}\mathbf{b}

\begin{bmatrix} a & b \\ c & d \end{bmatrix}^{-1} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}, \qquad \det A = 0 \;\Leftrightarrow\; \text{no inverse}, \qquad (AB)^{-1} = B^{-1}A^{-1}

Try it

Set the two equations with the sliders, or start from a preset. Then press “Nudge” to move the second right-hand side by 0.1 and see how far the solution travels. Compare a clear crossing with nearly parallel lines.

Try itSolve and nudge

Matrix calculator →

a (x in eq. 1)2

b (y in eq. 1)1

e (right side 1)5

c (x in eq. 2)1

d (y in eq. 2)3

f (right side 2)10

2x + 1y = 5

1x + 3y = 10

det A = ad − bc5

A⁻¹[0.6, −0.2; −0.2, 0.4]

Solution (x, y)(1, 3)

Worked examples

1. A 2 × 2 system by the inverse

Step 1 of 4

1
The set-up
$\begin{aligned} 2x + \phantom{3}y &= 5 \\ x + 3y &= 10 \end{aligned} \qquad\Longleftrightarrow\qquad \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}\begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 5 \\ 10 \end{bmatrix}$

2. A 3 × 3 system by elimination

Step 1 of 4

1
The set-up
$\begin{aligned} x + \phantom{3}y + \phantom{4}z &= 6 \\ 2x + 3y + \phantom{4}z &= 11 \\ x + 3y + 4z &= 19 \end{aligned} \qquad \left[\begin{array}{ccc|c} 1 & 1 & 1 & 6 \\ 2 & 3 & 1 & 11 \\ 1 & 3 & 4 & 19 \end{array}\right]$

3. Three funds: solving with the covariance matrix

Step 1 of 4

1
The set-up
For the three illustrative funds of M-E01: Vectors and matrices, the weights with the smallest variance come from a linear system: solve Σz = 1, with 1 a vector of ones, then rescale z so its entries add up to 1. (The Lagrange method of M-B04: Optimisation and Lagrange multipliers leads to this system; L3.1.07: Mean-variance optimisation does the finance.)
$\begin{bmatrix} 0.04 & 0.004 & 0.012 \\ 0.004 & 0.01 & 0.0045 \\ 0.012 & 0.0045 & 0.0225 \end{bmatrix}\begin{bmatrix} z_A \\ z_B \\ z_C \end{bmatrix} = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}$

Common mistakes

Each line below is a tempting slip. Spot what is wrong, then check.

Reaction

Check what stuck

Question 1 · objective 2

What is the determinant of the matrix with rows (4, 3) and (2, 5)?

Question 2 · objective 2

Same matrix. What is the top-left entry of its inverse, to three decimals?

Question 3 · objective 3

Solve x + y = 10 and x − y = 2. What is x?

Question 4 · objective 4

A 2 × 2 matrix A has det A = 0. What can you say about Ax = b?

AThere is either no solution or infinitely many, depending on b.BThere is exactly one solution, x = 0.CThere is exactly one solution, but it is hard to compute.DThe solution is x = b.

Question 5 · objective 4

Two equations give lines that cross at a very shallow angle. What happens if b changes slightly?

AThe solution can move a long way: the system is ill-conditioned.BThe solution hardly moves, because the lines still cross.CThe solution disappears.DThe determinant becomes negative.

Glossary

Linear equation: An equation in which each unknown appears only multiplied by a number and added: no squares, products of unknowns or other functions.
System of equations: Several equations that must all hold at once. A solution is a set of values for the unknowns that satisfies every one.
Determinant: A single number computed from a square matrix; for 2 × 2, det A = ad − bc. Zero means the matrix has no inverse.
Inverse matrix: For a square matrix A, the matrix A⁻¹ with A⁻¹A = AA⁻¹ = I. It exists only when det A ≠ 0.
Singular matrix: A square matrix with determinant zero. It has no inverse, and Ax = b has either no solution or infinitely many.
Elimination: Solving a system by subtracting multiples of one equation from another until the unknowns can be found one at a time.
Condition number: The condition number κ: how much a matrix can magnify small relative changes in b into changes in the solution. Near 1 is stable; very large is fragile.