Math Atom · M-E02

Matrix multiplication and quadratic forms

10 minMath AtomLinear algebra

Know these first

By the end you will be able to

  1. 1Multiply a matrix by a vector, and two matrices together, row times column, and check that the sizes fit.
  2. 2Use the rules of matrix products: order matters, the identity changes nothing, and (AB)ᵀ = BᵀAᵀ.
  3. 3Write out and evaluate a quadratic form xᵀAx, and expand a 2 × 2 one into squares and a cross term.
  4. 4Compute the variance of a weighted sum as wᵀΣw, and split it into pieces that add up to the total.

Why you need it

The variance of a weighted sum of two things has three terms. Of ten things, a hundred. Matrix multiplication packs all of them into wᵀΣw, five symbols long.

These Core Path lessons use it:

You need vectors, matrices and the dot product from M-E01: Vectors and matrices, and the variance of a weighted sum from M-C11: Combining random variables.

The idea

Row times column

Multiplying matrices is one move done many times: take a row from the left matrix and a column from the right one, and take their dot product. Click any entry of the answer to see which row and column made it. Then swap the order.

Row times column

Click an entry of the answer

Multiply

A
20113−1
B
1240−21
=

Entry (1, 1) = row 1 of A · column 1 of B

2 × 1 + 0 × 4 + 1 × (−2) = 0

A has 3 columns and B has 3 rows, so the rows and columns pair up. The answer is 2 × 2.

Every entry is a dot product, so a row of the left matrix must be as long as a column of the right one. And because rows and columns swap roles when you swap the order, A B and B A are different calculations.

The rules

A matrix times a vector. The simplest case: each row of the matrix gives one entry of the answer.

A matrix times a matrix. The same move for every column of the right-hand matrix, with a size rule to check first.

Quadratic forms. Put a square matrix between a vector and itself and you get a quadratic form: one number, built from squares and cross terms.

The variance of a weighted sum. The formula from M-C11: Combining random variables is a quadratic form in disguise, with the covariance matrix in the middle.

The rules, for reference

(AB)ij=∑kaikbkj,(m×n)(n×p)→m×p,AB≠BA,(AB)T=BTAT(AB)_{ij} = \sum_{k} a_{ik} b_{kj}, \qquad (m \times n)(n \times p) \to m \times p, \qquad AB \ne BA, \qquad (AB)^{\mathsf T} = B^{\mathsf T}A^{\mathsf T}
xT ⁣[abbc] ⁣x=ax12+2b x1x2+c x22,Var⁡(wTX)=wTΣ w=∑iwi(Σw)i\mathbf{x}^{\mathsf T}\!\begin{bmatrix} a & b \\ b & c \end{bmatrix}\!\mathbf{x} = a x_1^2 + 2b\,x_1x_2 + c\,x_2^2, \qquad \operatorname{Var}(\mathbf{w}^{\mathsf T}\mathbf{X}) = \mathbf{w}^{\mathsf T}\Sigma\,\mathbf{w} = \sum_i w_i(\Sigma\mathbf{w})_i

Try it

The quadratic form of a 2 × 2 matrix gives a number at every point of the plane, so you can draw it as a map. Change a, b and c and watch the shape change. Can you make a saddle? What does it take to keep the whole map blue?

Try itQuadratic form map

Matrix calculator →
q(x) = xᵀAx over −2 ≤ x₁, x₂ ≤ 2: blue above zero, red below, with contour lines.

A = [2, 1; 1, 1]

a x₁²2
2b x₁x₂−3
c x₂²2.25
q = xᵀAx1.25

A bowl: positive everywhere except at 0. (M-E04 calls this positive definite.)

Worked examples

1. A B and B A

Step 1 of 4
  1. 1

    The set-up

    A=[120−134],B=[210−152]A = \begin{bmatrix} 1 & 2 & 0 \\ -1 & 3 & 4 \end{bmatrix}, \qquad B = \begin{bmatrix} 2 & 1 \\ 0 & -1 \\ 5 & 2 \end{bmatrix}

2. A quadratic form two ways

Step 1 of 4
  1. 1

    The set-up

    A=[3112],x=[2−1]A = \begin{bmatrix} 3 & 1 \\ 1 & 2 \end{bmatrix}, \qquad \mathbf{x} = \begin{bmatrix} 2 \\ -1 \end{bmatrix}

3. The variance of three funds as wᵀΣw

Step 1 of 4
  1. 1

    The set-up

    The three illustrative funds from M-E01: Vectors and matrices: weights w and covariance matrix Σ of annual returns, in the order A, B, C.

    w=[0.50.30.2],Σ=[0.040.0040.0120.0040.010.00450.0120.00450.0225]\mathbf{w} = \begin{bmatrix} 0.5 \\ 0.3 \\ 0.2 \end{bmatrix}, \qquad \Sigma = \begin{bmatrix} 0.04 & 0.004 & 0.012 \\ 0.004 & 0.01 & 0.0045 \\ 0.012 & 0.0045 & 0.0225 \end{bmatrix}

Common mistakes

Each line below is a tempting slip. Spot what is wrong, then check.

Reaction

Check what stuck

Question 1 · objective 1

A has rows (1, 2) and (3, 4). What is the second entry of A times the vector (1, 1)?

Question 2 · objective 1

A is 3 × 2 and B is 2 × 4. What is the size of AB?

Question 3 · objective 3

A has rows (2, 1) and (1, 3), and x = (1, 2). What is xᵀAx?

Question 4 · objective 2

A and B are both 2 × 2. Which statement is always true?

Question 5 · objective 4

Two variables have variances 0.04 and 0.09 and covariance 0.01. What is the variance of 0.5X₁ + 0.5X₂, computed as wᵀΣw?

Glossary

Matrix product
AB: the matrix whose entry (i, j) is row i of A dotted with column j of B. It exists only when A has as many columns as B has rows.
Size rule
Of sizes that fit for multiplication: (m × n) times (n × p). The inner sizes must match; the outer sizes give the answer.
Quadratic form
xᵀAx for a square matrix A: a single number, the sum of aᵢⱼ xᵢ xⱼ over all i and j.
Cross term
A term with two different variables multiplied, such as x₁x₂. In a quadratic form, the off-diagonal entries go with the cross terms.
Covariance matrix (Σ)
The square matrix Σ whose entry (i, j) is Cov(Xᵢ, Xⱼ): variances on the diagonal, covariances off it.

Back to top ↑