Eigenvalues and positive definiteness · RiskAtoms Academy

By the end you will be able to

1Explain an eigenvector and its eigenvalue as a direction that a matrix only stretches: Av = λv.
2Find the eigenvalues and eigenvectors of a 2 × 2 matrix by hand, and read those of a 3 × 3 matrix from a computation.
3Say what positive definite and positive semi-definite mean, test a matrix, and explain why a covariance matrix must be positive semi-definite.
4Diagnose a broken correlation matrix from a negative eigenvalue, and find the weights that would have negative variance.

Why you need it

Three correlations can each look reasonable and still be impossible together. You cannot see it by staring at the numbers. One smallest eigenvalue tells you at once.

This Core Path lesson uses it:

L3.1.06: Many assets and the covariance matrix
A covariance matrix must be positive semi-definite; its eigenvalues check that, and show when it is nearly singular.

You need quadratic forms from M-E02: Matrix multiplication and quadratic forms, the determinant from M-E03: Matrix inverse and linear systems, and correlation from M-C10: Covariance and correlation.

The idea

Directions that only stretch

A matrix moves vectors around. Feed it every arrow of length 1, a circle, and out comes an ellipse. Most arrows are turned as well as stretched. Turn the arrow x and watch for the moment Ax lines up with it.

Directions that only stretch

Turn x round the circle

Matrix A

Angle of x10°

x(0.985, 0.174)

Ax(2.143, 1.332)

Angle between x and Ax21.9°

Ax points in a different direction. Turn x until the two line up (the dashed lines).

Eigenvalues of this A: 3 and 1.

The directions that only stretch are the eigenvectors, and the stretch factors are the eigenvalues. For a symmetric matrix they are the axes of the ellipse, at right angles. With the third matrix one eigenvalue is negative: along that direction the arrow flips round.

The rules

Eigenvalues and eigenvectors. The definition, in one line.

2 × 2 by hand. The eigenvalues solve the characteristic equation, a quadratic. You will also meet the trace.

3 × 3 and bigger. The characteristic equation becomes a cubic, and beyond that there is no formula worth using. Software finds the eigenvalues with iterative methods that transform the matrix step by step (on this page, the Jacobi method, which uses repeated small rotations). You read the result, and check it: the eigenvalues still add up to the trace and multiply to the determinant.

Positive definite. A bowl-shaped quadratic form, from M-E02: Matrix multiplication and quadratic forms, in the language of eigenvalues.

Why a covariance matrix must be positive semi-definite. It comes straight from what wᵀΣw means.

The rules, for reference

A\mathbf{v} = \lambda\mathbf{v}, \qquad \det(A - \lambda I) = 0, \qquad \lambda = \tfrac{a+c}{2} \pm \sqrt{\big(\tfrac{a+c}{2}\big)^2 - (ac - b^2)}, \qquad \textstyle\sum \lambda_i = \operatorname{tr}A, \;\; \prod \lambda_i = \det A

\text{positive definite: all } \lambda_i > 0 \qquad \text{semi-definite: all } \lambda_i \ge 0 \qquad \Sigma \text{ valid} \Rightarrow \lambda_{\min}(\Sigma) \ge 0

Try it

Each correlation can be anything from −1 to 1 on its own. Together they must make a positive semi-definite matrix. Move the three sliders and watch the smallest eigenvalue. How low can ρ₁₃ go when ρ₁₂ and ρ₂₃ are both 0.9?

Try itIs this correlation matrix possible?

Covariance matrix toolkit →

ρ₁₂0.2

ρ₁₃0.4

ρ₂₃0.3

R = \begin{bmatrix} 1 & 0.2 & 0.4 \\ 0.2 & 1 & 0.3 \\ 0.4 & 0.3 & 1 \end{bmatrix}

Eigenvalues (they always add up to the trace, 3):

λ₁

1.607

λ₂

0.811

λ₃

0.581

Determinant (product of the eigenvalues)0.758

Verdictpositive definite

Valid: every eigenvalue is positive.

Worked examples

1. Eigenvalues and eigenvectors of a 2 × 2 matrix

Step 1 of 4

1
The set-up
$A = \begin{bmatrix} 5 & 2 \\ 2 & 2 \end{bmatrix}$

2. Testing two matrices

Step 1 of 4

1
The set-up
$B = \begin{bmatrix} 4 & 2 \\ 2 & 1 \end{bmatrix}, \qquad C = \begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix}$

3. Checking correlation matrices

Step 1 of 4

1
A valid one
The correlation matrix of the three illustrative funds in M-E01: Vectors and matrices, and its eigenvalues computed:
$R = \begin{bmatrix} 1 & 0.2 & 0.4 \\ 0.2 & 1 & 0.3 \\ 0.4 & 0.3 & 1 \end{bmatrix}, \qquad \lambda = 1.607,\ 0.811,\ 0.581$
All positive, and they add up to 3: R is positive definite. The covariance matrix Σ has eigenvalues 0.04696, 0.017, 0.00855, also all positive, with condition number 5.5: comfortably far from singular.

Common mistakes

Each line below is a tempting slip. Spot what is wrong, then check.

Reaction

Check what stuck

Question 1 · objective 2

What is the larger eigenvalue of the matrix with rows (3, 1) and (1, 3)?

Question 2 · objective 3

Is the matrix with rows (2, 3) and (3, 2) positive definite?

ANo: its determinant is −5, so one eigenvalue is negative.BYes: every entry is positive.CYes: the diagonal entries are positive.DIt cannot be told without a computer.

Question 3 · objective 2

A 3 × 3 correlation matrix has eigenvalues 1.8 and 0.9 and one more. What is the third?

Question 4 · objective 3

Why must every covariance matrix be positive semi-definite?

ABecause wᵀΣw is the variance of a weighted sum, and a variance cannot be negative.BBecause all covariances are positive.CBecause it is symmetric.DBecause its diagonal entries are positive.

Question 5 · objective 4

What is the smaller eigenvalue of the 2 × 2 correlation matrix with correlation 0.6?

Glossary

Eigenvector: A non-zero vector v that a matrix only stretches, shrinks or flips: Av = λv. Its direction does not change.
Eigenvalue (λ): The number λ in Av = λv: how much the matrix stretches its eigenvector (negative means it flips it).
Characteristic equation: The equation det(A − λI) = 0, whose solutions are the eigenvalues. For 2 × 2 it is a quadratic.
Trace: The sum of the diagonal entries of a square matrix. It equals the sum of the eigenvalues.
Positive definite: A symmetric matrix A with xᵀAx > 0 for every non-zero x; equivalently, all its eigenvalues are positive.
Positive semi-definite: A symmetric matrix A with xᵀAx ≥ 0 for every x; equivalently, none of its eigenvalues is negative.
Indefinite: A symmetric matrix with both positive and negative eigenvalues: its quadratic form is a saddle.
Correlation matrix: The matrix of all pairwise correlations, with 1 on the diagonal. A valid one is positive semi-definite.