Many assets and the covariance matrix

RiskAtoms Group's £1bn portfolio spreads across five asset classes. Its volatility, 9.50%, comes from a sum of 25 terms. A bank's trading book with 3,000 risk factors needs nine million terms and 4,501,500 different inputs, and those inputs must fit together, or the model will report that some portfolio has negative risk.

Weights and returns as vectors

With two assets, L3.1.01: Portfolio return and risk wrote everything out term by term. With five, or five hundred, that becomes unreadable. Instead, line the numbers up as vectorsAn ordered list of numbers, written as a column. Here: one weight, or one expected return, per asset. (M-E01: Vectors and matrices): a column $w$ of weights and a column $\mu$ of expected returns, one entry per asset. A prime turns a column into a row. Then the expected return and the budget are each one short product:

For the group, with the weights as a row and the expected returns as a column:

The covariance matrix

Risk needs more than one number per asset. It needs every variance and every covariance, and they fit in a square table: the covariance matrixThe n × n table of every variance (on the diagonal) and every covariance (off it) of n assets' returns. Symbol Σ. $\Sigma$. The entry in row $i$ and column $j$ is $\sigma_{ij}$, the covariance of assets $i$ and $j$. On the diagonal, an asset's covariance with itself is its variance, $\sigma_{ii} = \sigma_i^2$. Because $\sigma_{ij} = \sigma_{ji}$, the matrix is symmetric: the triangle above the diagonal mirrors the one below. For three of the group's assets (global equities, emerging-market equities, government bonds), in full:

For all five, a table is easier to read. Entries are in %², so 256.0 means 0.0256, a volatility of 16%.

Portfolio variance is w′Σw

Each pair of assets adds $w_i\,w_j\,\sigma_{ij}$ to the variance. Lay those terms out in the same square as $\Sigma$ and you get a variance grid with one cell per pair: the variance is the sum of every cell. In matrix notation that sum is a quadratic formAn expression w′Aw: a vector times a square matrix times the same vector, giving one number. (M-E02: Matrix multiplication and quadratic forms):

For the group's five assets the grid has 25 cells: 5 on the diagonal, which add 47% of the variance, and 20 off it (10 pairs, each twice), which add the other 53%. If every correlation were zero, only the diagonal would be left and the volatility would be 6.50%. The real figure is 9.50%, because most of the pairs move together. That is still far below the weighted average of the five volatilities, 12.70%, which is what every correlation equal to +1 would give.

Σ = DRD: volatilities and correlations

People think in volatilities and correlations, not covariances. Put the volatilities on the diagonal of a matrix $D$ and the correlations in the correlation matrixThe n × n table of correlations, with ones on the diagonal. It is the covariance matrix of the returns divided by their volatilities. $R$. Then $\Sigma = DRD$: multiplying by $D$ on the left scales each row by $\sigma_i$, and on the right scales each column by $\sigma_j$, so every entry becomes $\sigma_i\rho_{ij}\sigma_j$. For the three assets:

Check one entry: global and emerging-market equities, $0.16 \times 0.75 \times 0.22 = 0.0264$. The split matters in practice: volatilities move quickly and are re-estimated often, while correlations are usually set more slowly, and stress tests often change one without the other.

How many numbers?

A symmetric $n \times n$ matrix has $n$ diagonal entries and $n(n-1)/2$ distinct pairs, so it needs $\tfrac{n(n+1)}{2}$ numbers. That grows with the square of $n$:

The inputs have to come from data, and data runs out. A covariance matrix estimated from $T$ observations can have at most $T - 1$ eigenvalues above zero (M-E04: Eigenvalues and positive definiteness). With 500 shares and ten years of monthly returns (120 observations), at least 381 eigenvalues are exactly zero, and each one is a portfolio the data says has no risk. A tiny case shows the danger: five asset classes, four months of simulated returns. The estimated matrix has two zero eigenvalues, and the fully invested portfolio 53% GE, 57% EM, −69% GB, −68% CB, 126% PR has an estimated volatility of exactly zero. Its true volatility, from the module's inputs, is 36.26%. An optimiser fed that estimate would pick it. L3.1.11: Factor models and Fama-French shows how factor models such as the single-index model in the last column cut the inputs down.

Not every table is a correlation matrix

A valid correlation matrix must be:

• symmetric, $\rho_{ij} = \rho_{ji}$;
• ones on the diagonal, every entry between −1 and 1;
• positive semi-definiteA symmetric matrix A is positive semi-definite if w′Aw ≥ 0 for every vector w. Equivalently, none of its eigenvalues is negative.: $w'Rw \ge 0$ for every $w$, because $w'Rw$ is the variance of a portfolio of the assets scaled to unit volatility, and a variance cannot be negative.

The first two are easy to see. The third is not, because it is about all the entries together. Take this table, put together pair by pair:

Every entry looks reasonable. But if A moves closely with B (0.9) and closely with C (0.8), then B and C must move fairly closely with each other. Given the first two, the third correlation must lie between 0.458 and 0.982. At 0.2 the table describes three assets that cannot exist.

The diagnosis: eigenvalues. A symmetric matrix is positive semi-definite exactly when none of its eigenvaluesA number λ with a vector q (its eigenvector) such that Aq = λq. A symmetric n × n matrix has n of them, and q′Aq = λ when q has length one. is negative (the proof is in the formal version below). This one has eigenvalues $2.308,\ 0.801,\ -0.109$. The negative one comes with a portfolio: hold +1 of A, −0.70 of B and −0.59 of C (in units of one volatility each), and the table says its variance is −0.202. Give the assets volatilities of 16%, 20% and 18% and the covariance matrix $\Sigma = DRD$ inherits the problem: positions $(1,\ -0.578,\ -0.530)$ have a variance of −0.0052. A risk model built on it can report negative risk, a square root that fails, or an optimiser that leverages up the “free” hedge without limit.

The repair: clip and rescale. Write $R = Q\Lambda Q'$, replace the negative eigenvalues by zero (or a small positive floor), rebuild the matrix, then divide each entry by $\sqrt{a_{ii}a_{jj}}$ to put ones back on the diagonal. For the broken table:

It works, but notice three things. Every correlation moved, not just the suspicious one: A–B fell from 0.9 to 0.822. The repaired B–C correlation, 0.219, sits exactly on the edge of its new allowed range. And with a floor of zero the result is singular, so some portfolio still has zero risk; in practice a small positive floor is used. Clipping is quick but not the closest valid matrix. The nearest correlation matrixThe valid correlation matrix closest to a given invalid one, usually in the sense of the smallest sum of squared differences (Higham, 2002). of Higham (2002) finds the valid matrix with the smallest total squared change, and lets you keep trusted entries fixed. The covariance matrix toolkit runs both repairs on matrices of any size.

Check the eigenvalues in the calculator

For n assets with weights w, expected returns μ and covariance matrix Σ (all from one model):

Why eigenvalues decide validity. Any symmetric matrix has an eigen-decomposition $\Sigma = Q\Lambda Q'$ with orthonormal eigenvectors. That turns the quadratic form into a weighted sum of squares:

Two useful facts follow. The eigenvalues of a correlation matrix add up to $n$ (the trace, the sum of the diagonal), so one large eigenvalue forces others to be small. And when every pair has the same correlation ρ, the eigenvalues are $1 + (n-1)\rho$ once and $1 - \rho$ the other $n - 1$ times, so the matrix is valid only if $\rho \ge -1/(n-1)$. Three assets cannot all have correlation below −0.5 with each other; ten cannot all be below about −0.11. Many assets cannot all hedge one another.

The three-asset bound. For a 3 × 3 correlation matrix the determinant is $1 - \rho_{12}^2 - \rho_{13}^2 - \rho_{23}^2 + 2\rho_{12}\rho_{13}\rho_{23}$. Setting it to zero and solving for $\rho_{23}$ gives the allowed range:

For the broken table the determinant is $-0.202$. A positive-definite matrix (all eigenvalues above zero) also has a Cholesky factor $\Sigma = LL'$, which simulations use to draw correlated returns; trying to compute it is a cheap validity test, since it fails when the matrix is not positive definite.

Start from the group's current allocation and watch the property row: it is the largest. Switch the correlations off to see how much of the risk comes from assets moving together. Then try the long EM, short GE preset and look for the red cells: a short position against a positively correlated asset takes risk away.

Check the volatility in the calculator

Regulatory capital formulas use the same quadratic form, with a correlation matrix set by the regulator. Under the Solvency II standard formula, an insurer such as RiskAtoms Insurance adds up its capital for market, counterparty, life, health and non-life risk as

with the correlations listed in Annex IV of the Solvency II Directive (2009/138/EC). That is $\sqrt{s'Rs}$, the volatility formula with capital amounts in place of weighted volatilities. The Basel market risk standard (FRTB) does the same within and across buckets of risk factors in its sensitivities-based method, and because its correlations are applied to signed amounts, it says what to do when the sum under the square root comes out negative (MAR21.4(4) and (5)(b)). A firm that builds its own internal model has to show its correlation assumptions are coherent: the checks in this lesson are the first test. Capital aggregation is covered in M4.6: Solvency II and insurance regulation.

• Expected return is w′μ and variance is w′Σw: one line for any number of assets, and the two-asset formula is the case n = 2.
• The variance grid has a cell w_i w_j σ_ij for every pair; the diagonal holds own variances, the rest covariances.
• Σ = DRD separates volatilities (D) from correlations (R).
• A covariance matrix needs n(n + 1)/2 inputs, and with fewer observations than assets the estimate is singular.
• A valid correlation matrix is symmetric, has a unit diagonal and is positive semi-definite: no negative eigenvalue. Clip and rescale, or Higham's nearest correlation matrix, repairs one that is not.

Vector

An ordered list of numbers, written as a column. Here: one weight, or one expected return, per asset.

Covariance matrix

The n × n table of every variance (on the diagonal) and every covariance (off it) of n assets' returns. Symbol Σ.

Quadratic form

An expression w′Aw: a vector times a square matrix times the same vector, giving one number.

Correlation matrix

The n × n table of correlations, with ones on the diagonal. It is the covariance matrix of the returns divided by their volatilities.

Positive semi-definite

A symmetric matrix A is positive semi-definite if w′Aw ≥ 0 for every vector w. Equivalently, none of its eigenvalues is negative.

Eigenvalue

A number λ with a vector q (its eigenvector) such that Aq = λq. A symmetric n × n matrix has n of them, and q′Aq = λ when q has length one.

Nearest correlation matrix

The valid correlation matrix closest to a given invalid one, usually in the sense of the smallest sum of squared differences (Higham, 2002).

Single-index model

A model in which every asset's return depends on one common index plus its own noise, so the model needs only 3n + 2 inputs, expected returns included (2n + 1 for the covariance matrix alone).