Mean-variance optimisation · RiskAtoms Academy

By the end you will be able to

1State the Markowitz problem: minimum variance for a target expected return, with weights adding up to one.
2Solve it with a Lagrangian and the inverse covariance matrix, and compute the answer for three assets by hand.
3Explain the two-fund theorem and why frontier weights are straight lines in the target return.
4Describe what changes when short sales are banned, and how such problems are solved numerically.
5Explain why optimised weights are so sensitive to the expected returns that go in.

Why it matters

RiskAtoms Group's current mix expects 5.45% at 9.50% volatility. Ask an optimiser for the least risky long-only portfolio with the same expected return and it answers 8.51%: about £10m less one-standard-deviation risk a year on £1bn. Raise one expected return by half a point and ask again, and it trades 15% of the portfolio.

Core explanation

From a picture to a recipe

Illustrative dataRiskAtoms Group's investment portfolio

The group invests £1bn of surplus assets across five asset classes. The expected returns, volatilities and correlations are round, made-up long-run assumptions in pounds, a year, not estimates from market data. The risk-free rate (a one-year Treasury bill) is 3.0%. The same numbers run through the whole module.

Asset class	Expected return	Volatility	Sharpe ratio
Global equities	7.0%	16%	0.250
Emerging-market equities	8.5%	22%	0.250
Government bonds	3.5%	6%	0.083
Corporate bonds	4.5%	8%	0.188
Property	6.0%	18%	0.167

The problem

L3.1.05: The efficient frontier drew the efficient frontier; L3.1.06: Many assets and the covariance matrix gave the notation to compute it. Mean-variance optimisation finds each frontier portfolio exactly. For a target expected return $m$ :

\min_w\ \tfrac12\,w'\Sigma w \qquad\text{subject to}\qquad w'\mu = m, \quad w'\mathbf 1 = 1

The inputs are the expected returns $\mu$ and the covariance matrix $\Sigma$ . The half changes nothing but tidies the algebra. With two assets, L3.1.04: The minimum-variance portfolio solved a version of this (no return target) with a single Lagrange multiplier. Here there are two constraints, so two multipliers.

Solving it with a Lagrangian

Build the Lagrangian, set its derivative to zero, and the weights come out in terms of the inverse of the covariance matrix (M-E03: Matrix inverse and linear systems). Follow it step by step:

The whole answer in four numbers

A = \mathbf 1'\Sigma^{-1}\mu, \quad B = \mu'\Sigma^{-1}\mu, \quad C = \mathbf 1'\Sigma^{-1}\mathbf 1, \quad D = BC - A^2

w^* = \frac{Cm - A}{D}\,\Sigma^{-1}\mu + \frac{B - Am}{D}\,\Sigma^{-1}\mathbf 1, \qquad \sigma^2(m) = \frac{Cm^2 - 2Am + B}{D}

Three assets, by hand

Take global equities, emerging-market equities, government bonds and a target of 5.5%. The covariance matrix and its inverse, in full:

\Sigma = \begin{pmatrix} 0.02560 & 0.02640 & -0.00096 \\ 0.02640 & 0.04840 & -0.00066 \\ -0.00096 & -0.00066 & 0.00360 \end{pmatrix}, \qquad \Sigma^{-1} = \begin{pmatrix} 90.09 & -48.94 & 15.05 \\ -48.94 & 47.29 & -4.38 \\ 15.05 & -4.38 & 280.99 \end{pmatrix}

Check: $\Sigma\,\Sigma^{-1} = I$ , the identity matrix. The large entry for government bonds ( $281.0$ ) reflects their low variance; the negative entry for the two equity markets reflects their 0.75 correlation, since each is a good substitute for the other. Next, multiply the inverse by the vector of ones and by the expected returns:

\Sigma^{-1}\mathbf 1 = \begin{pmatrix} 56.209 \\ -6.021 \\ 291.663 \end{pmatrix}, \qquad \Sigma^{-1}\mu = \begin{pmatrix} 2.6738 \\ 0.4412 \\ 10.5161 \end{pmatrix}

Add up the entries for A and C, and dot with μ for B:

A = 13.6310, \quad B = 0.5927, \quad C = 341.85, \quad D = BC - A^2 = 16.8190

\lambda = \frac{C(0.055) - A}{D} = 0.3074, \qquad \gamma = \frac{B - A(0.055)}{D} = -0.00933

w^* = \lambda\,\Sigma^{-1}\mu + \gamma\,\Sigma^{-1}\mathbf 1 = \begin{pmatrix} 0.2974 \\ 0.1918 \\ 0.5108 \end{pmatrix}, \qquad \sigma = \sqrt{\frac{Cm^2 - 2Am + B}{D}} = 8.70\%

The weights add up to one and give exactly 5.5%. In practice software does not form $\Sigma^{-1}$ : it solves the linear systems $\Sigma x = \mathbf 1$ and $\Sigma y = \mu$ directly, which is faster and more accurate. The answer is the same.

The two-fund theorem

Look at the form of the answer again: the same two vectors, $\Sigma^{-1}\mu$ and $\Sigma^{-1}\mathbf 1$ , for every target. Only the multipliers change, and they change in straight lines. That has a striking consequence, the two-fund theorem:

For the three assets: the frontier portfolio for 4.5% holds 20.9% global equities, 5.3% emerging-market equities, 73.7% government bonds; the one for 6.5% holds 38.5% global equities, 33.0% emerging-market equities, 28.4% government bonds. Half of each gives 29.7%, 19.2%, 51.1%, exactly the 5.5% portfolio computed above. An investor who wants any point on the frontier needs only two funds. L3.1.08: The risk-free asset and the capital market line makes one of them risk-free, and the result is even simpler.

Show

Global equitiesEmerging-market equitiesGovernment bondsCorporate bondsProperty

With short sales allowed, every weight is a straight line in the target return: the two-fund theorem at work for all five assets. Property is a small short at every efficient target; corporate bonds are shorted below a 4.97% target, government bonds above 6.53%. Illustrative inputs.

When short sales are banned

Most investors cannot, or may not, sell short. Add $w \ge 0$ (or a cap such as $w_i \le 40\%$ ) and the closed form is gone: the answer depends on which constraints bind, and that changes along the frontier. The conditions that replace it are the Karush-Kuhn-Tucker conditions. In words: every asset held must offer the same trade-off between return and marginal risk, and no asset left out may offer a better one.

Between two changes in the set of assets held, the problem is an ordinary equality-constrained one on the assets held, so the weights again move in straight lines. Where an asset enters or leaves sits a corner portfolio. Switch the chart above to long-only to see them. For the five asset classes:

Expected return	Volatility	What changes
4.01%	5.41%	minimum-variance portfolio: global equities and government bonds
4.11%	5.44%	emerging-market equities enters
5.01%	7.11%	corporate bonds enters
6.54%	12.58%	government bonds leaves
7.82%	18.11%	corporate bonds leaves
8.50%	22.00%	100% emerging-market equities

The lesson's code solves the long-only problem by trying every pattern of “held”, “at zero” and “at the cap” (3⁵ = 243 for five assets), solving each with the Lagrangian method and keeping the best answer that obeys the constraints. That is exact but grows as $3^n$ ; real quadratic-programming solvers search the patterns cleverly and handle thousands of assets.

Garbage in, garbage out

The optimiser treats its inputs as exact. They are not: an expected return is an estimate with an error of several points. Raise emerging-market equities from 8.5% to 9% and re-solve for the same 5.45% target, long-only: the weight in emerging markets goes from 17.5% to 21.6%, global equities from 27.8% to 21.1%, and 14.7% of the portfolio changes hands. Because the optimiser leans on whatever looks best, and what looks best is disproportionately what has been overestimated, optimised portfolios are often called error maximisers. The usual defences are constraints (the 40% cap halves the turnover here, to 7.1%), shrinking the inputs towards something simpler, and averaging the answers over many plausible inputs.

The formal version

With n assets, expected returns μ and a positive-definite covariance matrix Σ:

\begin{aligned} &\text{Problem:} && \min_w \tfrac12 w'\Sigma w \ \text{ s.t. } w'\mu = m,\ w'\mathbf 1 = 1 \\ &\text{Solution:} && w^*(m) = \lambda\,\Sigma^{-1}\mu + \gamma\,\Sigma^{-1}\mathbf 1, \quad \lambda = \tfrac{Cm - A}{D},\ \gamma = \tfrac{B - Am}{D} \\ &\text{Frontier:} && \sigma^2(m) = \tfrac{Cm^2 - 2Am + B}{D}, \qquad \tfrac{d}{dm}\big(\tfrac12\sigma^2\big) = \lambda \\ &\text{Two funds:} && w_g = \tfrac{\Sigma^{-1}\mathbf 1}{C}\ (\text{return } \tfrac AC,\ \text{variance } \tfrac1C), \qquad w_\mu = \tfrac{\Sigma^{-1}\mu}{A}\ (\text{return } \tfrac BA) \end{aligned}

The multiplier λ is the shadow price of the return target: raising the target by a small $\Delta m$ raises half the variance by about $\lambda\,\Delta m$ . $D > 0$ whenever the expected returns are not all equal (by the Cauchy-Schwarz inequality in the inner product defined by $\Sigma^{-1}$ ), so the 2 × 2 system always has a unique solution.

With bounds $\ell \le w \le u$ the problem is a convex quadratic programme. Written with a risk tolerance $t \ge 0$ , the KKT conditions are necessary and sufficient:

t\mu - \Sigma w + \gamma\mathbf 1 + \nu_\ell - \nu_u = 0, \quad \nu_\ell, \nu_u \ge 0, \quad \nu_{\ell,i}(w_i - \ell_i) = 0, \quad \nu_{u,i}(u_i - w_i) = 0

For three assets, $w_g$ holds 16.4%, −1.8%, 85.3% (return 3.99%, volatility 5.41%) and $w_\mu$ holds 19.6%, 3.2%, 77.1% (return 4.35%). The 5.5% portfolio is $4.191\,w_\mu - 3.191\,w_g$ : the two funds are close together, so reaching a target well above both takes a large long position in one and a short position in the other.

Try it

Start at the current expected return, 5.45%, long-only. Switch between the three constraints and watch how little the volatility changes. Then nudge emerging-market equities' expected return by a few tenths of a point and watch the weights move; the outlines show where they were.

Try itA five-asset optimiser

Target expected return5.45%

Constraint

Nudge EM equities' expected return±0.0 pt → 8.5%

Global eq.27.8%

EM eq.17.5%

Gov. bonds44.3%

Corp. bonds10.5%

Property0.0%

Lowest volatility for the target

8.51%

Expected return5.45%

Current mix9.50% at 5.45%

Turnover is the share of the portfolio that would have to be bought (or sold) to move to the new answer.

frontier for this constraintyour targetcurrent mix

Worked example

Five asset classes at the current expected return

Portfolio	Global eq.	EM eq.	Gov. bonds	Corp. bonds	Property	Volatility
Current	25.0%	5.0%	25.0%	20.0%	25.0%	9.50%
Short sales allowed	29.1%	17.6%	44.2%	11.6%	−2.5%	8.51%
Long-only	27.8%	17.5%	44.3%	10.5%	0.0%	8.51%
Long-only, at most 40% each	27.4%	16.6%	40.0%	16.0%	0.0%	8.52%

Reading the table

Step 1 of 4

1
Same return, less risk
Every row expects 5.45%. The current mix has 9.50% volatility; the long-only optimum has 8.51%, about £9.9m less one-standard-deviation risk a year on £1bn. Most of the gain comes from selling property, which has a lower Sharpe ratio than equities and a 0.6 correlation with them.

Illustrative inputs throughout. The weights come from the closed form (short sales allowed) and from the constrained solver in the module's code (the other two rows).

Regulatory lens

Constraints are where regulation usually enters an optimiser. A UCITS fund's diversification limits (the 5/10/40 rule of L3.1.01: Portfolio return and risk) are caps of exactly the kind solved here. Solvency II sets no fixed limits for an insurer such as RiskAtoms Insurance, but its prudent person principle (Article 132 of the Directive) requires it to invest only in assets whose risks it can “properly identify, measure, monitor, manage, control and report”, and in a way that ensures “the security, quality, liquidity and profitability” of the portfolio as a whole. An optimiser's answer is evidence for that judgement, not a substitute for it. UK banks with internal-model approval follow the PRA's model risk management principles (SS1/23), which expect a model's limitations, such as its sensitivity to inputs, to be understood, tested and reported.

Spark Atoms

Spark AtomThe critical line

In 1956 Markowitz published a method for the constrained problem, the critical line algorithm. It starts at the top of the frontier and walks down it corner by corner, changing the set of assets held one at a time: the corner portfolios in the table above. Versions of it are still used.

Spark AtomWhat Markowitz did with his own money

In an interview for Money magazine (January 1998), Markowitz said that when he first chose how to invest his retirement plan contributions in the 1950s, he did not compute an efficient frontier. He split them 50/50 between bonds and equities instead, “to minimize my future regret”.

Key takeaways

Mean-variance optimisation: minimise ½w′Σw subject to w′μ = m and w′1 = 1.
The Lagrangian gives w* = λΣ⁻¹μ + γΣ⁻¹1, with λ and γ from a 2 × 2 system in A, B and C.
Two-fund theorem: with short sales allowed, every frontier portfolio is a mix of any two others, and every weight is a straight line in the target.
Without short sales there is no closed form; the frontier is pieced together between corner portfolios and found numerically.
Optimised weights are very sensitive to expected returns: half a point on one asset moves 15% of this portfolio.

Reaction

Check what stuck

Glossary

Mean-variance optimisation: Choosing portfolio weights to minimise variance for a target expected return (or to maximise return for a level of risk), given expected returns and a covariance matrix.
Lagrangian: The objective plus each constraint times its own multiplier. Setting its derivatives to zero gives the constrained optimum.
Inverse matrix: The matrix Σ⁻¹ with ΣΣ⁻¹ = I. It exists when Σ is positive definite, and it undoes multiplication by Σ, so Σw = b has the solution w = Σ⁻¹b.
Two-fund theorem: With short sales allowed, every frontier portfolio is a mix of any two different frontier portfolios. Two funds are enough to serve every mean-variance investor.
KKT conditions: The Karush-Kuhn-Tucker conditions: the first-order conditions for an optimum with inequality constraints, such as w ≥ 0.
Corner portfolio: A portfolio on a constrained frontier where an asset enters or leaves. Between corners the weights change in straight lines.
Error maximisation: The tendency of an optimiser to overweight assets whose expected returns are overestimated and underweight those that are underestimated, so estimation errors are amplified.
Turnover: The share of a portfolio bought (equal to the share sold) to move from one set of weights to another.

Deep Dives & Origins

Deep Dives

R. C. Merton, “An Analytic Derivation of the Efficient Portfolio Frontier”, Journal of Financial and Quantitative Analysis, 7(4), 1972. Advanced.
R. O. Michaud, “The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal?”, Financial Analysts Journal, 45(1), 1989. Intermediate.

Origins

H. Markowitz, “Portfolio Selection”, Journal of Finance, 7(1), 1952, pp. 77–91.
H. Markowitz, “The Optimization of a Quadratic Function Subject to Linear Constraints”, Naval Research Logistics Quarterly, 3(1–2), 1956, pp. 111–133.
R. C. Merton, “An Analytic Derivation of the Efficient Portfolio Frontier”, Journal of Financial and Quantitative Analysis, 7(4), 1972, pp. 1851–1872.
M. J. Best and R. R. Grauer, “On the Sensitivity of Mean-Variance-Efficient Portfolios to Changes in Asset Means: Some Analytical and Computational Results”, Review of Financial Studies, 4(2), 1991, pp. 315–342.
R. O. Michaud, “The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal?”, Financial Analysts Journal, 45(1), 1989, pp. 31–42.
Directive 2009/138/EC (Solvency II), Article 132; PRA Supervisory Statement SS1/23.

Watch this lesson

Video version

Prefer to watch? The same lesson as a 12-minute video will sit here, made from this page's script.