Math Atom · M-B04

Optimisation and Lagrange multipliers

10 minMath AtomCalculus

Know these first

By the end you will be able to

  1. 1Find a minimum or maximum of a function of one variable with f′ = 0, and tell which it is from f″.
  2. 2Find the stationary point of a function of two variables by setting both partial derivatives to zero.
  3. 3Solve a problem with one equality constraint using a Lagrange multiplier, and read the multiplier as a shadow price.
  4. 4Set up and solve a problem with two equality constraints, one multiplier each.

Why you need it

The smallest risk for a given return. The largest return for a given risk. Optimisation finds the best choice, and Lagrange multipliers find it when the choice has to obey rules, such as weights that add up to 100%.

These Core Path lessons use it:

You need derivatives from M-B01: Derivatives and the second derivative from M-B03: Taylor expansion. The two-variable case uses partial derivatives, introduced here in one step.

The idea

The lowest point on a fence

Picture a bowl-shaped valley, drawn as contour lines, and a straight fence across it. You may only stand on the fence. Where is the lowest point you can reach? Walk along it and watch the height.

Walk along the fence

f = x² + 2y², fence x + y = 3
xy

Ellipses: contours of f (the highlighted one is f = 6). Purple line: the fence. Arrows: ∇f (blue) and ∇g (purple).

f along the fence, x from −0.5 to 4.5
Point (x, y)(0.5, 2.5)
f(x, y)12.75
∇f = (2x, 4y)(1, 10)
∇g for g = x + y(1, 1)

∇f and ∇g point different ways, so walking along the fence still changes f. Find where they line up.

At the lowest point on the fence, the fence just touches one contour without crossing it. There, the direction of steepest climb, the gradient ∇f, is at right angles to the fence, just like the fence's own gradient ∇g. So ∇f is a multiple of ∇g. That multiple is the Lagrange multiplier.

The rules

One variable. The first-order condition finds the flat spots; the second-order condition sorts them.

Two variables, no constraint. One partial derivative per variable, all set to zero.

One constraint. The tangency in the picture, turned into equations with a Lagrange multiplier.

What λ means. The multiplier is not just a helper: it is a shadow price.

Two constraints. One multiplier per constraint, and the same steps.

The rules, for reference

f′(x∗)=0;    f′′(x∗)>0⇒min⁡,  f′′(x∗)<0⇒max⁡∇f(x∗)=0,    H positive definite⇒min⁡f'(x^*) = 0;\;\; f''(x^*) > 0 \Rightarrow \min,\; f''(x^*) < 0 \Rightarrow \max \qquad \nabla f(\mathbf{x}^*) = \mathbf{0},\;\; H \text{ positive definite} \Rightarrow \min
L=f−λ1(g1−c1)−λ2(g2−c2),∇f=λ1∇g1+λ2∇g2,∂f∗∂ck=λk\mathcal L = f - \lambda_1(g_1 - c_1) - \lambda_2(g_2 - c_2), \qquad \nabla f = \lambda_1\nabla g_1 + \lambda_2\nabla g_2, \qquad \frac{\partial f^*}{\partial c_k} = \lambda_k

Try it

Minimise a x² + b y² on the fence x + y = c. Change the shape of the bowl and move the fence. Watch where the best point lands, and check that λ predicts how much the best value rises when the fence moves out by 0.1.

Try itMinimum on a fence

Calculus calculator →
xy

Minimise 1x² + 2y² subject to x + y = 3

x* = bc / (a + b)2
y* = ac / (a + b)1
λ* = 2a x* = 2b y*4
Best value f*6

Shadow price check: raise c by 0.1

New best value6.4067
Actual change in f*0.4067
λ* × 0.10.4

Worked examples

1. Stationary points of a cubic

Step 1 of 4
  1. 1

    The set-up

    f(x)=x3−6x2+9x+1f(x) = x^3 - 6x^2 + 9x + 1

2. One constraint, and its shadow price

Step 1 of 5
  1. 1

    The set-up

    min⁡x,y  x2+2y2subject tox+y=3\min_{x,y}\; x^2 + 2y^2 \quad \text{subject to} \quad x + y = 3

3. Two constraints: three funds

Step 1 of 5
  1. 1

    The set-up

    Three illustrative funds with volatilities 20%, 10%, 15% and expected returns 8%, 3%, 6%. To keep the algebra short, take them as uncorrelated, so by M-C11: Combining random variables the variance of a mix is a sum of squares. Find the weights with the smallest variance that add up to 1 and give an expected return of 6%:

    min⁡w  400 wA2+100 wB2+225 wC2subject to  wA+wB+wC=18wA+3wB+6wC=6\begin{aligned} \min_{\mathbf w}\;& 400\,w_A^2 + 100\,w_B^2 + 225\,w_C^2 \\ \text{subject to}\;& w_A + w_B + w_C = 1 \\ & 8w_A + 3w_B + 6w_C = 6 \end{aligned}

    Returns in %, so the variance is in %².

Common mistakes

Each line below is a tempting slip. Spot what is wrong, then check.

Reaction

Check what stuck

Question 1 · objective 1

f(x) = x² − 6x + 2. At what x is f′(x) = 0?

Question 2 · objective 1

At x*, f′(x*) = 0 and f″(x*) = −4. What is x*?

Question 3 · objective 2

Minimise x² + y² subject to x + y = 4, with 𝓛 = x² + y² − λ(x + y − 4). What is λ*?

Question 4 · objective 3

A minimum has multiplier λ* = 2.5. Roughly how much does the best value change if the constraint's c rises by 0.2?

Question 5 · objective 4

You minimise a function of three unknowns subject to two equality constraints. How many equations do you solve?

Glossary

Stationary point
A point where the derivative (or every partial derivative) is zero: a flat spot that may be a minimum, a maximum or neither.
First-order condition
The condition that the first derivative is zero at the best point: f′(x*) = 0, or ∇f = 0 in several variables.
Second-order condition
The check on the second derivative that tells a minimum (f″ > 0) from a maximum (f″ < 0).
Partial derivative
The derivative of a function of several variables with respect to one of them, holding the others fixed: ∂f/∂x.
Gradient (∇f)
The vector of all partial derivatives, ∇f. It points in the direction in which f rises fastest.
Constraint
A condition the answer must satisfy, such as x + y = 3, written g(x, y) = c.
Lagrangian (𝓛)
𝓛 = f − λ(g − c): the function whose partial derivatives, set to zero, give the constrained optimum.
Lagrange multiplier (λ)
The extra unknown λ in the Lagrangian, one per constraint. At the optimum, ∇f = λ∇g.
Shadow price
The rate at which the best value changes as a constraint is loosened: df*/dc = λ.

Back to top ↑