Constrained Optimisation and Lagrange Multipliers

By the end you should be able to: Solve a constrained optimisation problem by elimination and by Lagrange multipliers, show the two agree, and explain the geometric meaning of the multiplier condition.

Module 3 optimised with the variables free to take any value. Real problems are rarely so generous: a budget is fixed, a length of fencing is given, a probability must sum to one.

\text{minimise } f(x,y) \quad \text{subject to } g(x,y) = 0

Method 1 — elimination

Use the constraint to remove a variable, then optimise what is left with ordinary single-variable calculus.

Minimise $f = x^2+y^2$ subject to $x+y=1$ .

From the constraint, $y = 1-x$ . Substitute:

f = x^2 + (1-x)^2 = 2x^2 - 2x + 1

Now a one-variable problem:

\frac{df}{dx} = 4x - 2 = 0 \implies x = \tfrac12 \implies y = \tfrac12, \qquad f = \tfrac12

Second derivative is $4 > 0$ , so it is a minimum.

When it works: when the constraint can be rearranged cleanly. Often it cannot — try eliminating a variable from $x^2+y^2+z^2=1$ and the square roots make a mess.

Method 2 — Lagrange multipliers

Build one function combining objective and constraint:

\mathcal{L}(x,y,\lambda) = f(x,y) - \lambda\, g(x,y)

Set all three partial derivatives to zero:

\frac{\partial \mathcal L}{\partial x} = 0, \qquad \frac{\partial \mathcal L}{\partial y} = 0, \qquad \frac{\partial \mathcal L}{\partial \lambda} = 0

The third simply returns the constraint, so nothing is lost. The first two are equivalent to

\nabla f = \lambda \nabla g

What that condition means

At a constrained optimum the two gradients are parallel.

Picture the contours of $f$ and the curve $g=0$ . Walking along the constraint curve, you cross contours of $f$ and its value changes. You can keep improving until the constraint curve is tangent to a contour of $f$ — at that point moving either way along the constraint no longer improves anything.

Tangent curves have parallel normals. The gradient is normal to its own level curve, so $\nabla f$ and $\nabla g$ point the same way, differing only by the scale factor $\lambda$ .

That is the entire geometric content of the method, and stating it earns marks that reciting the recipe does not.

The same problem by Lagrange

f = x^2+y^2, \qquad g = x+y-1 = 0

\mathcal L = x^2+y^2-\lambda(x+y-1)

\mathcal L_x = 2x-\lambda = 0 \implies x = \tfrac{\lambda}{2}

\mathcal L_y = 2y-\lambda = 0 \implies y = \tfrac{\lambda}{2}

\mathcal L_\lambda = -(x+y-1) = 0 \implies x+y = 1

The first two give $x = y$ . With $x+y=1$ that forces $x = y = \tfrac12$ , and $\lambda = 1$ .

f\left(\tfrac12,\tfrac12\right) = \tfrac14+\tfrac14 = \tfrac12

Identical to elimination, as it must be. Two correct methods on one problem cannot disagree, which makes each a check on the other.

What $\lambda$ means

The multiplier is not a bookkeeping device to be discarded. It measures the sensitivity of the optimum to the constraint: relax the constraint by one unit and the optimal value changes by approximately $\lambda$ .

In economics this is the shadow price — what one more unit of a scarce resource is worth. Here $\lambda = 1$ : raising the constraint from $x+y=1$ to $x+y=2$ would change the minimum by roughly 1. (Exactly: the new minimum is $2 \times 1^2 = 2$ , an increase of $\tfrac32$ — approximate because $\lambda$ is a derivative, accurate for small changes.)

Choosing a method

	Elimination	Lagrange
Constraint easy to rearrange	Simpler	Works, more steps
Constraint implicit or messy	Often impossible	Works
Several constraints	Impractical	One multiplier each
Gives sensitivity information	No	Yes, via $\lambda$