Lagrange multiplier

To minimise f(x), subject to the constraint g(x) = c, consider

Λ(x, λ) = f(x) + λ { g(x) - c }.

λ is known as the "Lagrange multiplier."

 

Solve

∇ Λ(x, λ) = (∂Λ/∂x₁, ..., ∂Λ/∂x_n, ∂Λ/∂λ) = 0

 

For example, given positive integers {n₁, ..., n_k}, minimise

n₁ log p₁ + ... + n_k log p_k

subject to

p₁ + ... + p_k = 1,

let

Λ(p, λ) = n₁ log p₁ + ... + n_k log p_k + λ{p₁ + ... + p_k - 1},

∇ Λ = (n₁/p₁ + λ, ..., n_k/p_k + λ, p₁ + ... + p_k - 1) = 0,

so

p_i = - n_i / λ ∝ n_i   (λ can be negative)

and

∑ p_i = 1,

giving

p_i = n_i / ∑_j n_j   (and λ = - ∑_j n_j).

(Also see the [multinomial] probability distribution.)