# Calculus Optimization Methods

A key application of calculus is in optimization: finding maximum and minimum values of a function, and which points realize these extrema.

### Context

Formally, the field of mathematical optimization is called mathematical programming, and calculus methods of optimization are basic forms of nonlinear programming. We will primarily discuss finite-dimensional optimization, illustrating with functions in 1 or 2 variables, and algebraically discussing n variables. We will also indicate some extensions to infinite-dimensional optimization, such as calculus of variations, which is a primary application of these methods in physics.

### Techniques

Basic techniques include the first and second derivative test, and their higher-dimensional generalizations.

A more advanced technique is Lagrange multipliers, and generalizations as Karush–Kuhn–Tucker conditions and Lagrange multipliers on Banach spaces.

### Applications

Optimization, particularly via Lagrange multipliers, is particularly used in the following fields:

Further, several areas of mathematics can be understood as generalizations of these methods, notably Morse theory and calculus of variations.

### Terminology

• Input points, output values
• Maxima, minima, extrema, optima
• Stationary point, critical point; stationary value, critical value
• Objective function
• Constraints – equality and inequality
• Especially sublevel sets
• Feasible region, whose points are candidate solutions

## Statement

This tutorial presents an introduction to optimization problems that involve finding a maximum or a minimum value of an objective function $f(x_{1},x_{2},\ldots ,x_{n})$  subject to a constraint of the form $g(x_{1},x_{2},\ldots ,x_{n})=k$ .

## Maximum and minimum

Finding optimum values of the function $f(x_{1},x_{2},\ldots ,x_{n})$  without a constraint is a well known problem dealt with in calculus courses. One would normally use the gradient to find stationary points. Then check all stationary and boundary points to find optimum values.

### Example

• $f(x,y)=2x^{2}+y^{2}$
• $f_{x}(x,y)=4x=0$
• $f_{y}(x,y)=2y=0$

$f(x,y)$  has one stationary point at (0,0).

## The Hessian

A common method of determining whether or not a function has an extreme value at a stationary point is to evaluate the hessian of the function at that point. where the hessian is defined as

$H(f)={\begin{bmatrix}{\frac {{\partial }^{2}f}{\partial x_{1}^{2}}}&{\frac {{\partial }^{2}f}{\partial x_{1}\partial x_{2}}}&\dots &{\frac {{\partial }^{2}f}{\partial x_{1}\partial x_{n}}}\\{\frac {{\partial }^{2}f}{\partial x_{2}\partial x_{1}}}&{\frac {{\partial }^{2}f}{\partial x_{2}^{2}}}&\dots &{\frac {{\partial }^{2}f}{\partial x_{2}\partial x_{n}}}\\\vdots &\vdots &\ddots &\vdots \\{\frac {{\partial }^{2}f}{\partial x_{n}\partial x_{1}}}&{\frac {{\partial }^{2}f}{\partial x_{n}\partial x_{2}}}&\dots &{\frac {{\partial }^{2}f}{\partial x_{n}^{2}}}\\\end{bmatrix}}.$

## Second derivative test

The Second derivative test determines the optimality of stationary point $x$  according to the following rules :

• If $H(f)>0$  at point x then $f$  has a local minimum at x
• If $H(f)<0$  at point x then $f$  has a local maximum at x
• If $H(f)$  has negative and positive eigenvalues then x is a saddle point
• Otherwise the test is inconclusive

In the above example.

$H(f)={\begin{bmatrix}4&0\\0&2\end{bmatrix}}.$

Therefore $f(x,y)$  has a minimum at (0,0).