# R Programming/Optimization

## Numerical Methods

### One dimensional problem

The one dimensional problem :

```> func <- function(x){
+ 	return ( (x-2)^2 )
+ 	}
> (func(-2))
 16
>
> # plot your function using the 'curve function'
> curve(func,-4,8)
>
> # Here is another way to plot the function
> # using a grid
> grid <- seq(-10,10,by=.1)
> func(grid)
> plot(grid,func(grid))
>
> # you can find the minimum using the optimize function
> optimize(f=func,interval=c(-10,10))
\$minimum
 2

\$objective
 0
```

### Newton-Raphson

• nlm() provides a Newton algorithm.
• maxLik package for maximization of a likelihood function. This package includes the Newton Raphson method.
• newtonraphson() in the spuRs package.

### BFGS

```> func <- function(x){
+ 	out <- (x-2)^2 + (x-1)^2
+ 	return <- out
+ 	}>
> optim(par=c(0,0), fn=func, gr = NULL,
+       method = c("BFGS"),
+       lower = -Inf, upper = Inf,
+       control = list(), hessian = T)
> optim(par=c(0,0), fn=func, gr = NULL,
+       method = c("L-BFGS-B"),
+       lower = -Inf, upper = Inf,
+       control = list(), hessian = T)
```

### Conjugate gradient method

• `optim()` with `method="cg"`.

### Trust Region Method

• "trust" package for trust region method

### The Nelder-Mead simplex method

```> func <- function(x){
+ 	out <- (x-2)^2 + (x-1)^2
+ 	return <- out
+ 	}
>
> optim(par=c(0,0), fn=func, gr = NULL,
+       method = c("Nelder-Mead"),
+       lower = -Inf, upper = Inf,
+       control = list(), hessian = T)
```

• The boot package includes another simplex method

## Simulation methods

### Simulated Annealing

• The Simulated Annealing is an algorithm which is useful to maximise non-smooth functions. It is pre implemented in optim().
```> func <- function(x){
+ 	out <- (x-2)^2 + (x-1)^2
+ 	return <- out
+ 	}>
> optim(par=c(0,0), fn=func, gr = NULL,
+       method = c("SANN"),
+       lower = -Inf, upper = Inf,
+       control = list(), hessian = T)
```

### Genetic Algorithm

• rgenoud package for genetic algorithm
• gaoptim package for genetic algorithm
• ga general purpose package for optimization using genetic algorithms. It provides a flexible set of tools for implementing genetic algorithms search in both the continuous and discrete case, whether constrained or not.