# R Optimization

October 26, 2010 by
Filed under: Notes

Check eigenvalues of hessian of optimized sum of squares to check for singular gradient matrix.  A singular gradient matrix has infinite solutions, so the best you can do is the optimized set of values plus a linearly scaled vector.  The vector is equal to the eigenvector associated with the eigenvalue that is numerically indistinguishable from zero.

So if you have

```optimfunc <- function(x, data, sevcalcfunc, optimgoal,  ...) { calcsev <- sevcalcfunc(x, data, ...) sumsqerr <- sum((optimgoal - calcsev)^2) return(sumsqerr) }```

and some function, then you can optimize
`system.time(optimsimple <- optim(c(.3, -12614.716, .1), optimfunc, hessian = TRUE,data = reg.data, sevcalcfunc = calclosssimple, optimgoal = reg.data\$maxloss))`

and
`eigen(optimsimple\$hessian)`
```\$values [1] 1.498617e+16 1.868481e+14 8.412726e+04```

Since the 3rd eigenvalue is very small compared to the first two, adding any constant z times the 3rd eigenvector to the optimized solution doesn't really change the value of the optimized sum of squares, and can therefore be considered a solution as well.

```eigen(optimsimple\$hessian)\$vectors[,3] [1] 1.956755e-06 1.000000e+00 4.573782e-06```

```> optimfunc(optimsimple\$par, reg.data, calclosssimple, reg.data\$maxloss) [1] 3.793891e+14 > optimfunc(optimsimple\$par + 1000 * eigen(optimsimple\$hessian)\$vectors[,3], reg.data, calclosssimple, reg.data\$maxloss) [1] 3.794212e+14 > optimfunc(optimsimple\$par - 1000 * eigen(optimsimple\$hessian)\$vectors[,3], reg.data, calclosssimple, reg.data\$maxloss) [1] 3.794211e+14```