Tutorial: Optimization and related nonlinear modelling computations in R
John C. Nash, Tefler School of Management, University of Ottawa (retired), Canada
We will present an overview
of the tools in R for optimization and related tasks.
Discussion will be by examples, and
participants or others are welcome to send problems in advance for
possible inclusion in the discussion.
There are many uses of optimization in statistics, especially in model
fitting. However, numerical optimization of nonlinear functions,
particularly with constraints, is a rich and intricate subject where
statisticians are rarely experts. The tutorial will attempt to give R users
a handle on the available packages and functions for solving optimization
problems in R.
Largely using the optimx package (from the
and solving packages project)
that allows a number of the existing optimization tools to be
accessed via a common front-end, we will discuss via examples:
Example issues that may be addressed to provide participants with an informed view of the
optimization landscape are:
- Writing an objective function
- Scaling and constraints
- Starting values, and
- Interpretation of results
Optimization with box-constraints (bounds) as well as temporarily fixed parameters (masks). optimx can call optim with method='L-BFGS-B', nlminb, bobyqa from package minqa, spg from package BB, or Rcgmin among other approaches
Nonlinear regression with the objective function defined using an expression as in the function nls, or via a residual function as in the package minpack.lm
- Linear regression with constraints, comparing minimization of an objective function to the use of specialist packages such as nnls, bvls and quadprog
- Issues for development of specialist tools such as the maxLik or fitdistrplus packages that use optimization
- Global optimization issues, e.g., using package DEoptim, optim with method='SANN', or multiple start wrapper code
- The relative merits of gradient-based vs. derivative-free optimization methods
- Simplex-style algorithms e.g. optim with the default method='Nelder-Mead' but
also some alternative "plex" variants
- Quadratic Approximation methods (using the R package minqa)
- Using gradient methods e.g., optim method='BFGS' with numerically approximated derivatives
R users doing various forms of modelling and estimation who want to
know how to appropriately choose and use the available
and where to find help and advice.
The presenter is the author of
Numerical Methods for Computers, Taylor and Francis, 1990. Three
algorithms from this book are part of the now-aging optim function in R. He has been
recently working (with Ravi Varadhan, Kate Mullen and others) on improved tools and guidance for optimization in R, implementations
of which can be found in the
and solving packages project.
Slides are here.