 |
Tutorial: Optimization and related nonlinear modelling computations in R
|
John C. Nash, Tefler School of Management, University of Ottawa (retired), Canada
Abstract
We will present an overview
of the tools in R for optimization and related tasks.
Discussion will be by examples, and
participants or others are welcome to send problems in advance for
possible inclusion in the discussion.
There are many uses of optimization in statistics, especially in model
fitting. However, numerical optimization of nonlinear functions,
particularly with constraints, is a rich and intricate subject where
statisticians are rarely experts. The tutorial will attempt to give R users
a handle on the available packages and functions for solving optimization
problems in R.
Outline
Largely using the optimx package (from the
Optimization
and solving packages project)
that allows a number of the existing optimization tools to be
accessed via a common front-end, we will discuss via examples:
- Writing an objective function
- Scaling and constraints
- Starting values, and
- Interpretation of results
Example issues that may be addressed to provide participants with an informed view of the
optimization landscape are:
-
Optimization with box-constraints (bounds) as well as temporarily fixed parameters (masks). optimx can call optim with method='L-BFGS-B', nlminb, bobyqa from package minqa, spg from package BB, or Rcgmin among other approaches
-
Nonlinear regression with the objective function defined using an expression as in the function nls, or via a residual function as in the package minpack.lm
- Linear regression with constraints, comparing minimization of an objective function to the use of specialist packages such as nnls, bvls and quadprog
- Issues for development of specialist tools such as the maxLik or fitdistrplus packages that use optimization
- Global optimization issues, e.g., using package DEoptim, optim with method='SANN', or multiple start wrapper code
- The relative merits of gradient-based vs. derivative-free optimization methods
- Simplex-style algorithms e.g. optim with the default method='Nelder-Mead' but
also some alternative "plex" variants
- Quadratic Approximation methods (using the R package minqa)
- Using gradient methods e.g., optim method='BFGS' with numerically approximated derivatives
Intended audience
R users doing various forms of modelling and estimation who want to
know how to appropriately choose and use the available
optimization tools
and where to find help and advice.
Presenter
The presenter is the author of
Compact
Numerical Methods for Computers, Taylor and Francis, 1990. Three
algorithms from this book are part of the now-aging optim function in R. He has been
recently working (with Ravi Varadhan, Kate Mullen and others) on improved tools and guidance for optimization in R, implementations
of which can be found in the
Optimization
and solving packages project.
Tutorial Materials
Slides are here.