## Tutorial: Optimization and related nonlinear modelling computations in R

John C. Nash, Tefler School of Management, University of Ottawa (retired), Canada

### Abstract

We will present an overview of the tools in R for optimization and related tasks. Discussion will be by examples, and participants or others are welcome to send problems in advance for possible inclusion in the discussion.

There are many uses of optimization in statistics, especially in model fitting. However, numerical optimization of nonlinear functions, particularly with constraints, is a rich and intricate subject where statisticians are rarely experts. The tutorial will attempt to give R users a handle on the available packages and functions for solving optimization problems in R.

### Outline

Largely using the optimx package (from the Optimization and solving packages project) that allows a number of the existing optimization tools to be accessed via a common front-end, we will discuss via examples:
• Writing an objective function
• Scaling and constraints
• Starting values, and
• Interpretation of results
Example issues that may be addressed to provide participants with an informed view of the optimization landscape are:
• Optimization with box-constraints (bounds) as well as temporarily fixed parameters (masks). optimx can call optim with method='L-BFGS-B', nlminb, bobyqa from package minqa, spg from package BB, or Rcgmin among other approaches
• Nonlinear regression with the objective function defined using an expression as in the function nls, or via a residual function as in the package minpack.lm
• Linear regression with constraints, comparing minimization of an objective function to the use of specialist packages such as nnls, bvls and quadprog
• Issues for development of specialist tools such as the maxLik or fitdistrplus packages that use optimization
• Global optimization issues, e.g., using package DEoptim, optim with method='SANN', or multiple start wrapper code
• The relative merits of gradient-based vs. derivative-free optimization methods
• Simplex-style algorithms e.g. optim with the default method='Nelder-Mead' but also some alternative "plex" variants
• Quadratic Approximation methods (using the R package minqa)
• Using gradient methods e.g., optim method='BFGS' with numerically approximated derivatives

### Intended audience

R users doing various forms of modelling and estimation who want to know how to appropriately choose and use the available optimization tools and where to find help and advice.

### Presenter

The presenter is the author of Compact Numerical Methods for Computers, Taylor and Francis, 1990. Three algorithms from this book are part of the now-aging optim function in R. He has been recently working (with Ravi Varadhan, Kate Mullen and others) on improved tools and guidance for optimization in R, implementations of which can be found in the Optimization and solving packages project.

Slides are here.