C/C++ Minpack

What is Minpack?

This is the official description of Minpack, from the original ReadMe file: Minpack includes software for solving nonlinear equations and nonlinear least squares problems. Five algorithmic paths each include a core subroutine and an easy-to-use driver. The algorithms proceed either from an analytic specification of the Jacobian matrix or directly from the problem functions. The paths include facilities for systems of equations with a banded Jacobian matrix, for least squares problems with a large amount of data, and for checking the consistency of the Jacobian matrix with the functions.

The original authors of the FORTRAN version are Jorge More', Burt Garbow, and Ken Hillstrom from Argonne National Laboratory, and the code can be obtained from Netlib.

Minpack is probably the best open-source implementation of the Levenberg-Marquardt algorithm (in fact, it is even better, since it adds to L-M automatic variables scaling). There is another open-source L-M implementation in C/C++, levmar by Manolis Lourakis, but unfortunately is is released under the GPL, which restricts its inclusion in commercial software. Minpack is licensed under a BSD-like license (available in the distribution).

What about CMinpack?

In July 2002 (before levmar), Manolis Lourakis (lourakis at ics forth gr) released a C version of Minpack, called CMinpack, obtained from the FORTRAN version using f2c and some limited manual editing. However, this version had several problems, which came from the FORTRAN version:

  1. All the function prototypes were following the original FORTRAN call conventions, so that all parameters are passed by reference (if a function needs an int parameter, you have to pass a pointer to this int).
  2. There were lots of static variables in the code, thus you could not optimize a function which required calling Minpack to be evaluated (a minimization-of-minimization problem for example): The Minpack code is not reentrant.
  3. If the function to be optimized has to use extra parameters or data (this is the case most of the time), the only way to access them was though global variables, which is very bad, especially if you want to use the same function with different data in different threads: The Minpack code is not MT-Safe.
  4. There was no C/C++ include file.
  5. Examples and tests were missing from the distribution, although there are some FORTRAN examples in the documentation.

Why is C/C++ Minpack better?

I took a dozen of hours to rework all these problems, and came out with a pure C version of Minpack, with has standard (ISO C99) parameters passing, is fully reentrant, multithread-safe, and has a full set of examples and tests:

  1. Input variables are now passed by value, output variables are passed by reference. The keyword "const" is used as much as possible for constant arrays. The return value of each function is now used to get the function status (it was obtained via the IFLAG or INFO parameter in Minpack).
  2. All non-const static variables were removed, and the code was tested after that. Luckily, Minpack didn't use the nastiest feature in FORTRAN: all local variables are static, so that a function can behave differently when you call it several times.
  3. The function to be minimized and all the Minpack functions now take an extra "void*" argument, which can be used to pass any pointer-to-struct or pointer-to-class, and you can put all you extra parameters and data in that struct. Just cast this pointer to the appropriate pointer type in your function, and there they are! There is no need for global variables anymore. Be careful if you access the same object from different threads, though (a solution is to protect this extra data with a mutex).
  4. The Debian project did a C include file for Minpack. It still needed some work (add consts and C++ compatibility), so I did this work, and used the include file for the FORTRAN version as the base for my C/C++ version.
  5. The Debian project also translated all the FORTRAN examples to C. I worked from these to produce examples which also call my C/C++ version of Minpack instead of the FORTRAN version. Also included in the distribution are reference output files produced by the test runs (for comparison).

If you use C/C++ Minpack for a publication, you should cite it as:

@misc{cminpack,
  title={C/C++ Minpack},
  author={Devernay, Fr{\'e}d{\'e}ric},
  year={2007},
  howpublished = "\url{http://devernay.github.io/cminpack}",
}

Distribution

The distribution contains:

It is distributed under the original Minpack license (see the file CopyrightMINPACK.txt in the distribution).

Download

GitHub repository: https://github.com/devernay/cminpack

Building CMinpack

CMinpack can be built with CMake. By default, CMake will build a single precision library named cminpacks, and a double precision library named cminpack. You can choose to build only one of the single, double, or long double precision variants by setting CMINPACK_PRECISION to one of "s", "d", or "ld" respectively (e.g. cmake -DCMINPACK_PRECISION=d ..).

The source distribution also contains a Makefile which can be used to build cminpack for double precision (the default, make or make double) for single-precision (make float), half-precision (make half), long double precision (make longdouble), and even CUDA (make cuda), or using LAPACK for linear algebra (make lapack). Unit tests are also provided, but results may depend on the platform (make checkdouble checkfloat checkhalf checklongdouble checklapack).

Using CMinpack

The CMinpack calls have the same name as the FORTRAN functions, in lowercase (e.g. lmder(...)). See the links to the documentation below, or take a look at the simple examples in the examples directory of the distribution. The simple examples are named after the function they call: tlmder.c is the simple example for lmder.

If you want to use the single precision CMinpack, you should define __cminpack_float__ before including cminpack.h. __cminpack_half__ has to be defined for the half-precision version (and the code needs to be compiled with a C++ compiler).

The single-precision versions of the functions are prefixed by "s" (as in "slmder(...)"), and the half-precision are prefixed by "h".

CMinpack defines __cminpack_real__ as the floating point type, and the __cminpack_func__() macro can be used to call CMinpack functions independently of the precision used (as in the examples). However, you shouldn't use these macros in your own code, since your code is probably designed for a specific precision, and you should prefer calling directly lmder(...) or slmder(...).

Documentation

References

LMDER/LMDIF

HYBRJ/HYBRD

Simulating box constraints

Note that box constraints can easily be simulated in C++ Minpack, using a change of variables in the function (that hint was found in the lmfit documentation).

For example, say you want xmin[j] < x[j] < xmax[j], just apply the following change of variable at the beginning of fcn on the variables vector, and also on the computed solution after the optimization was performed:

  for (j = 0; j < 3; ++j) {
    real xmiddle = (xmin[j]+xmax[j])/2.;
    real xwidth = (xmax[j]-xmin[j])/2.;
    real th =  tanh((x[j]-xmiddle)/xwidth);
    x[j] = xmiddle + th * xwidth;
    jacfac[j] = 1. - th * th;
  }

This change of variables preserves the variables scaling, and is almost the identity near the middle of the interval.

Of course, if you use lmder, lmder1, hybrj or hybrj1, the Jacobian must be also consistent with that new function, so the column of the original Jacobian corresponding to x1 must be multiplied by the derivative of the change of variable, i.e jacfac[j].

Similarly, each element of the covariance matrix must be multiplied by jacfac[i]*jacfac[j].

For examples on how to implement this in practice, see the portions of code delimited by "#ifdef BOX_CONSTRAINTS" in the following source files: tlmderc.c, thybrj.c, tchkderc.c.

Equivalence table with other libraries

The following table may be useful if you need to switch to or from another library.

Equivalence table between MINPACK and NAG, NPL, SLATEC, levmar and GSL
MINPACK NAG NPL SLATEC levmar GSL
lmdif E04FCF LSQNDN DNLS1 dlevmar_dif
lmdif1 E04FYF LSNDN1 DNLS1E dlevmar_dif
lmder E04GDF/E04GBF LSQFDN DNLS1 dlevmar_der gsl_multifit_fdfsolver_lmsder
lmder1 E04GZF LSFDN2 DNLS1E dlevmar_der
lmstr * * DNLS1
hybrd C05NCF * DNSQ gsl_multiroot_fsolver_hybrids
hybrd1 C05NBF * DNSQE
hybrj C05PCF * DNSQ gsl_multiroot_fdfsolver_hybridsj
hybrj1 C05PBF * DNSQE
covar E04YCF * DCOV gsl_multifit_covar
chkder C05ZAF

Other MINPACK implementations

History

Future work

There is now a very powerful alternative to MINPACK, which is the Ceres Solver. You may want to consider using Ceres for any new project.

The main feature that's missing on cminpack is the possibility to add constraints on variables. Simple boundary constraints should be enough, as implemented in ALGLIB or MPFIT, and they can easily be implemented using the hack above (section "Simulating box constraints").

levmar also has linear constraints, but they should not be necessary since linear constraints can be changed to box constraints by a simple change of variables. If you really need nonlinear constraints, and no reparameterization of variables (which may be able to linearize these constraints), you should consider using NLopt instead of cminpack.

Please file a GitHub issue for any suggestion or request.

Frédéric Devernay

Valid XHTML 1.1! Valid CSS! Level Double-A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0