The following software is in general available under the GNU public license.
I prefer this license: it is simple, so lobbyists don't like it (and I don't
like lobbyists). Free software will die without something like the GPL.
The MEX files should work for Matlab 6.5 and newer, under Linux. They have
not been tested with other configurations. If you have problems compiling
them, you might want to check the following points:
I maintain a C++ class library supporting the development of efficient machine
learning code. If you are fed up with Matlab, like C++, and are
prepared to make some effort, this is for you.
[homepage]
License: GNU General Public License.
Set of Matlab and MEX functions. If you want to run other Matlab software by
me, you need these somewhere in your Matlab path.
The FST package is part of the essential package. It offers
a set of functions to speed up elementary full and sparse matrix commands.
It contains operators, which allow to avoid copying of large matrices.
[tar.gz]
Note: Also check out Tom Minka's lightspeed package for speeding up Matlab.
License: GNU General Public License.
Efficient implementation of penalized multiple logistic regression (aka
multi-class) with Mercer kernels, aka MAP approximation to the multi-class
Gaussian process model. This includes hyperparameter learning using an
approximation to the cross-validation log likelihood (you need an optimizer
to use this part). Runs Newton-Raphson with preconditioned conjugate
gradients as inner loop. The training complexity scales somewhat as O(n^2 C).
The basic operation is a matrix-vector multiplication with the kernel
matrices. This primitive can easily be replaced by customized code. Low-rank
kernel matrix approximations are supported. If you extend the package in
an interesting way, make it public (you have to under the license!) and send
me a link.
[tar.gz]
Note: Requires essential package.
Note: You need incomplete Cholesky decomposition code to run the low-rank matrix approximation part.
Note: For a much faster and slightly more general implementation,
consider LHOTSE (klr project).
License: GNU General Public License.
The incomplete Cholesky decomposition for a dense symmetric positive
definite matrix A is a simple way of approximating A by a matrix of
low rank (you can choose the rank). It has been used frequently in
machine learning (Fine, Scheinberg; Bach, Jordan). Here is an efficient
implementation. Supported kernels in the moment are RBF (Gaussian) and
squared-exponential, I might add some more if I need them. The package is
well-documented and it is very easy to add other kernels or matrix types.
[tar.gz]
Note: Requires essential package.
License: GNU General Public License.
You should never compute, maintain, or update the inverse of a
symmetric positive definite matrix if you do not have to (and, please, in these
cases do not talk about matrix inversion in your papers: people will just
literally implement it that way!). Computing the inverse or manipulating
it (by Sherman-Morrison-Woodbury) is inherently instable. You can almost always
do what you want to do using the Cholesky decomposition instead, and it is
really simple.
Unfortunately, Matlab does not support this, and while LINPACK
contains some code, these routines are not officially supported in the
successor LAPACK. Here is some code for stable updating/downdating of a
Cholesky factor after simple changes to the system matrix. We slightly
adapted the LINPACK code. Our modifications leave the routines numerically
identical, while making them faster (by using BLAS routines).
[tar.gz]
Here is a technical report, describing
the methods and showing how they can be used to purge
machine learning code from unnecessary inverse bombshells. If you use
the code provided here, you should cite this TR and the location where you
obtained the code.
Note: Requires essential
package.