LHOTSE

Toolbox for Adaptive

Statistical Models

NOTE: This page is still under development!

Synopsis

LHOTSE is a C++ class library designed for the implementation of large, efficient scientific applications in Machine Learning and Statistics. At present, LHOTSE can be used to create stand-alone executables, or as a class library within a larger project, but interfaces to Python and (possibly) to MATLAB(R) are planned.

LHOTSE has been designed and implemented by:

Matthias W. Seeger
MMCI Cluster of Excellence
Max Planck Institut fuer Informatik
Room 116, Campus E1 4
66123 Saarbruecken
Phone: ++49-681-9325452
people.mmci.uni-saarland.de/~mseeger/

Acknowledgments can be found in the THANKS file in any LHOTSE source package.

LHOTSE is free software, released under the GNU General Public License (a copy of the license comes with the sources). In the moment, it has been tested under Linux, using the GNU compilers gcc, g++, and g77. It comes with a GNU build system, ensuring a fairly simple installation.
The next section tells you about LHOTSE, why you may want to use it, or even contribute to it. You can skip this and download it directly. Here is some documentation.

What Is It?

Why You (Really) Should Use LHOTSE

Many researchers in Machine Learning come to a point where a project of theirs becomes too large and/or too computationally intensive to reasonably handle with prototyping tools such as MATLAB(R). They may find that they have to spend a lot of their time getting things just about running, using ever-weirder tricks to mask shortcomings of their tool, which lacks the structure of modern programming languages. This is their time wasted. However, re-coding their project in C++ may be a huge effort. The nice handling of their prototyping tool is not present in C++. While a lot of high-quality free software is available for scientific computation, every single package comes with its own sophisticated interface. Most of the packages are written in Fortran, and calling them from C++ is awkward. Finally, many such packages are extremely specialized, while Machine Learning is a very heterogenous field, often demanding the joint usage of techniques from different fields of scientific computation.

LHOTSE is a Machine Learning researcher's attempt to simplify development of scientific applications, while at the same time not to sacrifice running time and memory efficiency. Some of its strong points are

Transparency
The design of LHOTSE is kept as simple as possible. Learning its interfaces does not require months of study. All public interfaces are extensively documented. Some user convenience is sacrified in order to avoid complicated hidden mechanisms with their own inefficencies opaque to the developer, such as garbage collectors, arithmetic operator overloading. External packages are wrapped in simple selective interfaces, following general LHOTSE conventions, rather than trying to wrap every single feature.
Consequent use of object oriented design and modern features of C++ (templates, function objects, inlining, STL).
This allows the management of large projects (with many developers), which can easily be extended, while efficiency is not compromised.
Provision of low-level classes (smart pointers, access controlled memory regions), which are used by all other high-level classes.
This basically eliminates major bug sources in C / C++ due to memory allocation / deallocation, or boundary violations.
Matrix / vector functionality
LHOTSE provides extensive classes for matrices and vectors. All BLAS and some of the LAPACK routines are wrapped using an easy-to-use interface.
Vectorization (application of scalar functions to vectors and matrices) and accumulation is supported through C++ function objects (compiler inlining avoids losses in performance), eliminating the need for loops over elements (bug source, loss of code readability) entirely.
Masking allows to treat parts of matrices / vectors as matrices / vectors themselves, without the need for temporary copies. This allows convenience close to MATLAB(R), without its inefficencies.
Wrapping of scientific code is simple and robust.
LHOTSE wraps foreign code through well-defined C++ interfaces. Its matrix and vector datatypes do internally have the same structure as required by Fortran and BLAS, so foreign code can be called directly without expensive type conversions.
Provision of techniques required in Machine Learning and Statistics.
Features currently provided include:
- Matrix / vector classes, providing access to all of BLAS and some LAPACK methods (to be extended on demand).
- Optimizers
  LHOTSE includes code for linear conjugate gradients, nonlinear Quasi-Newton, and certain double loop optimizers. It wraps the L-BFGS-B limited memory Quasi-Newton code of Zhu, Byrd, Lu-Chen, Nocedal.
  All optimizers communicate with the objective function through a generic criterion function interface.
- Code for implementing Gaussian process techniques.
- A hierarchy of covariance functions (kernels), which allows to implement new kernels quickly using generic code, and to run kernel machine code with arbitrary kernels.
- Generic hierarchies for dataset representations, allowing for arbitrary data types being loaded, represented in memory or temporary files, and being accessed as variables by statistical methods.
- Code for incomplete Cholesky factorization, used in Machine Learning to speed up kernel methods.
- Code for stable low-rank Cholesky factorization updates.
- Scientific functions from the GNU numerical library.
- Code for uni- and multivariate quadrature, used for dealing with non-Gaussian likelihood or prior factors in statistical models.

What We Would Like You To Do With LHOTSE

Do refer to the LHOTSE web site.
Whenever you do something with LHOTSE, please acknowledge us by refering to the LHOTSE web site in your publications, source code, or project web sites.
Note that individual LHOTSE projects may come with additional requirements of this sort (they will all be free software, owing to the GPL).

Code your next project using LHOTSE.
In this case, if you have a project web page, please notify the LHOTSE developers.

Wrap your old project code for use in LHOTSE.
Again, please notify the LHOTSE developers of this. It might just make more people use your code.

Publish your code as a LHOTSE project.
The advantage of this is that other people using LHOTSE can install and use your project easily, by using the LHOTSE installation system. If you follow the LHOTSE style of structuring your project, and especially the LHOTSE style of documentation of class interfaces, other people working with LHOTSE may even be able to understand the workings of your project.
Note that LHOTSE is simply a toolbox, and publishing your code in form of a LHOTSE package is not much different from publishing MATLAB(R) code, in that it is just a bit more convenient for others to use.
Note also that publishing your own package in form of a LHOTSE project is different (in fact, independent) from extending the complete LHOTSE distribution. The latter would require you to become a LHOTSE developer, and to adhere to the LHOTSE style, documentation, and coding conventions. For the former, you do not need to do any of this.
More information about packaging a project is here.
Develop new LHOTSE features.
LHOTSE's key design principle is to add features on demand, simply because most of the LHOTSE developers cannot spend their time adding features they do not need. This is why LHOTSE most probably does not contain your favourite algorithm X for doing Y. Therefore, if you have added some features not yet present, or wrapped some numerical code (public or proprietary), please do contact the LHOTSE developers, who may consider adding your code to the official LHOTSE main library.
You may want to become a LHOTSE developer yourself. Help with this is highly welcome in general!
Our plans on top of the todo list are, apart from writing a bit more intro documentation:
- Interface to Python, to allow for interactive development.
- Interface to MATLAB(R) (unfortunately, this is what most people use). A prototype exists already.
- Simple extensions, like more random number methods
- Publishing a couple of LHOTSE projects of ours, with proper documentation

Download

Note that the present version of LHOTSE is not a stable release yet, in the sense that extensive test suites have not been written for any part of the code, and some of the code has not been tested under sufficiently many situations. Core parts of LHOTSE, such as the global and matrix libraries modules, have been refined over more than three years, and have been used in many different situations.

Complete LHOTSE package, version 0.16: lhotse-0.16.tar.gz

LHOTSE library package, version 0.16: lhotse-0.16-lib.tar.gz

LHOTSE ivmsing project package, version 0.16: lhotse-0.16-ivmsing.tar.gz

LHOTSE klr project package, version 0.16: lhotse-0.16-klr.tar.gz

Installation Of Complete LHOTSE Package

The installation of LHOTSE follows the GNU conventions. Place the tarball in some location, and unpack it using

tar xzf lhotse-VER.tar.gz

where VER is the LHOTSE version. Then, change to subdirectory lhotse-VER and read the INSTALL file for further instructions. Also, make sure to read the README file.
LHOTSE requires some external libraries. The INSTALL file describes how these may be obtained. The links mentioned there for a customized installation are collected here. For a trial installation of LHOTSE, you might want to use the precompiled library variants provided below.
The installation will buid the LHOTSE main libraries and executables for the projects currently shipped with LHOTSE. There are some examples you can run to get started.
It is important to keep the LHOTSE source tree around if you want to develop own projects in LHOTSE.

Selective Installation Of LHOTSE

The complete LHOTSE package contains a number of projects currently shipped with LHOTSE. You may also install projects selectively.
Any LHOTSE project requires the LHOTSE libraries, packaged as lhotse-VER-lib.tar.gz. Unpacking writes files into subdirectory lhotse-VER, the library sources lie in lhotse-VER/lhotse. The installation of the LHOTSE library package works in the same way as the installation of the complete package.

Afterwards, individual projects can be installed, packaged as lhotse-VER-PACK.tar.gz, where PACK is the package name. Unpacking writes files into subdirectory lhotse-VER, the project sources lie in lhotse-VER/src.
You can unpack the project sources at the same location as the library sources, or you can keep the library and project source trees separate. The latter is the preferred option, allowing you to remove projects without removing the library source tree, or to install new library sources without removing projects. It also avoids name clashes between different projects.
The installation of a LHOTSE project is described in section "Selective LHOTSE Installation" in file INSTALL.

Links Useful For Obtaining External Libraries

LHOTSE is using a number of external libraries, which you may have to install before you can install LHOTSE. This is described in the INSTALL file. The links mentioned there are collected below:

GNU Scientific Library
ATLAS: Self-Tuning Implementation of BLAS
LAPACK
Intel MKL Libraries
This is for non-commercial development only.

Precompiled Libraries

LHOTSE is using a number of external libraries, which are not always available on a Linux system. Details about how to obtain these libraries are given in the INSTALL file. We recommend that you follow these instructions, noting that the running time performance of LHOTSE applications can depend crucially on whether the external libraries are optimized for the target architecture.
We have collected precompiled variants of the required external libraries for some frequently used architectures here. If you just want to give LHOTSE a try, without having to install missing external libraries properly, you may use these. Note that this is just a working solution. If you intend to use LHOTSE or any LHOTSE application seriously, you should not use these variants.
The tarballs are good for a local installation of libraries (do not install the precompiled variants globally in your system!), starting from some directory DIR in your home. They will create subdirectories DIR/lib and DIR/include. To install a tarball foo.tar.gz, place it into DIR, and unpack it using

tar xzf foo.tar.gz

Repeat this for all tarballs you require. In the LHOTSE installation, you need to pass the library location as configure options:

--enable-lib-paths="-LDIR/lib" --enable-inc-paths="-IDIR/include"

These are described in more detail in section "Installation with Non-Standard Libraries" of the INSTALL file.

GNU Scientific Library (GSL)
32-bit, 64-bit
BLAS (ATLAS), LAPACK
The 32-bit variants are for Pentium IV (SSE2).
32-bit, 64-bit

Projects Contained In The LHOTSE Distribution

The full LHOTSE distribution is shipped with a number of projects which make use of LHOTSE. The installation process creates corresponding executables.
At present, these projects are not completely finalized, in that a user documentation still needs to be written, and further testing is probably required. This should not stop you, since the main code file tells you about how to control the program, and the LHOTSE are using the code since some time for their own work. Please do also look at the project sources, to see how they use LHOTSE features to get large computations done.

Documentation

A developers manual, describing essential LHOTSE features and classes, as well as matrix and vector support, is planned, but not yet available. In the meantime, the file doc/develop-hints.txt will get developers started. A copy of this file is here. All LHOTSE interfaces (exported classes) are extensively documented in the source code, using the JavaDoc convention. You can use doxygen to generate HTML and LaTeX documentations from the sources:

doxygen lhotse-doc.conf

This generates documentation for the LHOTSE main libraries, but not for the projects shipped with LHOTSE. For the HTML documentation, direct your browser to doc/doxy/html/index.html. The LaTeX documentation is in doc/doxy/latex. Depending on your taste, it is maybe easier to browse the LHOTSE class header files, where the comments can be found directly.

Examples

The current LHOTSE distribution comes with a number of examples, described in the file examples/README.

Develop Own Projects In LHOTSE

This will be described in more detail in the forthcoming developers manual. At present, some hints to get you started, can be found in doc/develop-hints.txt. A copy of this file is here. You should also browse through the example sources.
We encourage you to package your project sources in form of a LHOTSE project. The process of doing so will be described in the developers manual. At present, doc/develop-hints.txt gives you some hints. Do contact the LHOTSE developers if you have questions about how to package code of yours as a LHOTSE project.