Search

MiS Preprint Repository

We have decided to discontinue the publication of preprints on our preprint server as of 1 March 2024. The publication culture within mathematics has changed so much due to the rise of repositories such as ArXiV (www.arxiv.org) that we are encouraging all institute members to make their preprints available there. An institute's repository in its previous form is, therefore, unnecessary. The preprints published to date will remain available here, but we will not add any new preprints here.

MiS Preprint
87/2018

Restricted Boltzmann Machines: Introduction and Review

Guido Montúfar

Abstract

The restricted Boltzmann machine is a network of stochastic units with undirected interactions between pairs of visible and hidden units. This model was popularized as a building block of deep learning architectures and has continued to play an important role in applied and theoretical machine learning. Restricted Boltzmann machines carry a rich structure, with connections to geometry, applied algebra, probability, statistics, machine learning, and other areas. The analysis of these models is attractive in its own right and also as a platform to combine and generalize mathematical tools for graphical models with hidden variables. This article gives an introduction to the mathematical analysis of restricted Boltzmann machines, reviews recent results on the geometry of the sets of probability distributions representable by these models, and suggests a few directions for further investigation.

Received:
Oct 6, 2018
Published:
Oct 16, 2018
Keywords:
hierarchical model, latent variable model, exponential family, mixture model, Hadamard product, non-negative tensor rank, expected dimension, universal approximation

Related publications

inBook
2016 Repository Open Access
Guido Montúfar

Geometry of Boltzmann machines

In: International conference on information geometry and its applications IV : Liblice, June 12-17, 2016 ; in honor of Shun-ichi Amari / Nihat Ay... (eds.)
Praha : Matfyzpress, 2016. - pp. 25-25
inBook
2018 Repository Open Access
Guido Montúfar

Restricted Boltzmann machines : introduction and review

In: Information geometry and its applications : on the occasion of Shun-ichi Amari's 80th Birthday, IGAIA IV Liblice, Czech Republic, June 2016 / Nihat Ay... (eds.)
Cham : Springer, 2018. - pp. 75-115
(Springer proceedings in mathematics and statistics ; 252)