Restricted Boltzmann Machines: Introduction and Review

Guido Montúfar

Contact the author: Please use for correspondence this email.
Submission date: 06. Oct. 2018
Pages: 43
published as: Geometry of Boltzmann machines.
In: International conference on information geometry and its applications IV : Liblice, June 12-17, 2016 ; in honor of Shun-ichi Amari / N. Ay ... (eds.)
Praha : Matfyzpress, 2016. - P. 25 
published as: Restricted Boltzmann machines : introduction and review.
In: Information geometry and its applications : on the occasion of Shun-ichi Amari's 80th Birthday, IGAIA IV Liblice, Czech Republic, June 2016 / N. Ay ... (eds.)
Cham : Springer, 2018. - P. 75 - 115
(Springer proceedings in mathematics and statistics ; 252) 
DOI number (of the published article): 10.1007/978-3-319-97798-0_4
Bibtex
Keywords and phrases: hierarchical model, latent variable model, exponential family, mixture model, Hadamard product, non-negative tensor rank, expected dimension, universal approximation
Download full preprint: PDF (370 kB)
Link to arXiv: See the arXiv entry of this preprint.

Abstract:
The restricted Boltzmann machine is a network of stochastic units with undirected interactions between pairs of visible and hidden units. This model was popularized as a building block of deep learning architectures and has continued to play an important role in applied and theoretical machine learning. Restricted Boltzmann machines carry a rich structure, with connections to geometry, applied algebra, probability, statistics, machine learning, and other areas. The analysis of these models is attractive in its own right and also as a platform to combine and generalize mathematical tools for graphical models with hidden variables. This article gives an introduction to the mathematical analysis of restricted Boltzmann machines, reviews recent results on the geometry of the sets of probability distributions representable by these models, and suggests a few directions for further investigation.