Deep Narrow Boltzmann Machines are Universal Approximators

Guido Montúfar

Contact the author: Please use for correspondence this email.
Submission date: 14. Nov. 2014
Pages: 20
Bibtex
Download full preprint: PDF (347 kB)

Abstract:
We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. Besides from this existence statement, we provide upper and lower bounds on the sufficient number of layers and parameters. These bounds show that deep narrow Boltzmann machines are at least as compact universal approximators as restricted Boltzmann machines and narrow sigmoid belief networks, with respect to the currently available bounds for those models.