Search

MiS Preprint Repository

We have decided to discontinue the publication of preprints on our preprint server as of 1 March 2024. The publication culture within mathematics has changed so much due to the rise of repositories such as ArXiV (www.arxiv.org) that we are encouraging all institute members to make their preprints available there. An institute's repository in its previous form is, therefore, unnecessary. The preprints published to date will remain available here, but we will not add any new preprints here.

MiS Preprint
106/2014

Discrete Restricted Boltzmann Machines

Guido Montúfar and Jason Morton

Abstract

We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables. Examples are binary restricted Boltzmann machines and discrete naive Bayes models. We detail the inference functions and distributed representations arising in these models in terms of configurations of projected products of simplices and normal fans of products of simplices. We bound the number of hidden variables, depending on the cardinalities of their state spaces, for which these models can approximate any probability distribution on their visible states to any given accuracy.

In addition, we use algebraic methods and coding theory to compute their dimension.

Received:
Oct 13, 2014
Published:
Oct 14, 2014
MSC Codes:
51M20, 60C05, 68Q32, 14Q15
Keywords:
restricted Boltzmann machine, Naive Bayes Model, Representational Power, Distributed Representation, expected dimension

Related publications

inJournal
2015 Journal Open Access
Guido Montúfar and Jason Morton

Discrete restricted Boltzmann machines

In: Journal of machine learning research, 16 (2015), pp. 653-672