Multivalued Restricted Boltzmann Machines (Hadamard products of mixtures of Segre products), part II

A Restricted Boltzmann Machine (RBM) is a stochastic network with undirected bipartite interactions between a set of observed variables and a set of hidden variables. We are interested in the geometry of the set of all possible marginal probability distributions of the visible variables. In the first lecture I give a comprehensible survey on mathematical aspects of standard RBMs with binary variables.

In the second lecture I discuss RBMs with discrete, finite valued variables. The multivalued RBM hierarchy of models embraces all mixture models of discrete product distributions as well as the standard binary RBMs, and provides a natural view of semi-restricted Boltzmann machines.

From the perspective of algebraic geometry, these models are Hadamard products of secant varieties of Segre embeddings of products of projective spaces. I present a geometric picture of the 'inference functions' and 'distributed mixtures-of-products' represented by multivalued RBMs, and show results on universal approximation, approximation errors, tropicalization and dimension. Hereby extending previous work on the binary case.