Universality of Generalizations of Restricted Boltzmann Machines

Restricted Boltzmann machines (RBM) are graphical models that can be used to model probability distributions on the corners of the n dimensional unit cube. It can be shown that the distribution modeled by an RBM with m hidden nodes can parameterized as the Hadamard product of m tensors (called factors) that are super modular and have flattening rank at most 2. Inspired by this we look at two different models obtained by either removing the super modularity or the flattening rank condition. For both models, we try to answer the following question. Given an arbitrary tensor p what is the maximum number of tensors m so that p can be represented as the Hadamard product of m factors? We show that if p has strictly positive entries then it can be represented using n super modular factors. On the hand, for the model where our factors have flattening rank at most 2, we present lower and upper bounded for m. Using the proof idea, if we look at representing M by N matrices as Hadamard products of m rank r matrices, we present a new proof for the lower bound from Friedenberg, Onetto, and Williams 2017.