Hierarchical Models as Marginals of Hierarchical Models

Guido Montúfar and Johannes Rauh

Contact the author: Please use for correspondence this email.
Submission date: 22. Mar. 2016
Pages: 24
published as: Hierarchical models as marginals of hierarchical models.
In: International journal of approximate reasoning, 88 (2017), p. 531-546 
DOI number (of the published article): 10.1016/j.ijar.2016.09.003
published as: Hierarchical models as marginals of hierarchical models.
In: Proceedings of the 10th workshop on uncertainty processing WUPES '15, Moninec, Czech Republic, September 16-19, 2015 / V. Kratochvíl (ed.)
Praha : Oeconomica, 2015. - P. 131 - 145 
Bibtex
Keywords and phrases: hierarchical model, soft-plus network, interaction model, graphical model
Download full preprint: PDF (4074 kB)

Abstract:
We investigate the representation of hierarchical models in terms of marginals of other hierarchical models with smaller interactions. We focus on binary variables and marginals of pairwise interaction models whose hidden variables are conditionally independent given the visible variables. In this case the problem is equivalent to the representation of linear subspaces of polynomials by feedforward neural networks with soft-plus computational units. We show that every hidden variable can freely model multiple interactions among the visible variables, which allows us to generalize and improve previous results. In particular, we show that a restricted Boltzmann machine with [2(log(v) + 1)∕(v + 1)]2^v − 1 hidden binary variables can approximate every distribution of v visible binary variables arbitrarily well, which improves the previous bound 2^v−1 − 1.