We have decided to discontinue the publication of preprints on our preprint server as of 1 March 2024. The publication culture within mathematics has changed so much due to the rise of repositories such as ArXiV (www.arxiv.org) that we are encouraging all institute members to make their preprints available there. An institute's repository in its previous form is, therefore, unnecessary. The preprints published to date will remain available here, but we will not add any new preprints here.
MiS Preprint
27/2016
Hierarchical Models as Marginals of Hierarchical Models
Guido Montúfar and Johannes Rauh
Abstract
We investigate the representation of hierarchical models in terms of marginals of other hierarchical models with smaller interactions. We focus on binary variables and marginals of pairwise interaction models whose hidden variables are conditionally independent given the visible variables. In this case the problem is equivalent to the representation of linear subspaces of polynomials by feedforward neural networks with soft-plus computational units. We show that every hidden variable can freely model multiple interactions among the visible variables, which allows us to generalize and improve previous results. In particular, we show that a restricted Boltzmann machine with $[ 2(\log(v)+1) / (v+1) ] 2^v-1$ hidden binary variables can approximate every distribution of $v$ visible binary variables arbitrarily well, which improves the previous bound $2^{v-1}-1$.
hierarchical model, soft-plus network, interaction model, graphical model
Related publications
inBook
2015
Repository Open Access
Guido Montúfar and Johannes Rauh
Hierarchical models as marginals of hierarchical models
In: Proceedings of the 10th workshop on uncertainty processing WUPES '15, Moninec, Czech Republic, September 16-19, 2015 / Václav Kratochvíl (ed.) Praha : Oeconomica, 2015. - pp. 131-145
