Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks

Guido Montúfar

Contact the author: Please use for correspondence this email.
Submission date: 24. Mar. 2015
Pages: 15
Bibtex
Keywords and phrases: stochastic feedforward network, universal approximation, sufficiency bounds
Download full preprint: PDF (318 kB)

Abstract:
We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of n output units, given the states of k ≥ 1 input units, can be approximated arbitrarily well by a network with 2^k−1(2ⁿ⁻¹ − 1) hidden units.