Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks
Contact the author: Please use for correspondence this email.
Submission date: 24. Mar. 2015
Keywords and phrases: stochastic feedforward network, universal approximation, sufficiency bounds
Download full preprint: PDF (318 kB)
We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of n output units, given the states of k ≥ 1 input units, can be approximated arbitrarily well by a network with 2k−1(2n−1 − 1) hidden units.