Mathematical Machine Learning

Head:
Guido Montúfar (Email)

Phone:
+49 (0) 341 - 9959 - 880

Fax:
+49 (0) 341 - 9959 - 658

Address:
Inselstr. 22
04103 Leipzig

Publications Guido Montufar

Journal Articles To top of page

Li, Wuchen and Montúfar, Guido: Natural gradient via optimal transport
In: Information geometry, 1 (2018) 2, p. 181-214
Bibtex DOI: 10.1007/s41884-018-0015-3 ARXIV: https://arxiv.org/abs/1803.07033
Green Open Access

Seigal, Anna and Montúfar, Guido: Mixtures and products in two graphical models
In: Journal of algebraic statistics, 9 (2018) 1, p. 1-20
Bibtex DOI: 10.18409/jas.v9i1.90 ARXIV: https://arxiv.org/abs/1709.05276
Gold Open Access

Montúfar, Guido and Morton, Jason: Dimension of marginals of Kronecker product models
In: SIAM journal on applied algebra and geometry, 1 (2017) 1, p. 126-151
Bibtex MIS-Preprint: 75/2015 DOI: 10.1137/16M1077489 ARXIV: http://arxiv.org/abs/1511.03570
Gold Open Access

Montúfar, Guido and Rauh, Johannes: Hierarchical models as marginals of hierarchical models
In: International journal of approximate reasoning, 88 (2017), p. 531-546
Bibtex MIS-Preprint: 27/2016 DOI: 10.1016/j.ijar.2016.09.003 ARXIV: http://arxiv.org/abs/1508.03606
Green Open Access

Ghazi-Zahedi, Keyan ; Haeufle, Daniel F. B. ; Montúfar, Guido ; Schmitt, Syn and Ay, Nihat: Evaluating morphological computation in muscle and DC-motor driven models of hopping movements
In: Frontiers in robotics and AI, 3 (2016), 42
Bibtex DOI: 10.3389/frobt.2016.00042 ARXIV: http://arxiv.org/abs/1512.00250
Gold Open Access

Montúfar, Guido and Rauh, Johannes: Mode poset probability polytopes
In: Journal of algebraic statistics, 7 (2016) 1, p. 1-13
Bibtex MIS-Preprint: 22/2015 DOI: 10.18409/jas.v7i1.41 ARXIV: http://arxiv.org/abs/1503.00572
Gold Open Access

Montúfar, Guido ; Ay, Nihat and Ghazi-Zahedi, Keyan: Geometry and expressive power of conditional restricted Boltzmann machines
In: Journal of machine learning research, 16 (2015), p. 2405-2436
Bibtex MIS-Preprint: 16/2014 ARXIV: http://arxiv.org/abs/1402.3346 LINK: http://www.jmlr.org/papers/v16/montufar15b.html
Gold Open Access

Montúfar, Guido ; Ghazi-Zahedi, Keyan and Ay, Nihat: A theory of cheap control in embodied systems
In: PLoS computational biology, 11 (2015) 9, e1004427
Bibtex MIS-Preprint: 70/2014 DOI: 10.1371/journal.pcbi.1004427 ARXIV: http://arxiv.org/abs/1407.6836
Gold Open Access

Montúfar, Guido and Morton, Jason: Discrete restricted Boltzmann machines
In: Journal of machine learning research, 16 (2015), p. 653-672
Bibtex MIS-Preprint: 106/2014 ARXIV: http://arxiv.org/abs/1301.3529 LINK: http://jmlr.org/papers/v16/montufar15a.html
Gold Open Access

Montúfar, Guido and Morton, Jason: When does a mixture of products contain a product of mixtures?
In: SIAM journal on discrete mathematics, 29 (2015) 1, p. 321-347
Bibtex MIS-Preprint: 98/2014 DOI: 10.1137/140957081 ARXIV: http://arxiv.org/abs/1206.0387
Green Open Access

Montúfar, Guido: Universal approximation depth and errors of narrow belief networks with discrete units
In: Neural computation, 26 (2014) 7, p. 1386-1407
Bibtex MIS-Preprint: 74/2014 DOI: 10.1162/NECO_a_00601 ARXIV: http://arxiv.org/abs/1303.7461
Green Open Access

Montúfar, Guido and Rauh, Johannes: Scaling of model approximation errors and expected entropy distances
In: Kybernetika, 50 (2014) 2, p. 234-245
Bibtex DOI: 10.14736/kyb-2014-2-0234 ARXIV: http://de.arxiv.org/abs/1207.3399
Gold Open Access

Montúfar, Guido ; Rauh, Johannes and Ay, Nihat: On the Fisher metric of conditional probability polytopes
In: Entropy, 16 (2014) 6, p. 3207-3233
Bibtex MIS-Preprint: 87/2014 DOI: 10.3390/e16063207 ARXIV: http://arxiv.org/abs/1404.0198
Gold Open Access

Krüger, Tyll ; Montúfar, Guido ; Seiler, Ruedi and Siegmund-Schultze, Rainer: Universally typical sets for ergodic sources of multidimensional data
In: Kybernetika, 49 (2013) 6, p. 868-882
Bibtex MIS-Preprint: 20/2011 ARXIV: http://arxiv.org/abs/1105.0393
Gold Open Access

Montúfar, Guido: Mixture decompositions of exponential families - using a decomposition of their sample spaces
In: Kybernetika, 49 (2013) 1, p. 23-39
Bibtex MIS-Preprint: 39/2010 ARXIV: http://de.arxiv.org/abs/1008.0204 LINK: http://www.kybernetika.cz/content/2013/1/23
Gold Open Access

Montúfar, Guido and Ay, Nihat: Refinements of universal approximation results for deep belief networks and restricted Boltzmann machines
In: Neural computation, 23 (2011) 5, p. 1306-1319
Bibtex MIS-Preprint: 23/2010 DOI: 10.1162/NECO_a_00113 ARXIV: https://arxiv.org/abs/1005.1593
Green Open Access

Publications in Books and Conference Proceedings To top of page

Dukler, Yonatan ; Li, Wuchen ; Lin, Alex Tong and Montúfar, Guido: Wasserstein of Wasserstein loss for learning generative models
In: Proceedings of the 36th international conference on machine learning, 9-15 June 2019, Long Beach, California, USA / Kamalika Chaudhuri (ed.)
Long Beach, California : PMLR, 2019. - P. 1716-1725
(Proceedings of machine learning research ; 97)
Bibtex MIS-Preprint: 13/2019 LINK: http://proceedings.mlr.press/v97/dukler19a.html
Green Open Access

Banerjee, Pradeep Kumar ; Rauh, Johannes and Montúfar, Guido: Computing the unique information
In: IEEE international symposium on information theory (ISIT) from June 17 to 22, 2018 at the Talisa Hotel in Vail, Colorado, USA
Piscataway, NY : IEEE, 2018. - P. 141-145
Bibtex MIS-Preprint: 73/2017 DOI: 10.1109/ISIT.2018.8437757 ARXIV: https://arxiv.org/abs/1709.07487
Green Open Access

Montúfar, Guido: Restricted Boltzmann machines : introduction and review
In: Information geometry and its applications : on the occasion of Shun-ichi Amari's 80th Birthday, IGAIA IV Liblice, Czech Republic, June 2016 / Nihat Ay... (eds.)
Cham : Springer, 2018. - P. 75-115
(Springer proceedings in mathematics and statistics ; 252)
Bibtex MIS-Preprint: 87/2018 DOI: 10.1007/978-3-319-97798-0_4 ARXIV: https://arxiv.org/abs/1806.07066
Green Open Access

Ghazi-Zahedi, Keyan ; Deimel, Raphael ; Montúfar, Guido ; Wall, Vincent and Brock, Oliver: Morphological computation : the good, the bad, and the ugly
In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) : Vancouver, BC, Canada ; September 24-28, 2017
New York, NY : IEEE, 2017. - P. 464-469
Bibtex DOI: 10.1109/IROS.2017.8202194 LINK: https://www.researchgate.net/publication/318710693_Morphological_Computation_The_Good_the_Bad_and_the_Ugly
Green Open Access

Montúfar, Guido and Rauh, Johannes: Geometry of policy improvement
In: Geometric science of information : Third International Conference, GSI 2017, Paris, France, November 7-9, 2017, proceedings / Frank Nielsen... (eds.)
Cham : Springer, 2017. - P. 282-290
(Lecture notes in computer science ; 10589)
Bibtex DOI: 10.1007/978-3-319-68445-1_33 ARXIV: https://arxiv.org/abs/1704.01785
Green Open Access

Montúfar, Guido: Geometry of Boltzmann machines
In: International conference on information geometry and its applications IV : Liblice, June 12-17, 2016 ; in honor of Shun-ichi Amari / Nihat Ay... (eds.)
Praha : Matfyzpress, 2016. - P. 25-25
Bibtex MIS-Preprint: 87/2018 ARXIV: https://arxiv.org/abs/1806.07066
Green Open Access

Montúfar, Guido: Deep narrow Boltzmann machines are universal approximators
In: Third international conference on learning representations - ICLR 2015 : May 7-9 2015, San Diego, CA. USA
San Diego : ICLR, 2015
Bibtex MIS-Preprint: 113/2014 ARXIV: http://arxiv.org/abs/1411.3784
Green Open Access

Montúfar, Guido and Rauh, Johannes: Mode poset probability polytopes
In: Proceedings of the 10th workshop on uncertainty processing WUPES '15, Moninec, Czech Republic, September 16-19, 2015 / Václav Kratochvíl (ed.)
Praha : Oeconomica, 2015. - P. 147-154
Bibtex MIS-Preprint: 22/2015 ARXIV: http://arxiv.org/abs/1503.00572 LINK: http://wupes.fm.vse.cz/2015/data/Proceedings.pdf
Green Open Access

Montúfar, Guido and Rauh, Johannes: Hierarchical models as marginals of hierarchical models
In: Proceedings of the 10th workshop on uncertainty processing WUPES '15, Moninec, Czech Republic, September 16-19, 2015 / Václav Kratochvíl (ed.)
Praha : Oeconomica, 2015. - P. 131-145
Bibtex MIS-Preprint: 27/2016 ARXIV: http://arxiv.org/abs/1508.03606 LINK: http://wupes.fm.vse.cz/2015/data/Proceedings.pdf
Green Open Access

Montúfar, Guido and Morton, Jason: Geometry of hidden-visible products of statistical models
In: Algebraic Statistics 2014 : May 19-22
Chicago, IL : Illinois Institute of Technology, 2014
Bibtex LINK: http://mypages.iit.edu/~as2014/abstracts.html#Montufar

Montúfar, Guido ; Pascanu, Razvan ; Cho, Kyunghyun and Bengio, Yoshua: On the number of linear regions of deep neural networks
In: NIPS'14 Proceedings of the 27th international conference on neural information processing systems - volume 2 ; Montreal, Quebec, Canada, December 8th-13th
Cambridge, MA : MIT Press, 2014. - P. 2924-2932
Bibtex MIS-Preprint: 73/2014 ARXIV: http://arxiv.org/abs/1402.1869 LINK: https://papers.nips.cc/paper/5422-on-the-number-of-linear-regions-of-deep-neural-networks
Green Open Access

Pascanu, Razvan ; Montúfar, Guido and Bengio, Yoshua: On the number of response regions of deep feedforward networks with piecewise linear activations
In: Second international conference on learning representations - ICLR 2014 : 14-16 April 2014, Banff, Canada
Banff : ICLR, 2014
Bibtex MIS-Preprint: 72/2014 ARXIV: http://arxiv.org/abs/1312.6098
Green Open Access

Ay, Nihat ; Montúfar, Guido and Rauh, Johannes: Selection criteria for neuromanifolds of stochastic dynamics
In: Advances in cognitive neurodynamics III : proceedings of the 3rd International Conference on Cognitive Neurodynamics 2011 ; [June 9-13, 2011, Hilton Niseko Village, Hokkaido, Japan] / Yoko Yamaguchi (ed.)
Dordrecht : Springer, 2013. - P. 147-154
(Advances in cognitive neurodynamics)
Bibtex MIS-Preprint: 15/2011 DOI: 10.1007/978-94-007-4792-0_20
Green Open Access

Montúfar, Guido ; Rauh, Johannes and Ay, Nihat: Maximal information divergence from statistical models defined by neural networks
In: Geometric science of information : first international conference, GSI 2013, Paris, France, August 28-30, 2013. Proceedings / Frank Nielsen... (eds.)
Berlin [u. a.] : Springer, 2013. - P. 759-766
(Lecture notes in computer science ; 8085)
Bibtex MIS-Preprint: 31/2013 DOI: 10.1007/978-3-642-40020-9_85 ARXIV: http://arxiv.org/abs/1303.0268
Green Open Access

Montúfar, Guido and Morton, Jason: When does a mixture of products contain a product of mixtures?
In: NIPS 2012 - deep learning and unsupervised feature learning workshop : [be held in conjunction with neural information processing systems on December 8, 2012 (TBD) at Lake Tahoe, USA]
La Jolla, CA : Neural Information Processing Systems, 2012
Bibtex MIS-Preprint: 98/2014 ARXIV: http://arxiv.org/abs/1206.0387
Green Open Access

Montúfar, Guido and Rauh, Johannes: Scaling of model approximation errors and expected entropy distances
In: Proceedings of the 9th workshop on uncertainty processing WUPES '12 : Marianske Lazne, Czech Republik ; 12-15th September 2012
Praha : Academy of Sciences of the Czech Republik / Institute of Information Theory and Automation, 2012. - P. 137-148
Bibtex ARXIV: http://de.arxiv.org/abs/1207.3399 LINK: http://wupes.fm.vse.cz/2012/data/Wupes12_proceedings.pdf
Green Open Access

Montúfar, Guido ; Rauh, Johannes and Ay, Nihat: Expressive power and approximation errors of restricted Boltzmann machines
In: Advances in neural information processing systems 24 : 25th annual conference on neural information processing systems 2011, Granada, Spain December 12th - 15th ; NIPS 2011 / J. Shawe-Taylor (ed.)
La Jolla, CA : Neural Information Processing Systems, 2011. - P. 415-423
Bibtex MIS-Preprint: 27/2011 ARXIV: https://arxiv.org/abs/1406.3140 LINK: http://papers.nips.cc/paper/4380-expressive-power-and-approximation-errors-of-restricted-boltzmann-machines.pdf
Green Open Access

Montúfar, Guido ; Richter, Marten ; Brandes, Tobias and Knorr, Andreas: Theory of transport and photon-statistics in a biased nanostructure
In: 2008 International Nano-Optoelectronics workshop (iNOW 2008)
Piscataway, NJ : IEEE, 2008. - P. 243-244
Bibtex DOI: 10.1109/INOW.2008.4634528

Preprints To top of page

Li, Wuchen ; Lin, Alex Tong and Montúfar, Guido: Affine natural proximal learning
Green Open Access

Merkh, Thomas and Montúfar, Guido: Factorized mutual information maximization
Green Open Access

Banerjee, Pradeep Kumar and Montúfar, Guido: The variational deficiency bottleneck
Green Open Access

Li, Wuchen and Montúfar, Guido: Ricci curvature for parametric statistics via optimal transport
Green Open Access

Lin, Alex Tong ; Li, Wuchen ; Osher, Stanley and Montúfar, Guido: Wasserstein proximal of GANs
Green Open Access

Montúfar, Guido ; Ghazi-Zahedi, Keyan and Ay, Nihat: Information theoretically aided reinforcement learning for embodied agents
Green Open Access

Montúfar, Guido: Universal approximation of Markov kernels by shallow stochastic feedforward networks
Green Open Access

Montúfar, Guido ; Ghazi-Zahedi, Keyan and Ay, Nihat: Geometry and determinism of optimal stationary control in partially observable Markov decision processes
Green Open Access

Krüger, Tyll ; Montúfar, Guido ; Seiler, Ruedi and Siegmund-Schultze, Rainer: Sequential recurrence-based multidimensional universal source coding of Lempel-Ziv type
Green Open Access

Workshop Articles To top of page

Ay, Nihat ; Rauh, Johannes and Montúfar, Guido: A continuity result for optimal memoryless planning in POMDPs
(RLDM 2019 : 4th multidisciplinary conference on reinforcement learning and decision making ; July 7-10, 2019 ; Montréal, Canada)
Bibtex LINK: http://rldm.org/papers/extendedabstracts.pdf#page=362
Green Open Access

Montúfar, Guido ; Rauh, Johannes and Ay, Nihat: Task-agnostic constraining in average reward POMDPs
(Task-agnostic reinforcement learning : workshop at ICLR, 06 May 2019, New Orleans)
Bibtex LINK: https://tarl2019.github.io/assets/papers/montufar2019taskagnostic.pdf
Green Open Access

Montúfar, Guido ; Rauh, Johannes and Ay, Nihat: Uncertainty and stochasticity of optimal policies
(Proceedings of the 11th workshop on uncertainty processing WUPES '18, June 6-9, 2018)
Bibtex LINK: http://wupes.utia.cas.cz/proceedings/proceedings.pdf
Green Open Access

Montúfar, Guido: Notes on the number of linear regions of deep neural networks
(2017 international conference on sampling theory and applications (SampTA))
Bibtex LINK: https://www.researchgate.net/publication/322539221
Green Open Access

Montúfar, Guido: Restricted Boltzmann machines [In: Algebraic statistics : 16 April - 22 April 2017 ; report no. 20/2017]
(Oberwolfach reports ; 14 )
Bibtex DOI: 10.4171/OWR/2017/20

Montúfar, Guido ; Ghazi-Zahedi, Keyan and Ay, Nihat: Stochasticity of optimal policies for POMDPs
(RLDM 2017 : 3rd multidisciplinary conference on reinforcement learning and decision making ; June 11-14, 2017 ; Ann Arbor, Michigan, USA)
Bibtex LINK: http://rldm.org/wp-content/uploads/2017/06/RLDM17AbstractsBooklet.pdf

Montúfar, Guido: A comparison of neural network architectures
(Deep learning Workshop, ICML '15, Vauban Hall at Lille Grande Palais, France, July 10 and 11, 2015)
Bibtex

Montúfar, Guido and Morton, Jason: Kernels and submodels of deep belief networks
(NIPS 2012 - deep learning and unsupervised feature learning workshop : [be held in conjunction with neural information processing systems on December 8, 2012 (TBD) at Lake Tahoe, USA])
Bibtex ARXIV: https://arxiv.org/abs/1211.0932
Green Open Access

Montúfar, Guido: Mixture models and representational power of RBM's, DBN's, and DBM's
(Deep learning and unsupervised feature learning workshop : NIPS 2010 ; December 19, 2010, Hilton, Vancouver, Canada)
Bibtex LINK: http://deeplearningworkshopnips2010.files.wordpress.com/2010/11/nipsworkshoppaperr1.pdf

Academic Theses To top of page

Montúfar, Guido: On the expressive power of discrete mixture models, restricted Boltzmann machines, and deep belief networks - a unified mathematical treatment

Montúfar, Guido: Theory of transport and photon-statistics in a biased nanostructure
Diplomarbeit, Universität Berlin, 2008
Bibtex

Montúfar, Guido: Q-Sanov theorem for d 2
Diplomarbeit, Universität Berlin, 2007
Bibtex

Supplementary Resources To top of page

Banerjee, Pradeep Kumar ; Bansal, Sumukh ; Demir, Ilke ; Quang, Minh Ha ; Huang, Lin ; Hühnerbein, Ruben ; James, Scott C. ; Kachan, Oleg ; Ly, Louis ; Lysaker, Marius ; Maharjan, Samee ; Mallasto, Anton ; Montúfar, Guido ; Sandfort, Kai ; Schonsheck, Stefan C. ; Suárez-Serrato, Pablo ; Tóthová, Katarína ; Wang, Yu Guang ; Xian, Jia Le and Xiang, Rui: Geometry and learning from data in 3D and beyond : IPAM long program, Spring 2019 [Report]
Green Open Access

Montúfar, Guido: Contoursurf [Computer code]
Green Open Access

Montúfar, Guido: Computing the unique information - 1st workshop on semantic information - CVPR June 2019 - Long Beach [Slides]
Green Open Access

Montúfar, Guido: Wasserstein information geometry for learning from data : tutorial at geometry and learning from data, IPAM, March 2019 [Slides]
Green Open Access

Montúfar, Guido: Illustration of maxout layer upper bound [Suppl. to: On the number of linear regions of deep neural networks]
Green Open Access
21.08.2019, 06:43