Guido Montúfar
Max Planck Institute for Mathematics in the Sciences
Inselstrasse 22
04103 Leipzig
Germany

Phone: +49 (0) 341 9959 521
Office: A3 05
Email: montufarmis.mpg.de
Short CV
Postdoc, Max Planck Institute for Mathematics in the Sciences.
(since 06/2013)
Research Associate, Department of Mathematics, Pennsylvania State University.
(02/2012 - 05/2013)
PhD in Mathematics, MPI MIS, Leipzig University.
(10/2012)
PhD scholarship, IMPRS MPI MIS.
(01/2009 - 01/2012)
Research Assistant, Institute for Theoretical Physics, TU-Berlin.
(03/2008 - 12/2008)
Diplom Physiker, TU-Berlin.
(12/2008)
Teaching Assistant, Institute for Mathematics, TU-Berlin.
(03/2006 - 02/2008)
Diplom Mathematiker, TU-Berlin.
(08/2007)
Research Interests
Deep Learning
Design of Learning Systems
Embodied Intelligence
Data Compression
Graphical Models
Neural Networks
Information Geometry
Algebraic Statistics


Publications


Peer-Reviewed Articles

Hierarchical Models as Marginals of Hierarchical Models.
G. Montufar and J. Rauh. International Journal of Approximate Reasoning, in press, 2016. [BibTeX]. Workshop version WUPES 2015, pp 131-145. Preprint [MPI MIS 27/2016], [arXiv 1508.03606] . Supplement [starcover.m].
Mode Poset Probability Polytopes.
G. Montufar and J. Rauh. Journal of Algebraic Statistics, 7(1):1-13, 2016. [BibTeX]. Workshop version WUPES 2015, pp 147-154. Preprint [MPI MIS 22/2015], [arXiv 1503.00572] 
Evaluating Morphological Computation in Muscle and DC-motor Driven Models of Hopping Movements.
K. Ghazi-Zahedi, D. Haeufle, G. Montufar, S. Schmitt, and N. Ay. Frontiers in Robotics and AI 3(42):frobt.2016.00042, 2016. [BibTeX]. Preprint [arXiv 1512.00250] 
A Theory of Cheap Control in Embodied Systems.
G. Montufar, K. Zahedi, and N. Ay. PLoS Comput Biol 11(9):e1004427, 2015. [BibTeX]. Preprint [MPI MIS 70/2014], [arXiv 1407.6836] 
Geometry and Expressive Power of Conditional Restricted Boltzmann Machines.
G. Montufar, N. Ay, and K. Zahedi. JMLR 16(Dec):2405-2436, 2015. [BibTeX]. Preprint [MPI MIS 16/2014], [arXiv 1402.3346] 
Discrete Restricted Boltzmann Machines.
G. Montufar and J. Morton. JMLR 16(Apr):653-672, 2015. [BibTeX]. Conference version ICLR 2013. Preprint [MPI MIS 106/2014], [arXiv 1301.3529] 
When Does a Mixture of Products Contain a Product of Mixtures?
G. Montufar and J. Morton. SIAM Journal on Discrete Mathematics (SIDMA), 29(1):321-347, 2015. [BibTeX]. Preprint [MPI MIS 98/2014], [arXiv 1206.0387] 
Deep Narrow Boltzmann Machines are Universal Approximators.
G. Montufar. In Third International Conference on Learning Representations (ICLR 2015). [BibTeX]. Preprint [MPI MIS 113/2014], [arXiv 1411.3784] 
On the Number of Linear Regions of Deep Neural Networks.
G. Montufar, R. Pascanu, K. Cho, and Y. Bengio. NIPS 27, pp. 2924-2932, 2014. [BibTeX]. Preprint [MPI MIS 73/2014], [arXiv 1402.1869] 
On the Number of Response Regions of Deep Feedforward Networks with Piecewise Linear Activations.
R. Pascanu, G. Montufar, and Y. Bengio. In Second International Conference on Learning Representations (ICLR 2014). [BibTeX]. Preprint [MPI MIS 72/2014], [arXiv 1312.6098] 
On the Fisher Information Metric of Conditional Probability Polytopes.
G. Montufar, J. Rauh, and N. Ay. Entropy 16(6):3207-3233, 2014. [BibTeX]. Preprint [MPI MIS 87/2014], [arXiv 1404.0198] 
Scaling of Model Approximation Errors and Expected Entropy Distances.
G. Montufar and J. Rauh. Kybernetika 50(2):234-245, 2014. [BibTeX]. Workshop version WUPES 2012, pp. 137-148. Preprint [arXiv 1207.3399] 
Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units.
G. Montufar. Neural Computation 26(7):1386-1407, 2014. [BibTeX]. Preprint [MPI MIS 74/2014], [arXiv 1303.7461] 
Universally Typical Sets for Ergodic Sources of Multidimensional Data.
T. Krüger, G. Montufar, R. Seiler, and R. Siegmund-Schultze. Kybernetika 49(6):868-882, 2013. [BibTeX]. Preprint [MPI MIS 20/2011], [arXiv 1105.0393] 
Mixture Decompositions of Exponential Families Using a Decomposition of their Sample Spaces.
G. Montufar. Kybernetika 49(1):23-39, 2013. [BibTeX]. Preprint [MPI MIS 39/2010], [arXiv 1008.0204] 
Maximal Information Divergence from Statistical Models defined by Neural Networks.
G. Montufar, J. Rauh, and N. Ay. In Geometric Science of Information LNCS Vol. 8085, pp 759-766, 2013. [BibTeX]. Preprint [MPI MIS 31/2013], [arXiv 1303.0268] 
Selection Criteria for Neuromanifolds of Stochastic Dynamics.
N. Ay, G. Montufar, J. Rauh. In Advances in Cognitive Neurodynamics (III), pp 147-154, 2013. [BibTeX]. Preprint [MPI MIS 15/2011]
Expressive Power and Approximation Errors of Restricted Boltzmann Machines.
G. Montufar, J. Rauh, and N. Ay. NIPS 24, pp. 415-423, 2011. [BibTeX]. Preprint [MPI MIS 27/2011], [arXiv 1406.3140] 
Refinements of Universal Approximation Results for Restricted Boltzmann Machines and Deep Belief Networks.
G. Montufar and N. Ay. Neural Computation 23(5):1306-1319, 2011. [BibTeX]. Preprint [MPI MIS 23/2010], [arXiv 1005.1593] 

Workshop Articles

A Comparison of Neural Network Architectures.
G. Montufar. Deep Learning Workshop, ICML 2015. [pdf] 
Kernels and Submodels of Deep Belief Networks.
G. Montufar and J. Morton. NIPS 2012 - Deep Learning and Unsupervised Feature Learning Workshop. Preprint [arXiv 1211.0932] 
Mixture Models and Representational Power of RBMs, DBNs and DBMs.
G. Montufar. NIPS 2010 - Deep Learning and Unsupervised Feature Learning Workshop. [pdf] 

Preprints

Information Theoretically Aided Reinforcement Learning for Embodied Agents.
G. Montufar, K. Ghazi-Zahedi, and N. Ay. Preprint [arXiv 1605.09735] 
Dimension of Marginals of Kronecker Product Models; Geometry of hidden-visible products of exponential families.
G. Montufar and J. Morton. Preprint [MPI MIS 75/2015], [arXiv 1511.03570] . Supplement [JacobianKronecker.m].
Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes.
G. Montufar, K. Ghazi-Zahedi, and N. Ay. Preprint [MPI MIS 22/2016], [arXiv 1503.07206] 
Sequential Recurrence-Based Multidimensional Universal Source Coding of Lempel-Ziv Type.
T. Krueger, G. Montufar, R. Seiler, and R. Siegmund-Schultze. Preprint [MPI MIS 86/2014], [arXiv 1408.4433] 
Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks.
G. Montufar. Preprint [MPI MIS 23/2015], [arXiv 1503.07211] 

Theses

On the Expressive Power of Discrete Mixture Models, Restricted Boltzmann Machines, and Deep Belief Networks—A Unified Mathematical Treatment.
PhD Thesis, Leipzig University, October 2012. Supervisor: N. Ay. [pdf] (14.4 MB, 155 pages, 30 figures)
Theory of Transport and Photon-Statistics in a Biased Nanostructure.
German Diplom in Physics, Institute for Theoretical Physics, TU-Berlin, December 2008. Supervisor: A. Knorr and T. Brandes.
Q-Sanov Theorem for d 2.
German Diplom in Mathematics, Institute for Mathematics, TU-Berlin, August 2007. Supervisor: R. Seiler and J.-D. Deuschel.


Lectures


Introduction to the Theory of Neural Networks, MS/PhD Lecture, Summer Term 2016, Leipzig University and MPI MIS.
Geometric Aspects of Graphical Models and Neural Networks, with N. Ay, [Abstract], MS/PhD Lecture, Winter Term 2014/2015, Leipzig University and MPI MIS.


Talks


Conferences and Workshops

Geometric and Combinatorial Perspectives on Deep Neural Networks, [Slides], Theory of Deep Learning Workshop, ICML 2016, New York, NY, USA, June 2016.
Geometry of Boltzmann Machines, [Slides], [Abstract], IGAIA IV, Liblice, Czech Republic, June 2016.
Mode Poset Probability Polytopes, WUPES'15, Moninec, Czech Republic, September 18, 2015.
Hierarchical models as marginals of hierarchical models, WUPES'15, Moninec, Czech Republic, September 17, 2015.
Confining bipartite graphical models by simple classes of inequalities, Special Topics Session Algebraic and Geometric Approaches to Graphical Models, 60th World Statistics Congress - ISI 2015, Rio de Janeiro, Brazil, July 31, 2015.
Information Divergence from Statistical Models Defined by Neural Networks, Workshop: Information Geometry for Machine Learning, RIKEN BSI, Japan, December 2014.
Geometry of Hidden-Visible Products of Statistical Models, Joint Workshop on Limit Theorems and Algebraic Statistics, UTIA, Prague, August 25-29, 2014.
Maximal Information Divergence from Statistical Models defined by Neural Networks, GSI 2013, Mines ParisTech, Paris, France, August 29, 2013.
Discrete Restricted Boltzmann Machines, ICLR2013, Scottsdale, AZ, USA, May 2, 2013.
When Does a Mixture of Products Contain a Product of Mixtures?, Tensor network states and algebraic geometry, ISI Foundation, Torino, Italy, November 06-08, 2012.
Scaling of Model Approximation Errors and Expected Entropy Distances, WUPES'12, Mariánské Lázně, Czech Republic, September 13, 2012.
Simplex packings of marginal polytopes and mixtures of exponential families, SIAM Conference on Discrete Mathematics (DM 2012), Dalhousie University, Halifax, Nova Scotia, Canada, June 18-21, 2012.
On Secants of Exponential Families, Algebraic Statistics in the Alleghenies, Penn State, PA, USA, June 08-15, 2012.
Geometry of Restricted Boltzmann Machines Towards Geometry of Deep Belief Networks, RIKEN Workshop on Information Geometry, RIKEN BSI, Japan, August 31, 2011.
Selection Criteria for Neuromanifolds of Stochastic Dynamics, The 3rd International Conference on Cognitive Neurodynamics, Niseko Village, Hokkaido, Japan, June 12, 2011.
Information Geometry of Mean-Field Methods, Fall School on Statistical Mechanics and 5th annual PhD Student Conference in Probability, MPI MIS, Leipzig, Germany, September 07-12, 2009.
Quantum-Sanov-Theorem for correlated States in multidimensional Grids, Dies Mathematicus, TU-Berlin, Germany, February 2008.
Quanten-Sanov-Theorem im mehrdimensionallen Fall, Workshop on Complexity and Information Theory, MPI MIS, Leipzig, Germany, October 2007.

Seminars and Meetings

Dimension of Marginals of Kronecker Product Models, Seminar on Non-Linear Algebra, TU-Berlin, Germany, November 2016.
Artificial Intelligence Overview, LikBez Seminar, MPI MIS, January 2016.
Geometric Approaches to the Design of Embodied Learning Systems, Special Symposium on Intelligent Systems, MPI for Intelligent Systems, Tuebingen, Germany, March 2016.
A Theory of Cheap Control in Embodied Systems, Montreal Institute for Learning Algorithms (MILA), University of Montreal, Canada, December 2015.
Dimension of restricted Boltzmann machines, Department of Mathematics & Statistics, York University, Toronto, Canada, December 2015.
Sequential Recurrence-Based Multidimensional Universal Source Coding, Dynamical Systems Seminar, MPI MIS, November 2015.
Cheap Control of Embodied Systems, Aalto Science Institute, Espoo, Finland, November 2015.
On the Number of Linear Regions of Deep Neural Networks, Montreal Institute for Learning Algorithms (MILA), Université de Montréal, Montreal, Canada, December 15, 2014.
Geometry of Deep Neural Networks and Cheap Design for Autonomous Learning, Google DeepMind, London, UK, October 2014.
How size and architecture determine the learning capacity of neural networks, SFI Seminar, Santa Fe, NM, USA, October 23, 2013.
Naive Bayes models, Seminario de Postgrado en Ingenieria de Sistemas, Universidad del Valle, Santiago de Cali, Colombia, May 30, 2013.
On the Expressive Power of Discrete Mixture Models, Restricted Boltzmann Machines, and Deep Belief Networks—A Unified Mathematical Treatment, PhD thesis defense, Leipzig University, October 17, 2012.
Scaling of model approximation errors and expected entropy distances, Stochastic Modelling and Computational Statistics Seminar (Murali Haran), Penn State, PA, USA, October 11, 2012.
Universally typical sets for ergodic sources of multidimensional data, Seminar on probability and its applications (Manfred Denker), Penn State, PA, USA, October 05, 2012.
Multivalued Restricted Boltzmann Machines, [Abstract], MPI MIS, Leipzig, Germany, September 19, 2012.
Approximation Errors of Deep Belief Networks, Applied Algebraic Statistics Seminar, Penn State, PA, USA, February 08, 2012.
Submodels of Deep Belief Networks, [Abstract], Berkeley Algebraic Statistics Seminar, UC Berkeley, CA, USA, December 07, 2011.
Geometry and Approximation Errors of Restricted Boltzmann Machines, The 5th Statistical Machine Learning Seminar, Institute of Statistical Mathematics, Tachikawa, Tokyo, Japan, September 02, 2011.
On Exponential Families and the Expressive Power of Related Generative Models, [Abstract], Laboratoire d'Informatique des Systèmes Adaptatifs (LISA), Université de Montréal, Canada, March 14, 2011.
Mixtures from Exponential Families, Neuronale Netze und Kognitive Systeme Seminar, MPI MIS, Leipzig, Germany, March 02, 2011.
Universal approximation results for Restricted Boltzmann Machines and Deep Belief Networks, Neuronale Netze und Kognitive Systeme Seminar, MPI MIS, Leipzig, Germany, February 16, 2011.
Necessary conditions for RBM universal approximators, Meeting of the Department of Decision-Making Theory - Institute of Information Theory and Automation UTIA, Marianska, Czech Republic, January 18, 2011.

Poster

A comparison of neural network architectures, Deep Learning Workshop, ICML 2015.
Mode Poset Probability Polytopes, [pdf], Algebraic Statistics 2015, Department of Mathematics University of Genoa, Italy, June 8-11, 2015.
Deep Narrow Boltzmann Machines are Universal Approximators, [pdf], ICLR 2015.
On the Number of Linear Regions of Deep Neural Networks, [pdf], NIPS 2014.
A Framework for Cheap Universal Approximation in Embodied Systems, Autonomous Learning: 3. Symposium DFG Priority Programme 1527, Berlin, September 8-9, 2014.
Geometry of hidden-visible products of statistical models, [pdf], Algebraic Statistics at IIT, Chicago, IL, 2014.
When Does a Mixture of Products Contain a Product of Mixtures, [Abstract], NIPS 2012 - Deep Learning and Unsupervised Feature Learning Workshop.
Mixture Models and Representational Power of RBMs, DBNs and DBMs, NIPS 2010 - Deep Learning and Unsupervised Feature Learning Workshop, Whistler, Canada.
Faces of the probability simplex contained in the closure of an exponential family and minimal mixture representations, Information Geometry and its Applications III, Leipzig, Germany, 2010.


Activities


Oberwolfach Workshop Algebraic Statistics, Mathematisches Forschungsinstitut Oberwolfach, Germany, April 2017.
Santa Fe Institute, Visit for Research Collaboration (Nihat Ay), Santa Fe, NM, USA, October 2016.
ICML 2016, New York, USA.
IGAIA IV 2016, Liblice, Czech Republic.
NIPS 2015, Montréal, Canada.
WUPES'15, Monínec, Czech Republic.
ICML 2015, Lille, France.
NIPS 2014, Montréal, Canada.
Workshop: Information Geometry for Machine Learning, RIKEN BSI, Japan, December 2014.
Santa Fe Institute, Visit for Research Collaboration (Nihat Ay), Santa Fe, NM, USA, October 15-November 15, 2014.
Information Geometry in Learning and Optimization, University of Copenhagen, September 22-26, 2014.
Autonomous Learning: 3. Symposium DFG Priority Programme 1527, Magnus Haus Berlin, Germany, September 08-09, 2014.
Autonomous Learning: Summer School, MPI MIS, September 01-04, 2014.
Joint Workshop on Limit Theorems and Algebraic Statistics, UTIA, Prague, Czech Republic, August 25-29, 2014.
Algebraic Statistics at IIT, Chicago, IL, USA, May 2014.
Santa Fe Institute, Visit for Research Collaboration (Nihat Ay), October 1-27, 2013.
SFI Working Group ``Information Theory of Sensorimotor Loops'', Santa Fe Institute, Santa Fe, NM, USA, October 8-11, 2013.
Pennsylvania State University, Visit for Research Collaboration (Jason Morton), PA, USA, September 2013.
GSI 2013, Paris, August 28-30, 2013.
ICLR2013, Scottsdale, AZ, USA, May 2-4, 2013.
NIPS 2012Deep Learning Workshop, Lake Tahoe, NV, USA, December 7-8, 2012.
Algebraic Statistics in Europe, IST Austria, September 28-30, 2012.
WUPES'12, Marianske Lazne, Czech Republic, September 12-15, 2012.
SIAM Conference on Discrete Mathematics 2012, Dalhousie University, Halifax, Nova Scotia, Canada, June 18-21, 2012.
Algebraic Statistics in the Alleghenies, Penn State, PA, USA, June 08-15, 2012.
Singular Learning Theory, AIM Workshop, American Institute of Mathematics, Palo Alto, CA, USA, December 12-16, 2011.
RIKEN-BSI, Laboratory for Mathematical Neuroscience (Prof. S. Amari), Internship, Hirosawa, Wako, Saitama, Japan, August-October 2011.
SFI Complex Systems Summer School (CSSS11), Saint John's College, Santa Fe, NM, USA, June 8-July 1, 2011.
Information Geometry and its Applications (IGAIA III), Leipzig University, Germany, August 2010.


Gallery


When Does a Mixture of Products Contain a Product of Mixtures?
SIAM Journal on Discrete Mathematics, 29(1):321-347, 2015.


Imprint
G. Montúfar,  Email: montufarmis.mpg.de
Last updated: 08/2016