Guido Montúfar
Max Planck Institute for Mathematics in the Sciences
Inselstrasse 22
04103 Leipzig
Germany

Phone: +49 (0) 341 9959 521
Office: A 05
Email: montufarmis.mpg.de
CV:
Postdoc, Max Planck Institute for Mathematics in the Sciences.
(since 06/2013)
Research Associate (Postdoc), Department of Mathematics, Pennsylvania State University.
(02/2012 - 05/2013)
PhD scholarship, MPI MIS Leipzig.
(01/2009 - 01/2012)
Research Assistant, Institute for Theoretical Physics, TU-Berlin.
(03/2008 - 12/2008)
Teaching Assistant, Institute for Mathematics, TU-Berlin.
(03/2006 - 02/2008)
Diplom Physiker, TU-Berlin.
(12/2008)
Diplom Mathematiker, TU-Berlin.
(08/2007)
Research interests:
Deep Learning
Design of Learning Systems
Embodied Intelligence
Data Compression
Graphical Models
Neural Networks
Information Geometry



Peer-Reviewed Articles
Geometry and Expressive Power of Conditional Restricted Boltzmann Machines.
G. Montufar, N. Ay, and K. Zahedi. JMLR 16(Dec):2405-2436, 2015. [BibTeX], [MPI MIS Preprint 16/2014], [Preprint arXiv 1402.3346] 
Discrete Restricted Boltzmann Machines.
G. Montufar and J. Morton. JMLR 16(Apr):653-672, 2015. [BibTeX]. Also in International Conference on Learning Representations, Scottsdale, AZ, USA. ICLR 2013. [MPI MIS Preprint 106/2014], [Preprint arXiv 1301.3529] 
When Does a Mixture of Products Contain a Product of Mixtures?
G. Montufar and J. Morton. SIAM Journal on Discrete Mathematics (SIDMA), 29(1):321-347, 2015. [BibTeX]. [MPI MIS Preprint 98/2014], [Preprint arXiv 1206.0387] 
Deep Narrow Boltzmann Machines are Universal Approximators.
G. Montufar. The Third International Conference on Learning Representations, San Diego, CA, USA. ICLR 2015. [BibTeX]. [MPI MIS Preprint 113/2014], [Preprint arXiv 1411.3784] 
On the Number of Linear Regions of Deep Neural Networks.
G. Montufar, R. Pascanu, K. Cho, and Y. Bengio. Advances in Neural Information Processing Systems 27, Montreal, Canada. NIPS 27, pp. 2924-2932, 2014. [BibTeX]. [MPI MIS Preprint 73/2014], [Preprint arXiv 1402.1869] 
On the Number of Response Regions of Deep Feedforward Networks with Piecewise Linear Activations.
R. Pascanu, G. Montufar, and Y. Bengio. The Second International Conference on Learning Representations, Banff, Canada. ICLR 2014. [BibTeX]. [MPI MIS Preprint 72/2014], [Preprint arXiv 1312.6098] 
On the Fisher Information Metric of Conditional Probability Polytopes.
G. Montufar, J. Rauh, and N. Ay. Entropy 16(6):3207-3233, 2014. [BibTeX]. [MPI MIS Preprint 87/2014], [Preprint arXiv 1404.0198] 
Scaling of Model Approximation Errors and Expected Entropy Distances.
G. Montufar and J. Rauh. Kybernetika 50(2):234-245, 2014. [BibTeX]. Also in Proceedings of the 9th Workshop on Uncertainty Processing, Marianske Lazne, Czech Republic. WUPES 2012, pp. 137-148. [Preprint arXiv 1207.3399] 
Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units.
G. Montufar. Neural Computation 26(7):1386-1407, 2014. [BibTeX]. [MPI MIS Preprint 74/2014], [Preprint arXiv 1303.7461] 
Universally Typical Sets for Ergodic Sources of Multidimensional Data.
T. Krüger, G. Montufar, R. Seiler, and R. Siegmund-Schultze. Kybernetika 49(6):868-882, 2013. [BibTeX]. [MPI MIS Preprint 20/2011], [Preprint arXiv 1105.0393] 
Mixture Decompositions of Exponential Families Using a Decomposition of their Sample Spaces.
G. Montufar. Kybernetika 49(1):23-39, 2013. [BibTeX]. [MPI MIS Preprint 39/2010], [Preprint arXiv 1008.0204] 
Maximal Information Divergence from Statistical Models defined by Neural Networks.
G. Montufar, J. Rauh, and N. Ay. First International Conference, Geometric Science of Information, Paris, August 28-30, 2013. GSI LNCS Vol. 8085, 2013, pp 759-766 . [BibTeX]. [MPI MIS Preprint 31/2013], [Preprint arXiv 1303.0268] 
Selection Criteria for Neuromanifolds of Stochastic Dynamics.
N. Ay, G. Montufar, J. Rauh. Advances in Cognitive Neurodynamics (III), 2013, pp 147-154. Proceedings of the Third International Conference on Cognitive Neurodynamics, Niseko Village, Hokkaido, Japan, 2011. [BibTeX]. [MPI MIS Preprint 15/2011]
Expressive Power and Approximation Errors of Restricted Boltzmann Machines.
G. Montufar, J. Rauh, and N. Ay. Advances in Neural Information Processing Systems 24, Granada, Spain. NIPS 24, pp. 415-423, 2011. [BibTeX]. [MPI MIS Preprint 27/2011] [Preprint arXiv 1406.3140] 
Refinements of Universal Approximation Results for Restricted Boltzmann Machines and Deep Belief Networks.
G. Montufar and N. Ay. Neural Computation 23(5):1306-1319, 2011. [BibTeX]. [MPI MIS Preprint 23/2010], [Preprint arXiv 1005.1593] 
Workshop Articles
Hierarchical Models as Marginals of Hierarchical Models.
G. Montufar and J. Rauh. Proceedings of the 10th Workshop on Uncertainty Processing, Monínec, Czech Republic. WUPES 2015, pp 131-145. [MPI MIS Preprint 27/2016], [Preprint arXiv 1508.03606] , [starcover.m].
Mode Poset Probability Polytopes.
G. Montufar and J. Rauh. Proceedings of the 10th Workshop on Uncertainty Processing, Monínec, Czech Republic. WUPES 2015, pp 147-154. [MPI MIS Preprint 22/2015], [Preprint arXiv 1503.00572] 
A Comparison of Neural Network Architectures.
G. Montufar. Deep Learning Workshop, ICML 2015. [pdf] 
Kernels and Submodels of Deep Belief Networks.
G. Montufar and J. Morton. NIPS 2012 - Deep Learning and Unsupervised Feature Learning Workshop. [Preprint arXiv 1211.0932] 
Mixture Models and Representational Power of RBMs, DBNs and DBMs. G. Montufar. NIPS 2010 - Deep Learning and Unsupervised Feature Learning Workshop. [pdf] 
Preprints
Evaluating Morphological Computation in Muscle and DC-motor Driven Models of Human Hopping.
K. Ghazi-Zahedi, D. Haeufle, G. Montufar, S. Schmitt, and N. Ay. [Preprint arXiv 1512.00250] 
Dimension of Marginals of Kronecker Product Models; Geometry of hidden-visible products of exponential families.
G. Montufar and J. Morton. [MPI MIS Preprint 75/2015], [Preprint arXiv 1511.03570] , [JacobianKronecker.m].
Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes.
G. Montufar, K. Ghazi-Zahedi, and N. Ay. [MPI MIS Preprint 22/2016], [Preprint arXiv 1503.07206] 
Sequential Recurrence-Based Multidimensional Universal Source Coding of Lempel-Ziv Type.
T. Krueger, G. Montufar, R. Seiler, and R. Siegmund-Schultze. [MPI MIS Preprint 86/2014], [Preprint arXiv 1408.4433] 
Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks.
G. Montufar. [MPI MIS Preprint 23/2015], [Preprint arXiv 1503.07211] 
Theses
On the Expressive Power of Discrete Mixture Models, Restricted Boltzmann Machines, and Deep Belief Networks—A Unified Mathematical Treatment.
PhD Thesis, Leipzig University, 2012. Supervisor: N. Ay. [pdf] (14.4 MB, 155 pages, 30 figures)
Theory of Transport and Photon-Statistics in a Biased Nanostructure.
German Diplom in Physics, Institute for Theoretical Physics, TU-Berlin, December 2008. Supervisor: A. Knorr and T. Brandes.
Q-Sanov Theorem for d 2.
German Diplom in Mathematics, Institute for Mathematics, TU-Berlin, August 2007. Supervisor: R. Seiler and J.-D. Deuschel.
Lectures
Introduction to the Theory of Neural Netwokrs, Summer Term 2016, University of Leipzig and MPI MIS.
Geometric Aspects of Graphical Models and Neural Networks, with N. Ay, Winter Term 2014/2015, University of Leipzig and MPI MIS.
Talks
AI overview, LikBez MIS MPG, January 2016.
A Theory of Cheap Control in Embodied Systems, MILA UdeM, December 2015.
Dimension of restricted Boltzmann machines, York University, December 2015.
Sequential Recurrence-Based Multidimensional Universal Source Coding, Dynamical Systems Seminar MIS MPG, November 2015.
Cheap Control of Embodied Systems, Aalto Science Institute, November 2015.
Mode Poset Probability Polytopes, WUPES'15, September 18, 2015.
Hierarchical models as marginals of hierarchical models, WUPES'15, September 17, 2015.
Confining bipartite graphical models by simple classes of inequalities, 60th World Statistics Congress - ISI 2015, Special Topics Session Algebraic and geometric approaches to graphical models, July 31, 2015.
On the Number of Linear Regions of Deep Neural Networks, Université de Montréal, December 15, 2014.
Information Divergence from Statistical Models Defined by Neural Networks, Workshop: Information Geometry for Machine Learning , December, 2014, RIKEN BSI, Japan.
Geometry of Hidden-Visible Products of Statistical Models, Joint Workshop on Limit Theorems and Algebraic Statistics , August 25-29, 2014, UTIA, Prague.
How size and architecture determine the learning capacity of neural networks, October 23, 2013, Santa Fe Institute Seminar, NM, USA.
Maximal Information Divergence from Statistical Models defined by Neural Networks, August 29, 2013, Geometric Science of Information (GSI 2013), Mines ParisTech.
Naļve Bayes models, May 30, 2013, Seminario de Postgrado en Ingenierķa de Sistemas, Universidad del Valle, Santiago de Cali, Colombia.
Discrete Restricted Boltzmann Machines, International Conference on Learning Representations (ICLR2013), May 2, 2013, Scottsdale, AZ, USA.
When Does a Mixture of Products Contain a Product of Mixtures?, Tensor network states and algebraic geometry , November 06-08, 2012, ISI Torino, Italy.
On the Expressive Power of Discrete Mixture Models, Restricted Boltzmann Machines, and Deep Belief Networks—A Unified Mathematical Treatment (PhD thesis), October 17, 2012, Leipzig University.
Scaling of model approximation errors and expected entropy distances, October 11, 2012, Stochastic Modelling and Computational Statistics Seminar (Murali Haran) Penn State, PA, USA.
Universally typical sets for ergodic sources of multidimensional data, October 05, 2012, Seminar on probability and its applications (Manfred Denker) Penn State, PA, USA.
Multivalued Restricted Boltzmann Machines, September 19, 2012, MPI MIS, Leipzig, Germany.
Scaling of Model Approximation Errors and Expected Entropy Distances, September 13, 2012, WUPES'12 , Mariánské Lázně, Czech Republic.
Simplex packings of marginal polytopes and mixtures of exponential families, SIAM Conference on Discrete Mathematics (DM 2012) , June 18-21, 2012, Dalhousie University, Halifax, Nova Scotia, Canada.
Approximation Errors of Deep Belief Networks, Applied Algebraic Statistics Seminar, February 08, 2012, Penn State, PA, USA.
Submodels of Deep Belief Networks, Berkeley Algebraic Statistics Seminar, December 07, 2011, UC Berkeley, CA, USA.
Geometry of Restricted Boltzmann Machines Towards Geometry of Deep Belief Networks, Workshop on Information Geometry, August 31, 2011, RIKEN BSI, Japan.
Geometry and Approximation Errors of Restricted Boltzmann Machines, The 5th Statistical Machine Learning Seminar, September 02, 2011, Institute of Statistical Mathematics, Tachikawa, Tokyo, Japan.
Selection Criteria for Neuromanifolds of Stochastic Dynamics, June 12, 2011, The 3rd International Conference on Cognitive Neurodynamics , Niseko Village, Hokkaido, Japan.
On Exponential Families and the Expressive Power of Related Generative Models, March 14, 2011, Laboratoire d'Informatique des Systèmes Adaptatifs - Université de Montréal , [Abstract] 
Mixtures from Exponential Families, March 02, 2011, MPI MIS, Leipzig, Germany.
Universal approximation results for Restricted Boltzmann Machines and Deep Belief Networks, Neuronale Netze und Kognitive Systeme Seminar, February 16, 2011, MPI MIS, Leipzig, Germany.
Necessary conditions for RBM universal approximators, Meeting of the Department of Decision-Making Theory of the Institute of Information Theory and Automation UTIA, January 18, 2011, Marianska, Czech Republic.
Information Geometry of Mean-Field Methods, Fall School on Statistical Mechanics and 5th anual Ph.D. Student Conference in Probability, September 07-12, 2009, MPI MIS, Leipzig, Germany.
Quantum-Sanov-Theorem for correlated States in multidimensional Grids, Dies Mathematicus, February 2008, TU-Berlin.
Quanten-Sanov-Theorem im mehrdimensionallen Fall, Workshop on Complexity and Information Theory, October 2007, MPI MIS, Leipzig, Germany.
Poster
A comparison of neural network architectures, Deep Learning Workshop, ICML 2015.
Mode Poset Probability Polytopes [pdf]. Algebraic Statistics 2015, Department of Mathematics University of Genoa, Italy, June 8-11, 2015.
Deep Narrow Boltzmann Machines are Universal Approximators [pdf]. ICLR 2015.
On the Number of Linear Regions of Deep Neural Networks [pdf]. NIPS 2014.
A Framework for Cheap Universal Approximation in Embodied Systems, Autonomous Learning: 3. Symposium DFG Priority Programme 1527, Berlin, September 8-9, 2014.
Geometry of hidden-visible products of statistical models [pdf]. Algebraic Statistics at IIT, Chicago, IL, 2014.
When Does a Mixture of Products Contain a Product of Mixtures, NIPS 2012 - Deep Learning and Unsupervised Feature Learning Workshop. [abstract] 
Kernels and Submodels of Deep Belief Networks, NIPS 2012 - Deep Learning and Unsupervised Feature Learning Workshop. [abstract] 
Mixture Models and Representational Power of RBMs, DBNs and DBMs, NIPS 2010 - Deep Learning and Unsupervised Feature Learning Workshop, Whistler, Canada.
Faces of the probability simplex contained in the closure of an exponential family and minimal mixture representations, Information Geometry and its Applications III, Leipzig, Germany, 2010.
Theory of Transport and Photon-Statistics in a Biased Nanostructure. G.M., M. Richter, T. Brandes, and A. Knorr, Nano-Optoelectronics Workshop (i-NOW 2008), Tokyo and Shonan Village, Japan, 2008.
Recent and Upcoming
ICML, COLT 2016, New York, USA.
IGAIA IV 2016, Liblice, Czech Republic.
NIPS 2015, Montréal, Canada.
WUPES'15, Monínec, Czech Republic.
60th World Statistics Congress - ISI 2015, Rio de Janeiro, Brazil.
ICML 2015, Lille, France.
AS 2015, Genova, Italy.
NIPS 2014, Montréal, Canada.
Workshop: Information Geometry for Machine Learning, December 2014, RIKEN BSI, Japan.
Santa Fe Institute, Visit for Research Collaboration (Nihat Ay), October 15-November 15, 2014, Santa Fe, NM, USA.
Information Geometry in Learning and Optimization, September 22-26, 2014, University of Copenhagen.
Autonomous Learning: 3. Symposium DFG Priority Programme 1527, September 08-09, 2014, Magnus Haus Berlin.
Autonomous Learning: Summer School, September 01-04, 2014, MPI MIS.
Limit Theorems and Algebraic Statistics, Joint Workshop on, August 25-29, 2014, Prague.
Algebraic Statistics at IIT, May 2014, Chicago, IL, USA.
Santa Fe Institute, Visit for Research Collaboration (Nihat Ay), October 1-27, 2013.
SFI Working Group ``Information Theory of Sensorimotor Loops,'' October 8-11, 2013, Santa Fe Institute, Santa Fe, NM, USA.
Pennsylvania State University, Visit for Research Collaboration (Jason Morton), September 2013.
GSI 2013, August 28-30, 2013, Paris.
ICLR2013, May 2-4, 2013, Scottsdale, AZ, USA.
NIPS 2012Deep Learning Workshop, December 7-8, 2012, Lake Tahoe, NV, USA.
Algebraic Statistics in Europe, September 28-30, 2012, IST Austria.
WUPES 2012, September 12-15, 2012, Marianske Lazne, Czech Republic.
Deep Learning Summer School, IPAM - UCLA, July 9-27, 2012, Los Angeles, CA, USA.
SIAM conference on Discrete Mathematics 2012, June 18-21, 2012, Dalhousie University, Halifax, Nova Scotia, Canada.
Algebraic Statistics in the Alleghenies, June 08-15, 2012, Penn State, PA, USA.
Singular Learning Theory, AIM Workshop, December 12-16, 2011, American Institute of Mathematics, Palo Alto, CA, USA.
RIKEN-BSI, Laboratory for Mathematical Neuroscience (Prof. S. Amari), Internship, August-October 2011, Hirosawa, Wako, Saitama, Japan.
SFI Complex Systems Summer School, June 8-30, 2011, Santa Fe, NM, USA.
Third International Conference of Cognitive Neurodynamics (ICCN 2011), Niseko Village, Hokkaido, Japan.
When Does a Mixture of Products Contain a Product of Mixtures?
SIAM Journal on Discrete Mathematics, 29(1):321-347, 2015.

Imprint
G. Montúfar,  Email: montufarmis.mpg.de
Last updated: 03/2016