Universality Phenomena in Machine Learning, and Their Applications

Date & Time

About Simons Foundation Lectures

Simons Foundation Lectures are free public colloquia related to basic science and mathematics. These high-level talks are intended for professors, students, postdocs and business professionals, but interested people from the metropolitan area are welcome as well.
Video Thumbnail

By clicking to watch this video, you agree to our privacy policy.

A canonical task in machine learning is to fit a model (from a certain class) to a dataset. In many settings there is little theoretical understanding of the algorithms used for this task, since they involve nonconvex optimization.

We have empirically observed that in many settings the models fitted to real-life datasets display randomlike properties — the model parameters behave like random numbers for various tests. This is somewhat reminiscent of the ‘universality’ phenomenon in mathematics and physics, whereby matrices in a host of settings turn out to display properties similar to those of the Gaussian ensemble.

In this talk, Sanjeev Arora will describe how these randomlike properties can be used to gain a new understanding in some settings — for example, they can offer insights into linear algebraic properties of word meanings in natural languages, and reversibility properties of fully connected deep nets. In some cases, they can lead us to provably efficient algorithms, such as algorithms for making inferences in a topic model.

About the Speaker

Sanjeev Arora is the Charles C. Fitzmorris Professor of Computer Science at Princeton University. His research area spans several areas of theoretical Computer Science including computational complexity and algorithm design, and theoretical problems in machine learning. He has received the ACM-EATCS Gödel Prize (in 2001 and 2010), Packard Fellowship (1997), the ACM Infosys Foundation Award in the Computing Sciences (2012), the Fulkerson Prize (2012), and the Simons Investigator Award (2012).

Advancing Research in Basic Science and MathematicsSubscribe to our newsletters to receive news & updates