2026 Simons Collaboration on the Physics of Learning and Neural Computation Annual Meeting

Conference

Speakers

  • Yasman Bahri Google
  • Mike Douglas Harvard University
  • Surya Ganguli Stanford University
  • Andrea Montanari Stanford University
  • Cengiz Pehlevan Harvard University
  • Andrew Saxe Gatsby University College London
  • Eva Silverstein Stanford University
  • Matthieu Wyart Johns Hopkins University

When

  • All Day

  • All Day

Where

Gerald D. Fischbach Auditorium

160 5th Ave

New York, NY 10010, United States

Open Map for Gerald D. Fischbach Auditorium

Meeting Goals

Organized by Surya Ganguli (Stanford University), the Simons Collaboration on Physics of Learning and Neural Computation seeks to foster collaborations to build a new international community of researchers engaged in the physics of learning and neural computation. The collaboration’s fundamental aim is to develop a deeper scientific understanding of artificial intelligence leading to better, more efficient, and more robust AI through rational design principles and a wide array of disciplinary viewpoints rooted in the physics and mathematics of complex systems will be required to advance our understanding.

The annual meeting will welcome wide variety of researchers together to explore how AI learns, computes, scales, reasons and imagines.

Agenda

Thursday, September 24, 2026

9:30 AM Surya Ganguli I Overview of the Physics of Learning Collaboration
11:00 AM Eva Silverstein I Learning Dynamics, Representations and Symmetry breaking
1:00 PM Cengiz Pehlevan I Toward a Science of Efficient AI Training
2:30 PM Yasaman Bahri I Representations in Neural Computation
4:00 PM Andrea Montanari I The Phase Diagram of 2-layer Neural Networks

Friday, September 25, 2026

9:30 AM Matthieu Wyart I How Much Data is Required to Learn the Structure of Language?
11:00 AM Michael Douglas I Reasoning: From Open Models to Solvable Models
1:00 PM Andrew Saxe I Understanding Multistage Training and Strategy Emergence
Abstracts

Yasaman Bahri
Google

Representations in Neural Computation

Learned representations lie at the core of AI systems, connecting the statistics of data to nontrivial computation. What ingredients shape these representations, and what can we learn by studying their geometry? How do representations enable generalization and the emergence of new computational capabilities? Yasman Bahri will survey the collaboration’s efforts on these questions, including how representational structure directly affects generalization; how representational manifolds can yield key insights into model behavior; and how certain inductive biases can be crucial for learning useful representations. Bahri will then discuss a series of works in which we address open questions in our understanding of LLMs by considering representations that are learned only from two-token statistics in natural language. We show that representational geometry is directly shaped by various forms of hidden structure within these statistics, giving rise to nontrivial computation and explaining some universal features observed within LLMs.

Mike Douglas
Harvard University

Reasoning: From Open Models to Solvable Models

Over the past year, AI systems have crossed into research-level mathematics — solving competition and open problems whose solutions can be checked exactly. Mike Douglas will review the collaboration’s progress toward a physics of reasoning, organized around what is now measurable. Fully open math models (data, all checkpoints) let us treat reasoning as an experimental subject: solve rate versus distance to the training set, the emergence chronology of skills, and interventions on data and curricula. On the theory side, the collaboration now has its first solvable models of test-time reasoning — chain-of-thought in tractable attention models, with phase diagrams separating regimes where deeper reasoning helps, saturates, or hurts — and a unified view of test-time scaling laws spanning best-of-N search, LLM judges, and verifiers. Douglas will also survey progress on reinforcement learning and self-play at the edge of learnability, and on formalization: research-level mathematics and physics codified in Lean, where the proof checker provides a perfect reward signal. Finally, he will close with open problems: what these results predict for mathematical data, and what to measure next.

Surya Ganguli
Stanford University

Overview of the Physics of Learning Collaboration

Artificial intelligence has made tremendous engineering advances that stand poised to transform our society, yet alarmingly, we understand very little scientifically about how AI actually works. The time is now ripe to study AI as a complex system, by both exploiting and extending the power of physics and mathematics to open the black box of AI systems and improve them in a principled manner. In this talk, colloraboration director Surya Ganguli will present the overarching structure of the collaboration and frame the questions that the collaboration seeks to address concerning the physics of learning and neural computation, setting the stage for the other collaboration talks that explore how AI learns, scales, computes, and reasons, and how the structure of data itself makes all this possible. Ganguli will conclude with some new results on how phase transitions in the ability of generative AI to imagine new data can be derived by a mapping to spin glasses and information theory.

Andrea Montanari
Stanford University

The Phase Diagram of 2-Layer Neural Networks

Classical statistical models are constructed so that either the number of parameters is small with respect to the sample size, or the regularization is strong enough so that their “effective dimension” is small. In contrast, modern machine learning models are vastly miss-specified and overparametrized, and only very loosely regularized. These models are learnt using empirical risk minimization, but the empirical risk function has many near-global optima which are very far from each other in parameter space. How can we make sense of the good statistical properties of these models? Andrea Montanari will summarize what we learned over the years by studying a simple and yet extremely rich setting: learning in two-layer networks.

Cengiz Pehlevan
Harvard University

Toward a Science of Efficient AI Training

Training modern AI systems is increasingly expensive, yet many of the choices that determine this cost still depend on extensive empirical search. This talk will review how combining ideas from theoretical physics with targeted experiments make large-scale training more predictable, transferable, and efficient. Cengiz Pehlevan will begin by examining how parameterization and hyperparameter choices shape optimization dynamics. Pehlevan will then show how this understanding can guide the design of training schedules, emerging optimizers, and architectures. Throughout, Pehlevan will highlight the resulting empirical gains and the adoption of these ideas in industry.

Andrew Saxe
University College London

Understanding Multistage Training and Strategy Emergence

Modern foundation models are trained through multistage pipelines, yet basic questions remain poorly understood: how should limited compute be allocated across stages, and can reinforcement learning create new capabilities during post-training or only reweight existing primitives? In this talk, Andrew Saxe will address these questions with analytical and empirical methods. Saxe will introduce an analytically tractable model of multistage training, shedding light on the factors affecting optimal compute allocation to each stage. In the model, pretraining sets the ceiling on RL performance, with RL learning to reweight primitive strategies. Next, using a controlled rewrite-grammar environment where the pretraining distribution is known, Saxe will empirically show that transformers trained with reinforcement learning with verifiable rewards solve problems that remain nearly impossible after pretraining. By analyzing trajectories, RL first strengthens primitive reductions and then enters a chunking phase in which it forms valid compressed procedures that combine several primitive steps into one. Together, these results begin to shed light on the interactions across stages of modern training pipelines.

Eva Silverstein
Stanford University

Learning Dynamics, Representations and Symmetry Breaking

How do modern neural networks evolve from randomly initialized weights toward a useful representation of the data? How does this proceed layerwise, and what role does the structure of attention play in the case of language models?

Collaboration progress on this central problem draws from diverse methods including dynamical mean field theory, spectral methods, tractable toy models, symmetry analysis, and stochastic dynamics. Eva Silverstein will review several inter-related results exhibiting explicitly how interpretable representations emerge. These encompass multiple stages of learning: the escape from an initially uninformative regime, the emergence of structured directions during training, and, in simplified hierarchical models, the layer-by-layer construction of increasingly useful features. The structured directions that emerge during the learning dynamics include novel ones introduced via symmetry breaking in attention, enabling models to amplify or suppress meaningful token and representation classes.         

Matthieu Wyart
Johns Hopkins University

How Much Data Is Required to Learn the Structure of Language?

Despite the fact that experimental neural scaling laws have substantially guided empirical progress in large-scale machine learning, no existing theory can quantitatively predict the exponents of these important laws for any modern LLM trained on natural languages. It is also unclear whether they can be beaten by training with other objectives. Matthieu Wyart will review the efforts of the collaboration in studying this phenomenon, including the first such theory in the case of data-limited scaling laws. Wyart will then introduce synthetic languages displaying a hidden hierarchy (a central notion in both linguistics and computer vision), and discuss how these languages are learnt by deep, but not shallow, nets. Scaling laws in these synthetic languages follow from the same principle as in natural languages when trained at the token level, but these languages can be learnt much more efficiently by machines predicting in an abstract space — justifying the success of joint embedding predictive architectures (JEPA) and other machines learning from their own latents.

More Events

November 19–20, 2026

January 4–8, 2027

February 4–5, 2027

Speakers

Charlotte Burrows, Shafi Goldwasser, Tatsu Hashimoto, Jon Kleinberg, Ben Laufer, + 3 more