2026 Simons Collaboration on the Physics of Learning and Neural Computation Annual Meeting

Thursday, September 24, 2026

Friday, September 25, 2026

Yasaman Bahri
Google

Representations in Neural Computation

Learned representations lie at the core of AI systems, connecting the statistics of data to nontrivial computation. What ingredients shape these representations, and what can we learn by studying their geometry? How do representations enable generalization and the emergence of new computational capabilities? Yasman Bahri will survey the collaboration’s efforts on these questions, including how representational structure directly affects generalization; how representational manifolds can yield key insights into model behavior; and how certain inductive biases can be crucial for learning useful representations. Bahri will then discuss a series of works in which we address open questions in our understanding of LLMs by considering representations that are learned only from two-token statistics in natural language. We show that representational geometry is directly shaped by various forms of hidden structure within these statistics, giving rise to nontrivial computation and explaining some universal features observed within LLMs.

Mike Douglas
Harvard University

Reasoning: From Open Models to Solvable Models

Over the past year, AI systems have crossed into research-level mathematics — solving competition and open problems whose solutions can be checked exactly. Mike Douglas will review the collaboration’s progress toward a physics of reasoning, organized around what is now measurable. Fully open math models (data, all checkpoints) let us treat reasoning as an experimental subject: solve rate versus distance to the training set, the emergence chronology of skills, and interventions on data and curricula. On the theory side, the collaboration now has its first solvable models of test-time reasoning — chain-of-thought in tractable attention models, with phase diagrams separating regimes where deeper reasoning helps, saturates, or hurts — and a unified view of test-time scaling laws spanning best-of-N search, LLM judges, and verifiers. Douglas will also survey progress on reinforcement learning and self-play at the edge of learnability, and on formalization: research-level mathematics and physics codified in Lean, where the proof checker provides a perfect reward signal. Finally, he will close with open problems: what these results predict for mathematical data, and what to measure next.

Surya Ganguli
Stanford University

Overview of the Physics of Learning Collaboration

Artificial intelligence has made tremendous engineering advances that stand poised to transform our society, yet alarmingly, we understand very little scientifically about how AI actually works. The time is now ripe to study AI as a complex system, by both exploiting and extending the power of physics and mathematics to open the black box of AI systems and improve them in a principled manner. In this talk, colloraboration director Surya Ganguli will present the overarching structure of the collaboration and frame the questions that the collaboration seeks to address concerning the physics of learning and neural computation, setting the stage for the other collaboration talks that explore how AI learns, scales, computes, and reasons, and how the structure of data itself makes all this possible. Ganguli will conclude with some new results on how phase transitions in the ability of generative AI to imagine new data can be derived by a mapping to spin glasses and information theory.

Andrea Montanari
Stanford University

The Phase Diagram of 2-Layer Neural Networks

Classical statistical models are constructed so that either the number of parameters is small with respect to the sample size, or the regularization is strong enough so that their “effective dimension” is small. In contrast, modern machine learning models are vastly miss-specified and overparametrized, and only very loosely regularized. These models are learnt using empirical risk minimization, but the empirical risk function has many near-global optima which are very far from each other in parameter space. How can we make sense of the good statistical properties of these models? Andrea Montanari will summarize what we learned over the years by studying a simple and yet extremely rich setting: learning in two-layer networks.

Cengiz Pehlevan
Harvard University

Toward a Science of Efficient AI Training

Training modern AI systems is increasingly expensive, yet many of the choices that determine this cost still depend on extensive empirical search. This talk will review how combining ideas from theoretical physics with targeted experiments make large-scale training more predictable, transferable, and efficient. Cengiz Pehlevan will begin by examining how parameterization and hyperparameter choices shape optimization dynamics. Pehlevan will then show how this understanding can guide the design of training schedules, emerging optimizers, and architectures. Throughout, Pehlevan will highlight the resulting empirical gains and the adoption of these ideas in industry.

Andrew Saxe
University College London

Understanding Multistage Training and Strategy Emergence

Modern foundation models are trained through multistage pipelines, yet basic questions remain poorly understood: how should limited compute be allocated across stages, and can reinforcement learning create new capabilities during post-training or only reweight existing primitives? In this talk, Andrew Saxe will address these questions with analytical and empirical methods. Saxe will introduce an analytically tractable model of multistage training, shedding light on the factors affecting optimal compute allocation to each stage. In the model, pretraining sets the ceiling on RL performance, with RL learning to reweight primitive strategies. Next, using a controlled rewrite-grammar environment where the pretraining distribution is known, Saxe will empirically show that transformers trained with reinforcement learning with verifiable rewards solve problems that remain nearly impossible after pretraining. By analyzing trajectories, RL first strengthens primitive reductions and then enters a chunking phase in which it forms valid compressed procedures that combine several primitive steps into one. Together, these results begin to shed light on the interactions across stages of modern training pipelines.

Eva Silverstein
Stanford University

Learning Dynamics, Representations and Symmetry Breaking

How do modern neural networks evolve from randomly initialized weights toward a useful representation of the data? How does this proceed layerwise, and what role does the structure of attention play in the case of language models?

Collaboration progress on this central problem draws from diverse methods including dynamical mean field theory, spectral methods, tractable toy models, symmetry analysis, and stochastic dynamics. Eva Silverstein will review several inter-related results exhibiting explicitly how interpretable representations emerge. These encompass multiple stages of learning: the escape from an initially uninformative regime, the emergence of structured directions during training, and, in simplified hierarchical models, the layer-by-layer construction of increasingly useful features. The structured directions that emerge during the learning dynamics include novel ones introduced via symmetry breaking in attention, enabling models to amplify or suppress meaningful token and representation classes.         

Matthieu Wyart
Johns Hopkins University

How Much Data Is Required to Learn the Structure of Language?

Despite the fact that experimental neural scaling laws have substantially guided empirical progress in large-scale machine learning, no existing theory can quantitatively predict the exponents of these important laws for any modern LLM trained on natural languages. It is also unclear whether they can be beaten by training with other objectives. Matthieu Wyart will review the efforts of the collaboration in studying this phenomenon, including the first such theory in the case of data-limited scaling laws. Wyart will then introduce synthetic languages displaying a hidden hierarchy (a central notion in both linguistics and computer vision), and discuss how these languages are learnt by deep, but not shallow, nets. Scaling laws in these synthetic languages follow from the same principle as in natural languages when trained at the token level, but these languages can be learnt much more efficiently by machines predicting in an abstract space — justifying the success of joint embedding predictive architectures (JEPA) and other machines learning from their own latents.

Participation in the meeting falls into the following four categories. An individual’s participation category is communicated via their letter of invitation.

The Simons Foundation will never ask for credit card information or require payment for registration to our events.

Group A – Speakers & Organizers

Individuals in Group A receive travel and hotel coordination within the following parameters:

Travel
Economy Class: For flights that are three hours or less to your destination, the maximum allowable class of service is Economy class.
Premium Economy Class: For flights where the total air travel time (excluding connection time) is more than three hours and less than seven hours per segment to your destination, the maximum allowable class of service is premium economy.
Business Class: When traveling internationally (or to Hawaii/Alaska) travelers are permitted to travel in Business Class on those segments that are seven hours or more. If the routing is over budget, a premium economy or mixed-class ticket will be booked.

Hotel
Up to 3 nights at the conference hotel, arriving on Wednesday, September 23 and departing on Saturday, September 26.

Group B – Funded Participants

Individuals in Group B receive travel and hotel coordination within the following parameters:

Travel
Economy class travel will be booked regardless of flight length.

Hotel
Up to 3 nights at the conference hotel, arriving on Wednesday, September 23 and departing on Saturday, September 26.

Group C – Unfunded Participants

Individuals in Group C will not receive financial support but are encouraged to enjoy all conference-hosted meals.

Group D – Remote Participants

Individuals in Group D will participate in the meeting remotely.

Air and Rail

For funded individuals, the foundation will arrange and pay for round-trip travel from their home city to the conference city. All travel and hotel arrangements must be booked through the Simons Foundation’s preferred travel agency.

Travel Deviations

The following travel specifications are considered deviations and will only be accommodated if the cost is less than or equal to the amount the Simons Foundation would pay for a standard round-trip ticket from your home city to the conference city:

• Preferred airline
• Preferred travel class
• Specific flights/flight times
• Travel dates outside those associated with the conference
• Arriving or departing from an airport other than your home city or conference city airports, i.e. multi-segment or triangle trips.

All deviations must be reviewed and approved by the Simons Foundation and, if the cost is more than what would normally be paid, a reimbursement quote must be obtained through the foundation’s travel agency before proceeding to booking and paying for travel out of pocket. All reimbursements for travel booked directly will be paid after the conclusion of the meeting.

Changes After Ticketing

All costs related to changes made to ticketed travel are to be paid for by the participant and are not reimbursable. Please contact the foundation’s travel agency for further assistance.

Personal & Rental Cars

Personal car and rental trips over 250 miles each way require prior approval from the Simons Foundation via email.

Rental cars must be pre-approved by the Simons Foundation.

The Hotel Seville NoMad offers valet parking. Please note there are no in-and-out privileges when using the hotel’s garage, therefore it is encouraged that participants walk or take public transportation to the Simons Foundation.

Hotel

Funded individuals who require hotel accommodations are hosted by the foundation for a maximum of 3 nights at the conference hotel, arriving on Wednesday, September 23 and departing on Saturday, September 26.

Any additional nights are at the attendee’s own expense. To arrange accommodations, please register at the link included in your invitation.

Hotel Seville NoMad
22 E 29th St
New York, NY 10016
(between 28th and 29th Streets)
https://www.hyatt.com/unbound-collection/en-US/nycud-hotel-seville-nomad

For driving directions to the Hotel Seville NoMad, please click here.

Overview

In-person participants will be reimbursed for meals and local expenses including ground transportation. Expenses should be submitted through the foundation’s online expense reimbursement platform after the meeting’s conclusion.

Expenses accrued because of meetings not directly related to the Simons Foundation-hosted meeting (a satellite meeting or meeting held at another institution, for example) will not be reimbursed by the Simons Foundation and should be paid by other sources.

Below are key reimbursement takeaways; a full policy will be provided with the final logistics email circulated approximately 2 weeks prior to the meeting’s start.

Meals

The daily meal limit is $125; itemized receipts are required for expenses over $24 USD. The foundation DOES NOT provide a meal per diem and only reimburses actual meal expenses up the following amounts.

• Breakfast $20
• Lunch $30
• Dinner $75

Allowable Meal Expenses

• Meals taken on travel days (when you traveled by air or train).
• Meals not provided on a meeting day, dinner on Friday for example.
• Group dinners consisting of fellow meeting participants paid by a single person will be reimbursed up to $75 per person and the amount will count towards the $125 daily meal limit.

Unallowable Meal Expenses

• Meals taken outside those provided by the foundation (breakfast, lunch, breaks and/or dinner).
• Meals taken on days not associated with Simons Foundation-coordinated events.
• Minibar expenses.
• Meal expenses for a non-foundation guest.

• Ubers, Lyfts, taxis, etc., taken to and from restaurants in Manhattan.

  • Accommodations will be made for those with mobility restrictions.

Ground Transportation

Expenses for ground transportation will be reimbursed for travel days (i.e. traveling to/from the airport or train station) as well as subway and bus fares while in Manhattan are reimbursable.

Transportation to/from satellite meetings are not reimbursable.

Attendance

In-person participants and speakers are expected to attend all meeting days. Participants receiving hotel and travel support wishing to arrive on meeting days which conclude at 2:00 PM will be asked to attend remotely.

Entry & Building Access

Upon arrival, guests will be required to show their photo ID to enter the Simons Foundation and Flatiron Institute buildings. After checking-in at the meeting reception desk, guests will be able to show their meeting name badge to re-enter the building. If you forget your name badge, you will need to provide your photo ID.

The Simons Foundation and Flatiron Institute buildings are not considered “open campuses” and meeting participants will only have access to the spaces in which the meeting will take place. All other areas are off limits without prior approval.

If you require a private space to conduct a phone call or remote meeting, please contact your meeting manager at least 48-hours ahead of time so that they may book a space for you within the foundation’s room reservation system.

Guests & Children

Meeting participants are required to give 24-hour advance notice of any guests meeting them at the Simons Foundation either before or after the meeting. Outside guests are discouraged from joining meeting activities, including meals.

With the exception of Simons Foundation and Flatiron Institute staff, ad hoc meeting participants who did not receive a meeting invitation directly from the Simons Foundation are not permitted.

Children under the age of 18 are not permitted to attend meetings at the Simons Foundation. Furthermore, the Simons Foundation does not provide childcare facilities or support of any kind. Special accommodations will be made for nursing parents.

Meeting & Policy Questions

Meghan Fazzi
Senior Manager, Events & Administration
[email protected]

Travel & Hotel Support

FCM Travel Meetings & Events
[email protected]
Hours: M-F, 8:30 AM-5:00 PM ET
+1-888-789-6639