These New AI Models Are Trained on Physics, Not Words, and They’re Driving Discovery

While most AI models — including ChatGPT — are trained on text and images, a multidisciplinary team of scientists has something different in mind: AI trained on physics.

Recently, members of the Polymathic AI collaboration presented two new AI models trained using real scientific datasets to tackle problems in astronomy and fluidlike systems.

The models — called Walrus and AION-1 — are unique in that they can apply the knowledge they gain from one class of physical systems to seemingly completely different problems. For instance, Walrus can tackle systems ranging from exploding stars to Wi-Fi signals to the movement of bacteria.

That cross-disciplinary skill set is particularly exciting because it can accelerate scientific discovery and give researchers a leg up when faced with small samples or budgets, says Walrus lead developer Michael McCabe, a research scientist at Polymathic AI.

“Maybe you have new physics in your scenario that your field isn’t used to handling. Maybe you’re using experimental data, and you’re not quite sure what class it fits into. Maybe you’re just not a machine-learning researcher and just can’t burn the time working through all the possible models that might fit your scenario,” McCabe explains. “Our hope is that training on these broader classes makes something that is both easier to use and has a better chance of generalizing for those users, as the ‘new’ physics to them might be something another field has been handling for a while.”

Using cross-disciplinary models can also improve predictions when data is sparse or when studying rare events, says Liam Parker, a Ph.D. student at the University of California, Berkeley, and a lead researcher developing for AION-1.

The Polymathic AI team recently announced Walrus in a preprint on arXiv.org and presented AION-1 on Friday, December 5, at the NeurIPS conference in San Diego.

Walrus and AION-1 are ‘foundational models,’ meaning they’re trained on colossal sets of training data from different research areas or experiments. That’s unlike most AI models in science, which are trained with a particular subfield or problem in mind. Rather than learning the ins and outs of a particular situation or starting from a set of fundamental equations, foundational models instead learn the basis, or foundation, of the physical processes at work. Since these physical processes are universal, the knowledge that the AI learns can be applied to various fields or problems that share the same underlying physical principles. Foundational models have a host of benefits — from speeding up computations to performing well in low-data regimes to finding physics shared across different fields.

AION-1 is a foundational model for astronomy. It is trained on data from astronomical surveys that are already massive in their own right: the Legacy Survey, the Hyper Suprime-Cam (HSC), the Sloan Digital Sky Survey (SDSS), the Dark Energy Spectroscopic Instrument (DESI) and Gaia. All in all, that’s more than 200 million observations of stars, quasars and galaxies totaling around 100 terabytes of data. AION-1 uses images, spectra and a variety of other measurements to learn as much as it can about astronomical objects. Then, when a scientist obtains a low-resolution image of a galaxy, for example, AION-1 can extract more information about it, learned from the physics of millions of other galaxies.

Walrus’ domain is fluids and fluidlike systems. Walrus utilizes the Well — a massive dataset compiled by the Polymathic AI team. The Well’s data encompasses 19 different scenarios and 63 different fields in fluid dynamics. All in all, it contains 15 terabytes of data describing parameters such as density, velocity and pressure in physical systems as wide-ranging as merging neutron stars, acoustic waves and shifting layers in Earth’s atmosphere.

Such foundational models can be powerful. AION-1 and Walrus can utilize physics seen in a different case and apply it to learn about something new. It is similar to our senses. “Multiple senses together — rather than one at a time — gives you a fuller understanding of an experience,” the AION-1 team explained in a blog post about the project. “Over time, your brain learns associations between how things look, taste and smell, so if one sense is unavailable, you can often infer the missing information from the others.”

Then, when a scientist is performing a new experiment or observation, they have a starting point — a map of how physics behaves in other similar situations. “It’s like seeing many, many humans,” says Shirley Ho, Polymathic AI’s principal investigator and an astrophysicist and machine learning expert. Ho is a senior research scientist at the Flatiron Institute and a professor at New York University. When “you meet a new friend, because you’ve met so many people before now, you are able to map in your head … what this human is going to be like compared to all your friends before,” she says.

Foundational models make scientists’ lives easier by streamlining data processing. Scientists will no longer have to create a new framework from scratch for every project or task; instead, they can start with an already trained AI to use as a foundation. “I think our vision for some of this foundation model is that it enables anyone to start from a really powerful embedding of the data that they’re interested in … and still achieve state-of-the-art accuracy without having to build this whole pipeline from scratch,” says AION-1 lead researcher Parker.

Their goal is to make tools that scientists can use in their day-to-day research. “We want to bring all this AI intelligence” to the scientists who need it, Ho says.

CosmoBench: CosmoBench is a multiview, multiscale, multitask cosmology benchmark for geometric deep learning. Curated from the state-of-the-art cosmological simulations, CosmoBench is the largest benchmark of its kind, with over 34,000 point clouds and 25,000 directed trees. CosmoBench features challenging evaluation tasks from cosmology and diverse baselines, including cosmological methods, simple linear models and graph neural networks. This presentation will show how CosmoBench is pushing the frontiers of cosmology and geometric deep learning.

Lost in Latent Space: Physicists model and predict the behavior of physical systems using their understanding of the laws of physics. However, these calculations require significant computing power. Flatiron Institute scientists and other members of the Polymathic AI collaboration studied whether a less taxing form of computing can still yield accurate results. Known as ‘latent diffusion modeling,’ this computational model utilizes artificial intelligence to generate high-quality images at a lower computational cost while accurately capturing physical behavior.

Neurons as Detectors of Coherent Sets in Sensory Dynamics: Our perception of touch, taste, sight and pain is mediated by neurons that carry signals from peripheral receptors to the brain. This work shows that these neurons can be understood as detecting ‘coherent sets’ within the sensory stream — groups of stimulus trajectories that evolve together over time and therefore share a common past or a common future. By distinguishing these coherent sets, some neurons predominantly encode what has just occurred, while others reliably signal what is likely to happen next. Traditional classifications of sensory neurons can thus be reinterpreted as reflecting a division between past-focused and future-predictive processing. Understanding how the nervous system separates and transforms sensory input in this way may offer new routes for treating mental illness and may also guide the development of biologically inspired artificial intelligence.

Predicting Partially Observable Dynamical Systems: Scientists can predict the motion of a falling object or the evolution of fluids using deterministic models that compute a single future outcome from past observations. But this approach breaks down for physical systems where much of the state is hidden. A prominent example is the sun: We can observe the activity on its surface, but the processes deep inside remain largely invisible. Without access to those internal conditions, there isn’t enough information to forecast a single ‘correct’ future. Researchers at the Flatiron Institute, together with collaborators in the Polymathic AI project, have developed a probabilistic approach that can infer these hidden solar processes. By incorporating information from the distant past into a diffusion-based generative model, their method produces an ensemble of plausible futures, offering a clearer understanding of how past sunspot activity shapes its future evolution.

For more information, please contact [email protected].