2697 Publications

Polar prediction of natural videos

Observer motion and continuous deformations of objects and surfaces imbue natural videos with distinct temporal structures, enabling partial prediction of future frames from past ones. Conventional methods first estimate local motion, or optic flow, and then use it to predict future frames by warping or copying content. Here, we explore a more direct methodology, in which each frame is mapped into a learned representation space where the structure of temporal evolution is more readily accessible. Motivated by the geometry of the Fourier shift theorem and its group-theoretic generalization, we formulate a simple architecture that represents video frames in learned local polar coordinates. Specifically, we construct networks in which pairs of convolutional channel coefficients are treated as complex-valued, and are optimized to evolve with slowly varying amplitudes and linearly advancing phases. We train these models on next-frame prediction in natural videos, and compare their performance with that of conventional methods using optic flow as well as predictive neural networks. We find that the polar predictor achieves better performance while remaining interpretable and fast, thereby demonstrating the potential of a flow-free video processing methodology that is trained end-to-end to predict natural video content.

Show Abstract

Catalyzing next-generation Artificial Intelligence through NeuroAI

Anthony Zador, Sean Escola, Blake Richards, Bence Ölveczky, Yoshua Bengio, Kwabena Boahen, Matthew Botvinick, D. Chklovskii, Anne Churchland, Claudia Clopath, James DiCarlo, Surya Ganguli, Jeff Hawkins, Konrad Körding, Alexei Koulakov, Yann LeCun, Timothy Lillicrap, Adam Marblestone, Bruno Olshausen, Alexandre Pouget, Cristina Savin, Terrence Sejnowski, E. P. Simoncelli, Sara Solla, David Sussillo, Andreas S. Tolias, Doris Tsao

Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities – inherited from over 500 million years of evolution – that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.

Show Abstract

Quantum electrodynamics of chiral and antichiral waveguide arrays

Jeremy G. Hoskins, M. Rachh, John C. Schotland

We consider the quantum electrodynamics of single photons in arrays of one-way waveguides, each containing many atoms. We investigate both chiral and antichiral arrays, in which the group velocities of the waveguides are the same or alternate in sign, respectively. We find that in the continuum limit, the one-photon amplitude obeys a Dirac equation. In the chiral case, the Dirac equation is hyperbolic, while in the antichiral case it is elliptic. This distinction has implications for the nature of photon transport in waveguide arrays. Our results are illustrated by numerical simulations.

Show Abstract

Catalyzing next-generation [Artificial Intelligence} through {NeuroAI}

A Zador, S Escola, B Richards, B Ölveczky, Y Bengio, K Baohen, M Botvinick, D. Chklovskii, A Churchland, C Clopath, J DiCarlo, S Ganguli, J Hawkins, K Körding, A Koulakov, Y LeCun, T Lillicrap, A Marblestone, B Olshausen, A Pouget, Cristina Savin, T Sejnowski, E. P. Simoncelli, S Solla, D Sussillo, AS Tolias, D Tsao

Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities – inherited from over 500 million years of evolution – that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.

Show Abstract

Fast quantum circuit cutting with randomized measurements

We propose a new method to extend the size of a quantum computation beyond the number of physical qubits available on a single device. This is accomplished by randomly inserting measure-and-prepare channels to express the output state of a large circuit as a separable state across distinct devices. Our method employs randomized measurements, resulting in a sample overhead that is O(4
Show Abstract

Ion filling of a one-dimensional nanofluidic channel in the interaction confinement regime

Ion transport measurements are widely used as an indirect probe for various properties of confined electrolytes. It is generally assumed that the ion concentration in a nanoscale channel is equal to the ion concentration in the macroscopic reservoirs it connects to, with deviations arising only in the presence of surface charges on the channel walls. Here, we show that this assumption may break down even in a neutral channel, due to electrostatic correlations between the ions arising in the regime of interaction confinement, where Coulomb interactions are reinforced due to the presence of the channel walls. We focus on a one-dimensional channel geometry, where an exact evaluation of the electrolyte's partition function is possible with a transfer operator approach. Our exact solution reveals that in nanometre-scale channels, the ion concentration is generally lower than in the reservoirs, and depends continuously on the bulk salt concentration, in contrast to conventional mean-field theory that predicts an abrupt filling transition. We develop a modified mean-field theory taking into account the presence of ion pairs that agrees quantitatively with the exact solution and provides predictions for experimentally-relevant observables such as the ionic conductivity. Our results will guide the interpretation of nanoscale ion transport measurements.
Show Abstract

Attosecond magnetization dynamics in non-magnetic materials driven by intense femtosecond lasers

Irradiating solids with ultrashort laser pulses is known to initiate femtosecond timescale magnetization dynamics. However, sub-femtosecond spin dynamics have not yet been observed or predicted. Here, we explore ultrafast light-driven spin dynamics in a highly non-resonant strong-field regime. Through state-of-the-art ab-initio calculations, we predict that a non-magnetic material can be transiently transformed into a magnetic one via dynamical extremely nonlinear spin-flipping processes, which occur on attosecond timescales and are mediated by a combination of multi-photon and spin-orbit interactions. These are non-perturbative non-resonant analogues to the inverse Faraday effect that build up from cycle-to-cycle as electrons gain angular momentum. Remarkably, we show that even for linearly polarized driving, where one does not intuitively expect any magnetic response, the magnetization transiently oscillates as the system interacts with light. This oscillating response is enabled by transverse anomalous light-driven currents in the solid, and typically occurs on timescales of 500 attoseconds. We further demonstrate that the speed of magnetization can be controlled by tuning the laser wavelength and intensity. An experimental set-up capable of measuring these dynamics through pump-probe transient absorption spectroscopy is outlined and simulated. Our results pave the way for new regimes of ultrafast manipulation of magnetism.
Show Abstract
March 1, 2023

Electronic Descriptors for Supervised Spectroscopic Predictions

Spectroscopic properties of molecules holds great importance for the description of the molecular response under the effect of an UV/Vis electromagnetic radiation. Computationally expensive ab initio (e.g. MultiConfigurational SCF, Coupled Cluster) or TDDFT methods are commonly used by the quantum chemistry community to compute these properties. In this work, we propose a (supervised) Machine Learning approach to model the absorption spectra of organic molecules. Several supervised ML methods have been tested such as Kernel Ridge Regression (KRR), Multiperceptron Neural Networs (MLP) and Convolutional Neural Networks. The use of only geometrical descriptors (e.g. Coulomb Matrix) proved to be insufficient for an accurate training. Inspired on the TDDFT theory, we propose to use a set of electronic descriptors obtained from low-cost DFT methods: orbital energy differences, transition dipole moment between occupied and unoccupied Kohn-Sham orbitals and charge-transfer character of mono-excitations. We demonstrate that with this electronic descriptors and the use of Neural Networks we can predict not only a density of excited states, but also getting very good estimation of the absorption spectrum and charge-transfer character of the electronic excited states, reaching results close to the chemical accuracy ( 2 kcal/mol or 0.1eV).
Show Abstract
March 1, 2023
  • Previous Page
  • Viewing
  • Next Page
Advancing Research in Basic Science and MathematicsSubscribe to Flatiron Institute announcements and other foundation updates