Publications

Learning to detect content-independent transformations from data is one of the central problems in biological and artificial intelligence. An example of such problem is unsupervised learning of a visual motion detector from pairs of consecutive video frames. Rao and Ruderman formulated this problem in terms of learning infinitesimal transformation operators (Lie group generators) via minimizing image reconstruction error. Unfortunately, it is difficult to map their model onto a biologically plausible neural network (NN) with local learning rules. Here we propose a biologically plausible model of motion detection. We also adopt the transformation-operator approach but, instead of reconstruction-error minimization, start with a similarity-preserving objective function. An online algorithm that optimizes such an objective function naturally maps onto an NN with biologically plausible learning rules. The trained NN recapitulates major features of the well-studied motion detector in the fly. In particular, it is consistent with the experimental observation that local motion detectors combine information from at least three adjacent pixels, something that contradicts the celebrated Hassenstein-Reichardt model.

Deep convolutional neural networks (CNNs) for image denoising are usually trained on large datasets. These models achieve the current state of the art, but they have difficulties generalizing when applied to data that deviate from the training distribution. Recent work has shown that it is possible to train denoisers on a single noisy image. These models adapt to the features of the test image, but their performance is limited by the small amount of information used to train them. Here we propose “GainTuning”, in which CNN models pre trained on large datasets are adaptively and selectively adjusted for individual test images. To avoid overfitting, GainTuning optimizes a single multiplicative scaling parameter (the “Gain”) of each channel in the convolutional layers of the CNN. We show that GainTuning improves state-of-the-art CNNs on standard image-denoising benchmarks, boosting their denoising performance on nearly every image in a held-out test set. These adaptive improvements are even more substantial for test images differing systematically from the training data, either in noise level or image type. We illustrate the potential of adaptive denoising in a scientific application, in which a CNN is trained on synthetic data, and tested on real transmission-electronmicroscope images. In contrast to the existing methodology, GainTuning is able to faithfully reconstruct the structure of catalytic nanoparticles from these data at extremely low signal-to-noise ratios.

Deep convolutional neural networks (CNNs) for video denoising are typically trained with supervision, assuming the availability of clean videos. However, in many applications, such as microscopy, noiseless videos are not available. To address this, we propose an Unsupervised Deep Video Denoiser (UDVD1), a CNN architecture designed to be trained exclusively with noisy data. The performance of UDVD is comparable to the supervised state-of-the-art, even when trained only on a single short noisy video. We demonstrate the promise of our approach in real-world imaging applications by denoising raw video, fluorescencemicroscopy and electron-microscopy data. In contrast to many current approaches to video denoising, UDVD does not require explicit motion compensation. This is advantageous because motion compensation is computationally expensive, and can be unreliable when the input data are noisy. A gradient-based analysis reveals that UDVD automatically adapts to local motion in the input noisy videos. Thus, the network learns to perform implicit motion compensation, even though it is only trained for denoising.

As the study of the human brain is complicated by its sheer scale, complexity, and impracticality of invasive experiments, neuroscience research has long relied on model organisms. The brains of macaque, mouse, zebrafish, fruit fly, nematode, and others have yielded many secrets that advanced our understanding of the human brain. Here, we propose that adding miniature insects to this collection would reduce the costs and accelerate brain research. The smallest insects occupy a special place among miniature animals: despite their body sizes, comparable to unicellular organisms, they retain complex brains that include thousands of neurons. Their brains possess the advantages of those in insects, such as neuronal identifiability and the connectome stereotypy, yet are smaller and hence easier to map and understand. Finally, the brains of miniature insects offer insights into the evolution of brain design.

Neurons in primate visual cortex (area V1) are tuned for spatial frequency, in a manner that depends on their position in the visual field. Several studies have examined this dependency using fMRI, reporting preferred spatial frequencies (tuning curve peaks) of V1 voxels as a function of eccentricity, but their results differ by as much as two octaves, presumably due to differences in stimuli, measurements, and analysis methodology. Here, we characterize spatial frequency tuning at a millimeter resolution within human primary visual cortex, across stimulus orientation and visual field locations. We measured fMRI responses to a novel set of stimuli, constructed as sinusoidal gratings in log-polar coordinates, which include circular, radial, and spiral geometries. For each individual stimulus, the local spatial frequency varies inversely with eccentricity, and for any given location in the visual field, the full set of stimuli span a broad range of spatial frequencies and orientations. Over the measured range of eccentricities, the preferred spatial frequency is well-fit by a function that varies as the inverse of the eccentricity plus a small constant. We also find small but systematic effects of local stimulus orientation, defined in both absolute coordinates and relative to visual field location. Specifically, peak spatial frequency is higher for tangential than radial orientations and for horizontal than vertical orientations.

The development of neuromorphic hardware and modeling of biological neural networks requires algorithms with local learning rules. Artificial neural networks using local learning rules to perform principal subspace analysis (PSA) and clustering have recently been derived from principled objective functions. However, no biologically plausible networks exist for minor subspace analysis (MSA), a fundamental signal processing task. MSA extracts the lowest-variance subspace of the input signal covariance matrix. Here, we introduce a novel similarity matching objective for extracting the minor subspace, Minor Subspace Similarity Matching (MSSM). Moreover, we derive an adaptive MSSM algorithm that naturally maps onto a novel neural network with local learning rules and gives numerical results showing that our method converges at a competitive rate.

The core collapse of rapidly rotating massive ~10 Msun stars ("collapsars"), and resulting formation of hyper-accreting black holes, are a leading model for the central engines of long-duration gamma-ray bursts (GRB) and promising sources of r-process nucleosynthesis. Here, we explore the signatures of collapsars from progenitors with extremely massive helium cores >130 Msun above the pair-instability mass gap. While rapid collapse to a black hole likely precludes a prompt explosion in these systems, we demonstrate that disk outflows can generate a large quantity (up to >50 Msun) of ejecta, comprised of >5-10 Msun in r-process elements and ~0.1-1 Msun of 56Ni, expanding at velocities ~0.1c. Radioactive heating of the disk-wind ejecta powers an optical/infrared transient, with a characteristic luminosity ∼1042 erg s−1 and spectral peak in the near-infrared (due to the high optical/UV opacities of lanthanide elements) similar to kilonovae from neutron star mergers, but with longer durations ≳ 1 month. These "super-kilonovae" (superKNe) herald the birth of massive black holes >60 Msun, which, as a result of disk wind mass-loss, can populate the pair-instability mass gap 'from above' and could potentially create the binary components of GW190521. SuperKNe could be discovered via wide-field surveys such as those planned with the Roman Space Telescope or via late-time infrared follow-up observations of extremely energetic GRBs. Gravitational waves of frequency ~0.1-50 Hz from non-axisymmetric instabilities in self-gravitating massive collapsar disks are potentially detectable by proposed third-generation intermediate and high-frequency observatories at distances up to hundreds of Mpc; in contrast to the "chirp" from binary mergers, the collapsar gravitational-wave signal decreases in frequency as the disk radius grows ("sad trombone").

We perform 2D and 3D kinetic simulations of reconnection-mediated turbulent flares in a magnetized electron-positron plasma, with weak and strong radiative cooling. Such flares can be generated around neutron stars and accreting black holes. We focus on the magnetically dominated regime where tension of the background magnetic field lines exceeds the plasma rest-mass density by a factor σ0 > 1. In the simulations, turbulence is excited on a macroscopic scale l0, and we observe that it develops by forming thin, dynamic current sheets on various scales. The deposited macroscopic energy dissipates by energizing thermal and nonthermal particles. The particle energy distribution is shaped by impulsive acceleration in reconnecting current sheets, gradual stochastic acceleration, and radiative losses. We parameterize radiative cooling by the ratio ${ \mathcal A }$ of light-crossing time l0/c to a cooling timescale, and study the effect of increasing ${ \mathcal A }$ on the flare. When radiative losses are sufficiently weak, ${ \mathcal A }\lt {\sigma }_{0}^{-1}$, the produced emission is dominated by stochastically accelerated particles, and the radiative power depends logarithmically on ${ \mathcal A }$. The resulting radiation spectrum of the flare is broad and anisotropic. In the strong-cooling regime, ${ \mathcal A }\gt {\sigma }_{0}^{-1}$, stochastic acceleration is suppressed, while impulsive acceleration in the current sheets continues to operate. As ${ \mathcal A }$ increases further, the emission becomes dominated by thermal particles. Our simulations offer a new tool to study particle acceleration by turbulence, especially at high energies, where cooling competes with acceleration. We find that the particle distribution is influenced by strong intermittency of dissipation, and stochastic acceleration cannot be described by a universal diffusion coefficient.

Numerical methods for approximately solving partial differential equations (PDE) are at the core of scientific computing. Often, this requires high-resolution or adaptive discretization grids to capture relevant spatio-temporal features in the PDE solution, e.g., in applications like turbulence, combustion, and shock propagation. Numerical approximation also requires knowing the PDE in order to construct problem-specific discretizations. Systematically deriving such solution-adaptive discrete operators, however, is a current challenge. Here we present an artificial neural network architecture for data-driven learning of problemand resolution-specific local discretizations of nonlinear PDEs. Our proposed method achieves numerically stable discretization of the operators in an unknown nonlinear PDE by spatially and temporally adaptive parametric pooling on regular Cartesian grids, and by incorporating knowledge about discrete time integration. Knowing the actual PDE is not necessary, as solution data is sufficient to train the network to learn the discrete operators. A once-trained neural architecture model can be used to predict solutions of the PDE on larger spatial domains and for longer times than it was trained for, hence addressing the problem of PDE-constrained extrapolation from data. We present demonstrative examples on long-term forecasting of hard numerical problems including equation-free forecasting of non-linear dynamics of forced Burgers problem on coarse spatio-temporal grids.

We introduce c-lasso, a Python package that enables sparse and robust linear regression and classification with linear equality constraints. The underlying statistical forward model is assumed to be of the following form: [ y = X β+ σε subject to Cβ=0 ] Here, X ∈ℝ