Publications

It has been reported that upon doping a Mott insulator, there can be a crossover to a pseudogaped metallic phase followed by a first-order transition to another thermodynamically stable metallic phase. We call this first-order metal-metal transition the Sordi transition. It was argued that the initial reports of Sordi transitions at finite temperature could be explained by finite size effects and biases related to the model and method used. In this work, we report the Sordi transition on larger clusters at finite temperature on a triangular lattice, where long-range antiferromagnetic fluctuations are frustrated, using a different method, the dynamical cluster approximation instead of the cellular dynamical mean-field theory. This demonstrates that this first-order transition is a directly observable transition in doped Mott insulators and that it is relevant for experiments on candidate spin-liquid organic materials.

In magic angle twisted bilayer graphene, transport, thermodynamic and spectroscopic experiments pinpoint at a competition between distinct low-energy states with and without electronic order. We use Dynamical Mean Field Theory (DMFT) on the topological heavy Fermion (THF) model of twisted bilayer graphene to investigate the emergence of electronic correlations and long-range order in the absence of strain. We contrast moment formation, Kondo screening and ordering on a temperature basis and explain the nature of emergent correlated states based on three central phenomena: (i) the formation of local spin and valley isospin moments around 100K, (ii) the ordering of the local isospin moments around 10K preempting Kondo screening, and (iii) a cascadic redistribution of charge between localized and delocalized electronic states upon doping. At integer fillings, we find that low energy spectral weight is depleted in the symmetric phase, while we find insulating states with gaps enhanced by exchange coupling in the zero-strain ordered phases. Doping away from integer filling results in distinct metallic states: a "bad metal" above the ordering temperature, where scattering off the disordered local moments suppresses electronic coherence, and a "good metal" in the ordered states with coherence of quasiparticles facilitated by isospin order. This finding reveals coherence from order as the microscopic mechanism behind the Pomeranchuk effect observed experimentally. Upon doping, there is a periodic charge reshuffling between localized and delocalized electronic orbitals leading to cascades of doping-induced Lifshitz transitions, local spectral weight redistributions and periodic variations of the electronic compressibility. Our findings provide a unified understanding of the most puzzling aspects of scanning tunneling spectroscopy, transport, and compressibility experiments.

We investigate the application of matrix product state (MPS) representations of the influence functionals (IFs) for the calculation of real-time equilibrium correlation functions in open quantum systems. Focusing specifically on the unbiased spin-boson model, we explore the use of IF-MPSs for complex time propagation, as well as IF-MPSs for constructing correlation functions in the steady state. We examine three different IF approaches: one based on the Kadanoff–Baym contour targeting correlation functions at all times, one based on a complex contour targeting the correlation function at a single time, and a steady state formulation, which avoids imaginary or complex times, while providing access to correlation functions at all times. We show that within the IF language, the steady state formulation provides a powerful approach to evaluate equilibrium correlation functions.

Behavior changes over the course of learning a task. This behavioral change is due to shifts in neural responses that support improved performance. Here, we investigated how the underlying representational geometries in visual areas V4 and IT of the Macaque visual system change during a categorization learning task. Visual stimuli varied in two independent attributes, and monkeys learned to categorize them based on a category boundary in the stimulus space that was defined by a combination of the attributes. Chronic neural population recordings were obtained from V4 and IT over multiple days of training while a monkey learned the task through receiving correct/incorrect feedback. Additionally, we recorded from the same neural populations while a monkey performed a fixation task viewing the same sets of stimuli. In all eight analyzed tasks, the monkey’s performance on the categorization task improved with training. To link this behavioral improvement to the underlying population responses, we investigated how the geometry of neural population activity changed over the course of learning. We treated population responses to all stimuli in each of the two categories as manifold-like representations, and analyzed the geometric properties of these representations using mean-field theoretic manifold capacity analysis. As the monkey learned the task, we observed that the representations in both V4 and IT for the two classes became more separable as measured by an increase in manifold capacity. This increase in capacity was associated with a characteristic geometric change in the neural population response geometry. Our results suggest that both V4 and IT responses actively change during category learning in ways that directly lead to increased separability and improved readouts for downstream neural areas, and point towards future work linking these population-level geometric changes to local changes at the single-neuron level.

All organisms make temporal predictions, and their evolutionary fitness generally depends on the accuracy of these predictions. Understanding what structure enables computing temporal predictions in complex natural scenarios is central to computational neuroscience and machine learning. This thesis focuses on visual processing and describes a unified approach for the representation and estimation of dynamic visual signals, as computed with neural elements in brains or machines. We propose that temporal prediction can serve as a general objective function, both for unsupervised learning of visual representations and for estimat- ing probable future frames under uncertainty. Optimizing for next-frame prediction leverages the order of time, especially visual motion, and extracts predictive information from image sequences, without requiring labeled data. The architecture of a predictive system plays a critical role and we hypothesize that it should reflect the symmetry properties of the physical world. Specifically, a model that discovers the transformations acting in a visual scene should exploit these transformations to predict accurately and should remain agnostic when these transformations are ambiguous. Such interpretable models can serve as guides to explain the responses of neurons in sensory cortices and their functional role in visual perception.
We first describe the empirical structure of dynamic visual scenes and then develop a mathematical theory for exploiting that structure. The movement of observers and objects creates distinct temporal structures in visual signals, allowing for partial prediction of future signals based on past ones. Motivated by group representation theory, we propose a method to discover and utilize the transformation structures of image sequences and show that local phase measurements play a fundamental role. The proposed model extrapolates visual signals in a local polar representation, this representation is learned via next-frame prediction. This polar prediction model successfully recovers simple transformations in synthetic datasets and scales to natural image sequences. The architecture is simple yet effective: it contains a single hidden stage with one non-linearity that factorizes slow form and steady motion signals. We demonstrate that polar prediction achieves better prediction performance than traditional approaches based on motion compensation, and that it rivals conventional deep networks trained on prediction.

We then confront the inherent uncertainty of visual temporal prediction and develop a framework for learning and sampling the conditional density of the next frame given the past few observed frames. Casting prediction as a probabilistic inference problem is motivated by the need to cope with ambiguity in natural image sequences. We describe a regression- based framework that implicitly estimates the distribution of the next frame, effectively learning a conditional image density from high-dimensional signals. This is achieved with a simple resilience-to-noise objective function: a deep neural network is trained to map to past conditioning frames and a noisy observation of the next frame to an estimated denoised next frame. The network is trained over a range of noise levels without access to that noise level, i.e., it is blind and universal. This denoising objective has the desirable property of being local in the space of densities, and training across noise levels forces the network to extract information about the stable underlying distribution of probable next frame given past conditioning. We consider synthetic image sequences composed of moving disks that occlude each other and demonstrate that trained networks can handle challenging cases of bifurcating temporal trajectories -- effectively choosing one occlusion or another when the observation is ambiguous. Furthermore, local linear analysis of a network trained on natural image sequences reveals that the model automatically weights evidence by reliability: the model integrates information from past conditioning and noisy observation, adapting to the amount of predictive information in the conditioning, and the noise level in the observation.

Finally, we discuss the implications of this work for understanding biological vision. Starting from the polar prediction model, we derive a circuit algorithm composed of local neural computations for predicting natural image sequences. The circuit is composed of canonical computational elements that have received ample physiological evidence: its components resemble the normalized simple and direction-selective complex cell models of primate V1 neurons. Unlike the polar prediction model, this circuit algorithm does not impose a polar factorization, instead, it lets complex-cell-like units learn a combination of quadratic responses. Furthermore, we outline a method for gradually extracting slower and more abstract features by cascading this biologically plausible mechanism. These models offer a normative framework for understanding how the visual system represents sensory inputs in a form that simplifies temporal prediction. Together, our work on visual temporal prediction builds connections between computational modeling and brain sciences. These connections can guide the design and analysis of physiological and perceptual experiments, and can also motivate further developments in machine learning.

Many problems in image processing and computer vision rely, explicitly or implicitly, on statistical density models. Describing the full density of natural images, 𝑝(𝑥), is a daunting problem given the dimensionality of the signal space. Traditionally, models have been developed by combining assumed symmetry properties, with simple parametric forms, often within pre- specified transformed coordinate systems. While these models have led to steady advances in problems such as denoising, they are too simplistic to generate complex features that occur in our visual world.
Deep neural networks have provided state-of-the-art solutions for problems such as denoising, which implicitly rely on a prior probability model of natural images. Here, we first develop a robust and general methodology for extracting the prior. We rely on a statistical result due to Tweedie (1956) and Miyasawa (1961), who showed that the least-squares solution for removing additive Gaussian noise can be written directly in terms of the gradient of the log of the noisy signal density. We use this fact to develop a stochastic coarse-to-fine gradient ascent procedure for drawing high-probability samples from the implicit prior embedded within a neural network trained to perform blind (i.e., unknown noise level) least-squares denoising. This algorithm is similar to score-based diffusion framework, yet different in several ways.

Unlike the classical framework, we do not have direct access to the learned density, which gives rise to a crucial question: what is the prior? The rest of the thesis focuses on understanding and using this prior.

At the core of our coarse-to-fine gradient ascent sampling algorithm is a deep neural network (DNN) denoiser. Despite their success, we lack an understanding of the DNN denoiser mechanisms and more importantly what priors are being learned by these models. In order to make the DNN denoiser interpretable, we remove all network biases (i.e. additive constants), to enforce the denoising mapping to become locally linear. This architecture lends itself to local linear algebraic analysis through the Jacobian of the denoising map, which provides a high level interpretability. A desired side effect of locally linear models is that they generalize automatically across noise levels.

Next, we study the continuity of the implicit image prior. We design an experiment to investigate whether the prior interpolates between the training examples or consists of a discrete set of delta functions corresponding to a memorized set of training examples. We find that for small datasets, the latter is the case. But with large enough datasets, the network generalizes beyond training examples, evidenced by high quality novel generated samples. Surprisingly, we observe that, for large enough datasets, two models trained on non-overlapping subsets of a dataset learn nearly the same density. We analyze the learned denoising functions and show that the inductive biases give rise to a shrinkage operation in a basis adapted to the underlying image. Examination of these bases reveals oscillating harmonic structures along contours and in homogeneous regions. We demonstrate that trained denoisers are inductively biased towards these geometry-adaptive harmonic bases.

Having established that a DNN denoiser can generalize, we employ the learned image density to study the question of low-dimensionality of image priors. The goal is to exploit image properties to factorize the density into low dimensional densities, thereby reducing the number of parameters and training examples. To this end, we develop a low-dimensional probability model for images decomposed into multi-scale wavelet sub-bands. The image probability distribution is factorized as a product of conditional probabilities of its wavelet coefficients conditioned by coarser scale coefficients. We assume that these conditional probabilities are local and stationary, and hence can be captured with low-dimensional Markov models. Each conditional score can thus be estimated with a conditional CNN (cCNN) with a small receptive field (RF). The effective size of Markov neighborhoods (i.e. the size w.r.t to the grid size) grows from fine to coarser scales. The score of the coarse-scale low-pass band (a low-resolution version of the image) is modeled using a CNN with a global RF, enabling representation of large-scale image structures and organization. We evaluate our model and show that locality and stationarity assumptions hold for conditional RF sizes as small as 9 × 9 without harming performance. Thus, high-dimensional score estimation for images can be reduced to low-dimensional Markov conditional models, alleviating the curse of dimensionality.

Finally, we put the denoiser prior into use. A generalization of the coarse-to-fine gradient ascent sampling algorithm to constrained sampling provides a method for using the implicit prior to solve any linear inverse problem, with no additional training. We demonstrate the generality of the algorithm by using it to produce high-quality solutions in multiple applications, such as deblurring, colorization, compressive sensing, and super resolution.

We introduce OpenRAND, a C++17 library aimed at facilitating reproducible scientific research by generating statistically robust yet replicable random numbers in as little as two lines of code, overcoming some of the unnecessary complexities of existing RNG libraries. OpenRAND accommodates single and multi-threaded applications on CPUs and GPUs and offers a simplified, user-friendly API that complies with the C++ standard’s random number engine interface. It is lightweight; provided as a portable, header-only library. It is statistically robust: a suite of built-in tests ensures no pattern exists within single or multiple streams. Despite its simplicity and portability, it remains performant—matching and sometimes outperforming native libraries. Our tests, including a Brownian walk simulation, affirm its reproducibility and ease-of-use while highlight its computational efficiency, outperforming CUDA’s cuRAND by up to 1.8 times.

Notwithstanding the evidence against them, classical variational phase-field models continue to be used and pursued in an attempt to describe fracture nucleation in elastic brittle materials. In this context, the main objective of this paper is to provide a comprehensive review of the existing evidence against such a class of models as descriptors of fracture nucleation. To that end, a review is first given of the plethora of experimental observations of fracture nucleation in nominally elastic brittle materials under quasi-static loading conditions, as well as of classical variational phase-field models, without and with energy splits. These models are then confronted with the experimental observations. The conclusion is that they cannot possibly describe fracture nucleation in general. This because classical variational phase-field models cannot account for material strength as an independent macroscopic material property. The last part of the paper includes a brief summary of a class of phase-field models that can describe fracture nucleation. It also provides a discussion of how pervasively material strength has been overlooked in the analysis of fracture at large, as well as an outlook into the modeling of fracture nucleation beyond the basic setting of elastic brittle materials.

The motion of rigid particles in complex fluids is ubiquitous in natural and industrial processes. The most popular toy model for understanding the physics of such systems is the settling of a solid sphere in a viscoelastic fluid. There is general agreement that an elastic wake develops downstream of the sphere, causing the breakage of fore-and-aft symmetry, while the flow remains axisymmetric, independent of fluid viscoelasticity and flow conditions. Using a continuum mechanics model, we reveal that axisymmetry holds only for weak viscoelastic flows. Beyond a critical value of the settling velocity, steady, non-axisymmetric disturbances develop peripherally of the rear pole of the sphere, giving rise to a four-lobed fingering instability. The transition from axisymmetric to non-axisymmetric flow fields is characterized by a regular bifurcation and depends solely on the interplay between shear and extensional properties of the viscoelastic fluid under different flow regimes. At higher settling velocities, each lobe tip is split into two new lobes, resembling fractal fingering in interfacial flows. For the first time, we capture an elastic fingering instability under steady-state conditions, and provide the missing information for understanding and predicting such instabilities in the response of viscoelastic fluids and soft media.

We present a deterministic algorithm for the efficient evaluation of imaginary time diagrams based on the recently introduced discrete Lehmann representation (DLR) of imaginary time Green's functions. In addition to the efficient discretization of diagrammatic integrals afforded by its approximation properties, the DLR basis is separable in imaginary time, allowing us to decompose diagrams into linear combinations of nested sequences of one-dimensional products and convolutions. Focusing on the strong coupling bold-line expansion of generalized Anderson impurity models, we show that our strategy reduces the computational complexity of evaluating an $M$th-order diagram at inverse temperature $\beta$ and spectral width $\omega_{\max}$ from $\mathcal{O}((\beta \omega_{\max})^{2M-1})$ for a direct quadrature to $\mathcal{O}(M (\log (\beta \omega_{\max}))^{M+1})$, with controllable high-order accuracy. We benchmark our algorithm using third-order expansions for multi-band impurity problems with off-diagonal hybridization and spin-orbit coupling, presenting comparisons with exact diagonalization and quantum Monte Carlo approaches. In particular, we perform a self-consistent dynamical mean-field theory calculation for a three-band Hubbard model with strong spin-orbit coupling representing a minimal model of Ca$_2$RuO$_4$, demonstrating the promise of the method for modeling realistic strongly correlated multi-band materials. For both strong and weak coupling expansions of low and intermediate order, in which diagrams can be enumerated, our method provides an efficient, straightforward, and robust black-box evaluation procedure. In this sense, it fills a gap between diagrammatic approximations of the lowest order, which are simple and inexpensive but inaccurate, and those based on Monte Carlo sampling of high-order diagrams.