Publications

We explore the nonlinear response of ultrafast strong-field driven excitons in a one-dimensional solid with ab initio simulations. We demonstrate from our simulations and analytical model that a finite population of excitons imprints unique signatures to the high-harmonic spectra of materials. We show the exciton population can be retrieved from the spectra. We further demonstrate signatures of exciton recombination and that a shift of the exciton level is imprinted into the harmonic signal. The results open the door to high-harmonic spectroscopy of excitons in condensed-matter systems.

The prediction and realization of the quantum anomalous Hall effect are often intimately connected to honeycomb lattices in which the sublattice degree of freedom plays a central role in the nontrivial topology. Two-dimensional Wigner crystals, on the other hand, form triangular lattices without sublattice degrees of freedom, resulting in a topologically trivial state. In this Letter, we discuss the possibility of spontaneously formed honeycomb-lattice crystals that exhibit the quantum anomalous Hall effect. Starting from a single-band system with nontrivial quantum geometry, we derive the mean-field energy functional of a class of crystal states and express it as a model of sublattice pseudospins in momentum space. We find that nontrivial quantum geometry leads to extra terms in the pseudospin model that break an effective `time-reversal symmetry' and favor a topologically nontrivial pseudospin texture. When the effects of these extra terms dominate over the ferromagnetic exchange coupling between pseudospins, the anomalous Hall crystal state becomes energetically favorable over the trivial Wigner crystal state.

The two-dimensional electron gas is of fundamental importance in quantum many-body physics. We study a minimal extension of this model with C4 (as opposed to full rotational) symmetry and an electronic dispersion with two valleys with anisotropic effective masses. Electrons in our model interact via Coulomb repulsion, screened by distant metallic gates. Using variational Monte Carlo simulations, we find a broad intermediate range of densities with a metallic valley-polarized, spin-unpolarized ground state. Our results are of direct relevance to the recently discovered “nematic” state in AlAs quantum wells. For the effective mass anisotropy relevant to this system, mx/my ≈5.2, we obtain a transition from an anisotropic metal to a valley-polarized metal at rs ≈12 (where rs is the dimensionless Wigner-Seitz radius). At still lower densities, we find a (possibly metastable) valley and spin-polarized state with a reduced electronic anisotropy.

Transformers have revolutionized machine learning across diverse domains, yet understanding their behavior remains crucial, particularly in high-stakes applications. This paper introduces the contextual counting task, a novel toy problem aimed at enhancing our understanding of Transformers in quantitative and scientific contexts. This task requires precise localization and computation within datasets, akin to object detection or region-based scientific analysis. We present theoretical and empirical analysis using both causal and non-causal Transformer architectures, investigating the influence of various positional encodings on performance and interpretability. In particular, we find that causal attention is much better suited for the task, and that no positional embeddings lead to the best accuracy, though rotary embeddings are competitive and easier to train. We also show that out of distribution performance is tightly linked to which tokens it uses as a bias term.

In applied statistics and machine learning, the "gold standards" used for training are often biased and almost always noisy. Dawid and Skene's justifiably popular crowdsourcing model adjusts for rater (coder, annotator) sensitivity and specificity, but fails to capture distributional properties of rating data gathered for training, which in turn biases training. In this study, we introduce a general purpose measurement-error model with which we can infer consensus categories by adding item-level effects for difficulty, discriminativeness, and guessability. We further show how to constrain the bimodal posterior of these models to avoid (or if necessary, allow) adversarial raters. We validate our model's goodness of fit with posterior predictive checks, the Bayesian analogue of χ

Neurosift, a browser-based visualization tool, is designed for the interactive exploration of Neurodata Without Borders (NWB) files, whether stored locally, on remote servers, or within the Distributed Archives for Neurophysiology Data Integration (DANDI). NWB (Rübel et al., 2022; Teeters et al., 2015) is an open data standard for neurophysiology that enables the sharing, archiving, and analysis of various types of neurophysiology data. DANDI (Rübel et al., 2022) is a cloud-based platform that supports the storage, sharing, and analysis of neurophysiology data including NWB files. With Neurosift integration, users browsing DANDI can easily open any NWB file in the browser and explore its contents, including timeseries data, images, and more. Neurosift can also be used to browse the DANDI database or individual Dandisets. Overall, Neurosift simplifies the visualization and exploration of complex NWB file structures, making it a valuable tool for neuroscientists.

Bayesian Neural Nets are proposed as flexible models that can provide calibrated uncertainty estimates for out-of-distribution data. Due to the high dimensionality of BNN posteriors and the intractability of exact inference, numerous approximate inference techniques have been proposed. However, issues persist. Some approaches lack a proper Bayesian formulation while others result in inexpressive or uncalibrated posteriors, defeating the primary purpose of BNNs. Recently, subspace inference has been proposed to overcome these challenges by running the inference on a lower-dimensional subspace of network parameters. While achieving promising results, these methods are mathematically involved and therefore extending them to general architectures and problems is challenging. Here, we propose a new subspace inference method---called HybridBNN---that divides the network weights into deterministic and stochastic subsets before training. We develop an expectation-maximization algorithm for the joint inference of the posterior over the stochastic weights as well as the optimization of the deterministic ones. HybridBNN achieves competitive prediction and calibration performance on two regression and classification toy datasets and a benchmark dataset for in and out-of-domain distributions. The simplicity and flexibility of HybridBNN make it a favorable candidate for developing generic calibrated models.

It is often observed that stochastic gradient descent (SGD) and its variants implicitly select a solution with good generalization performance; such implicit bias is often characterized in terms of the sharpness of the minima. Kleinberg et al. (2018) connected this bias with the smoothing effect of SGD which eliminates sharp local minima by the convolution using the stochastic gradient noise. We follow this line of research and study the commonly-used averaged SGD algorithm, which has been empirically observed in Izmailov et al. (2018) to prefer a flat minimum and therefore achieves better generalization. We prove that in certain problem settings, averaged SGD can efficiently optimize the smoothed objective which avoids sharp local minima. In experiments, we verify our theory and show that parameter averaging with an appropriate step size indeed leads to significant improvement in the performance of SGD.

High throughput gene expression profiling measures individual gene expression across conditions. However, genes are regulated in complex networks, not as individual entities, limiting the interpretability of gene expression data. Machine learning models that incorporate prior biological knowledge are a powerful tool to extract meaningful biology from gene expression data. Pathway-level information extractor (PLIER) is an unsupervised machine learning method that defines biological pathways by leveraging the vast amount of published transcriptomic data. PLIER converts gene expression data into known pathway gene sets, termed latent variables (LVs), to substantially reduce data dimensionality and improve interpretability. In the current study, we trained the first mouse PLIER model on 190,111 mouse brain RNA-sequencing samples, the greatest amount of training data ever used by PLIER. We then validated the mousiPLIER approach in a study of microglia and astrocyte gene expression across mouse brain aging. mousiPLIER identified biological pathways that are significantly associated with aging, including one latent variable (LV41) corresponding to striatal signal. To gain further insight into the genes contained in LV41, we performed k-means clustering on the training data to identify studies that respond strongly to LV41. We found that the variable was relevant to striatum and aging across the scientific literature. Finally, we built a web server (http://mousiplier.greenelab.com/) for users to easily explore the learned latent variables. Taken together this study defines mousiPLIER as a method to uncover meaningful biological processes in mouse brain transcriptomic studies.

We examine the problem of selecting a small set of linear measurements for reconstructing high-dimensional signals. Well-established methods for optimizing such measurements include principal component analysis (PCA), independent component analysis (ICA) and compressed sensing (CS) based on random projections, all of which rely on axis- or subspace-aligned statistical characterization of the signal source. However, many naturally occurring signals, including photographic images, contain richer statistical structure. To exploit such structure, we introduce a general method for obtaining an optimized set of linear measurements for efficient image reconstruction, where the signal statistics are expressed by the prior implicit in a neural network trained to perform denoising (known as a ``diffusion model''). We demonstrate that the optimal measurements derived for two natural image datasets differ from those of PCA, ICA, or CS, and result in substantially lower mean squared reconstruction error. Interestingly, the marginal distributions of the measurement values are asymmetrical (skewed), substantially more so than those of previous methods. We also find that optimizing with respect to perceptual loss, as quantified by structural similarity (SSIM), leads to measurements different from those obtained when optimizing for MSE. Our results highlight the importance of incorporating the specific statistical regularities of natural signals when designing effective linear measurements.