Publications

In clinical In-Vitro Fertilization (IVF), identifying the most viable embryo for transfer is important to increasing the likelihood of a successful pregnancy. Traditionally, this process involves embryologists manually assessing embryos’ static morphological features at specific intervals using light microscopy. This manual evaluation is not only time-intensive and costly, due to the need for expert analysis, but also inherently subjective, leading to variability in the selection process. To address these challenges, we develop a multimodal model that leverages both time-lapse video data and Electronic Health Records (EHRs) to predict embryo viability. A key challenge of our research is to effectively combine time-lapse video and EHR data, given their distinct modality characteristic. We comprehensively analyze our multimodal model with various modality inputs and integration approaches. Our approach will enable fast and automated embryo viability predictions in scale for clinical IVF.

We introduce cppdlr, a C++ library implementing the discrete Lehmann representation (DLR) of functions in imaginary time and Matsubara frequency, such as Green's functions and self-energies. The DLR is based on a low-rank approximation of the analytic continuation kernel, and yields a compact and explicit basis consisting of exponentials in imaginary time and simple poles in Matsubara frequency. cppdlr constructs the DLR basis and associated interpolation grids, and implements standard operations. It provides a flexible yet high-level interface, facilitating the incorporation of the DLR into both small-scale applications and existing large-scale software projects.

Volume electron microscopy (VEM) is an essential tool for studying biological structures. Due to the challenges of sample preparation and continuous volumetric imaging, image artifacts are almost inevitable. Such image artifacts complicate further processing both for automated computer vision methods and human experts. Unfortunately, the widely used contrast limited adaptive histogram equalization (CLAHE) can alter the essential relative contrast information about some biological structures. We developed an image-processing pipeline to remove the artifacts and enhance the images without CLAHE. We apply our method to VEM datasets of a Microwasp head. We demonstrate that our method restores the images with high fidelity while preserving the original relative contrast. This pipeline is adaptable to other VEM datasets.

The visual world is richly adorned with texture, which can serve to delineate important elements of natural scenes. In anesthetized macaque monkeys, selectivity for the statistical features of natural texture is weak in V1, but substantial in V2, suggesting that neuronal activity in V2 might directly support texture perception. To test this, we investigated the relation between single cell activity in macaque V1 and V2 and simultaneously measured behavioral judgments of texture. We generated stimuli along a continuum between naturalistic texture and phase-randomized noise and trained two macaque monkeys to judge whether a sample texture more closely resembled one or the other extreme. Analysis of responses revealed that individual V1 and V2 neurons carried much less information about texture naturalness than behavioral reports. However, the sensitivity of V2 neurons, especially those preferring naturalistic textures, was significantly closer to that of behavior compared with V1. The firing of both V1 and V2 neurons predicted perceptual choices in response to repeated presentations of the same ambiguous stimulus in one monkey, despite low individual neural sensitivity. However, neither population predicted choice in the second monkey. We conclude that neural responses supporting texture perception likely continue to develop downstream of V2. Further, combined with neural data recorded while the same two monkeys performed an orientation discrimination task, our results demonstrate that choice-correlated neural activity in early sensory cortex is unstable across observers and tasks, untethered from neuronal sensitivity, and therefore unlikely to directly reflect the formation of perceptual decisions.

To facilitate single-cell multi-omics analysis and improve reproducibility, we present single-cell pipeline for end-to-end data integration (SPEEDI), a fully automated end-to-end framework for batch inference, data integration, and cell-type labeling. SPEEDI introduces data-driven batch inference and transforms the often heterogeneous data matrices obtained from different samples into a uniformly annotated and integrated dataset. Without requiring user input, it automatically selects parameters and executes pre-processing, sample integration, and cell-type mapping. It can also perform downstream analyses of differential signals between treatment conditions and gene functional modules. SPEEDI’s data-driven batch-inference method works with widely used integration and cell-typing tools. By developing data-driven batch inference, providing full end-to-end automation, and eliminating parameter selection, SPEEDI improves reproducibility and lowers the barrier to obtaining biological insight from these valuable single-cell datasets. The SPEEDI interactive web application can be accessed at https://speedi.princeton.edu/. A record of this paper’s transparent peer review process is included in the supplemental information.

Score diffusion methods can learn probability densities from samples. The score of the noise-corrupted density is estimated using a deep neural network, which is then used to iteratively transport a Gaussian white noise density to a target density. Variants for conditional densities have been developed, but correct estimation of the corresponding scores is difficult. We avoid these difficulties by introducing an algorithm that guides the diffusion with a projected score. The projection pushes the image feature vector towards the feature vector centroid of the target class. The projected score and the feature vectors are learned by the same network. Specifically, the image feature vector is defined as the spatial averages of the channels activations in select layers of the network. Optimizing the projected score for denoising loss encourages image feature vectors of each class to cluster around their centroids. It also leads to the separations of the centroids. We show that these centroids provide a low-dimensional Euclidean embedding of the class conditional densities. We demonstrate that the algorithm can generate high quality and diverse samples from the conditioning class. Conditional generation can be performed using feature vectors interpolated between those of the training set, demonstrating out-of-distribution generalization.

Given an intractable target density p, variational inference (VI) attempts to find the best approximation q from a tractable family Q. This is typically done by minimizing the exclusive Kullback-Leibler divergence, KL(q||p). In practice, Q is not rich enough to contain p, and the approximation is misspecified even when it is a unique global minimizer of KL(q||p). In this paper, we analyze the robustness of VI to these misspecifications when p exhibits certain symmetries and Q is a location-scale family that shares these symmetries. We prove strong guarantees for VI not only under mild regularity conditions but also in the face of severe misspecifications. Namely, we show that (i) VI recovers the mean of p when p exhibits an \textit{even} symmetry, and (ii) it recovers the correlation matrix of p when in addition~p exhibits an \textit{elliptical} symmetry. These guarantees hold for the mean even when q is factorized and p is not, and for the correlation matrix even when~q and~p behave differently in their tails. We analyze various regimes of Bayesian inference where these symmetries are useful idealizations, and we also investigate experimentally how VI behaves in their absence.

Cryogenic electron microscopy (cryo-EM) has emerged as a powerful method for resolving the atomistic details of cellular components. In recent years, several computational methods have been developed to study the heterogeneity of molecules in single-particle cryo-EM. In this study, we analyzed a publicly available single-particle dataset of TRPV1 using five of these methods: 3D Flexible Refinement, 3D Variability Analysis, cryoDRGN, ManifoldEM, and Bayesian ensemble reweighting. Beyond what we initially expected, we have found that this dataset contains significant heterogeneity— indicating that single particle datasets likely contain a rich spectrum of biologically relevant states. Further, we have found that different methods are best suited to studying different kinds of heterogeneity, with some methods being more sensitive to either compositional or conformational heterogeneity. We also apply a combination of Bayesian ensemble reweighting and molecular dynamics as supporting evidence for the presence of these rarer states within the sample. Finally, we developed a quantitative metric based on the analysis of the singular value decomposition and power spectra to compare the resulting volumes from each method. This work represents a detailed view of the variable outcomes of different heterogeneity methods used to analyze a single real dataset and presents a pathway to a deeper understanding of the biology of complex macromolecules like the TRPV1 ion channel.

We propose a framework for comparing a set of image representations (artificial or biological) in terms of their sensitivities to local distortions. We quantify the local geometry of a representation using the Fisher information matrix (FIM), a standard statistical tool for characterizing the sensitivity to local distortions of a stimulus, and use this as a substrate for a metric on the local geometry of representations in the vicinity of a base image. This metric may then be used to optimally differentiate a set of models, by optimizing for a pair of distortions that maximize the variance of the models under this metric. We use the framework to compare a set of simple models of the early visual system, identifying a novel set of image distortions that allow immediate comparison of the models by visual inspection. In a second example, we show that the method can reveal distinctions between standard and adversarially trained object recognition networks.

Centered kernel alignment (CKA) and representational similarity analysis (RSA) of dissimilarity matrices are two popular methods for comparing neural systems in terms of representational geometry. Although they follow a conceptually similar approach, typical implementations of CKA and RSA tend to result in numerically different outcomes. Here, I show that these two approaches are largely equivalent once one incorporates a mean-centering step into RSA. This equivalence holds for both linear and nonlinear variants of these methods. These connections are simple to derive, but appear to have been thus far overlooked in the context of comparing neural representations. By unifying these measures, this paper hopes to simplify a complex and fragmented literature on this subject.Competing Interest StatementThe authors have declared no competing interest.