CCN: Publications

Clear Filters

Filter by Group

Computational Vision

Neural Circuits and Algorithms

NeuroAI and Geometric Data Analysis

NeuroRSE

Statistical Analysis of Neural Data

220 Publications

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration

James Ryu, A. Nejatbakhsh, Mahdi Torkashvand, Sahana Gangadharan, Maedeh Seyedolmohadesin, Jinmahn Kim, L. Paninski, Vivek Venkatachalam

bioRxiv:2022.07.18.500485v1

PLOS Computational Biology, 20: 1*5

Export Citation

May 20, 2024

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration

"@article{Nejatbakhsh_biorxiv:2022.07.18.500485_PLOSCB_2024,
title = {{Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration}},
author = {Ryu, James AND Nejatbakhsh, Amin AND Torkashvand, Mahdi AND Gangadharan, Sahana AND Seyedolmohadesin, Maedeh AND Kim, Jinmahn AND Paninski, Liam AND Venkatachalam, Vivek},
acrosscenters = {no},
external = {yes},
abstract = {Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results.},
eprint = {https://www.biorxiv.org/content/10.1101/2022.07.18.500485v1.full},
highlight = {no},
archivePrefix = {biorxiv},
year_posted = {2022},
Journal = {PLOS Computational Biology},
center = {CCN},
OpenAccessURL = {https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012075},
year = {2024},
publisher = {Public Library of Science},
month = {05},
    volume = {20},
    number = {5},
    pages = {1-23},
doi = {10.1371/journal.pcbi.1012075}
}"

Download Citation (BibTex)

A unified framework for perceived magnitude and discriminability of sensory stimuli

J. Zhou, L. Duong, E. P. Simoncelli

bioXriv:2022.04.30.490146v3

Proceedings of the National Academy of Sciences of the United States of America, 121: e2312293121

Export Citation

April 25, 2024

A unified framework for perceived magnitude and discriminability of sensory stimuli

@article{Zhou_Simoncelli_biorxiv:2022.04.30.490146_PNAS_2024,
title = {{A unified framework for perceived magnitude and discriminability of sensory stimuli}},
author = {Zhou,  Jingyang and Duong,  Lyndon R. and Simoncelli,  Eero P.},
abstract = {The perception of sensory attributes is often quantified through measurements of sensitivity (the ability to detect small stimulus changes), as well as through direct judgements of appearance or intensity. Despite their ubiquity, the relationship between these two measurements remains controversial and unresolved. Here, we propose a framework in which they arise from different aspects of a common representation. Specifically, we assume that judgements of stimulus intensity (e.g., as measured through rating scales) reflect the mean value of an internal representation, and sensitivity reflects a combination of mean value and noise properties, as quantified by the statistical measure of Fisher Information. Unique identification of these internal representation properties can be achieved by combining measurements of sensitivity and judgments of intensity. As a central example, we show that Weber{\textquoteright}s law of perceptual sensitivity can co-exist with Stevens{\textquoteright} power-law scaling of intensity ratings (for all exponents), when the noise amplitude increases in proportion to the representational mean. We then extend this result beyond the Weber{\textquoteright}s law range by incorporating a more general and physiology-inspired form of noise, and show that the combination of noise properties and sensitivity measurements accurately predicts intensity ratings across a variety of sensory modalities and attributes. Our framework unifies two primary perceptual measurements {\textendash} thresholds for sensitivity and rating scales for intensity {\textendash} and provides a neural interpretation for the underlying representation.Significance Statement Perceptual measurements of sensitivity to stimulus changes and stimulus appearance (intensity) are ubiquitous in the study of perception. However, the relationship between these two seemingly disparate measurements remains unclear. Proposals for unification have been made for over 60 years, but they generally lack support from perceptual or physiological measurements. Here, we provide a framework that offers a unified interpretation of perceptual sensitivity and intensity measurements, and we demonstrate its consistency with experimental measurements across multiple perceptual domains.Competing Interest StatementThe authors have declared no competing interest.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {https://www.biorxiv.org/content/early/2024/05/29/2022.04.30.490146.full.pdf},
archivePrefix = {bioRxiv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://www.pnas.org/doi/10.1073/pnas.2312293121},
Journal = {Proceedings of the National Academy of Sciences of the United States of America},
volume = {121},
ISSN = {1091-6490},
month = jun,
DOI = {10.1073/pnas.2312293121},
number = {25},
publisher = {Proceedings of the National Academy of Sciences},
year = {2024}
}

Download Citation (BibTex)

Estimating Shape Distances on Neural Representations with Limited Samples

Dean A. Pospisil, B. Larsen, S. Harvey, A. Williams

arXiv:2310.05742

The Twelfth International Conference on Learning Representations, 2024

Export Citation

March 15, 2024

Estimating Shape Distances on Neural Representations with Limited Samples

@article{Pospisil_Larsen_Williams_arxiv:2310.05742_ICLR24_2024,
title = {{Estimating Shape Distances on Neural Representations with Limited Samples}},
author = {Dean A Pospisil and Brett W. Larsen and Sarah E Harvey and Alex H Williams},
abstract = {Measuring geometric similarity between high-dimensional network representations is a topic of longstanding interest to neuroscience and deep learning. Although many methods have been proposed, only a few works have rigorously analyzed their statistical efficiency or quantified estimator uncertainty in data-limited regimes. Here, we derive upper and lower bounds on the worst-case convergence of standard estimators of shape distance—a measure of representational dissimilarity proposed by Williams et al. (2021). These bounds reveal the challenging nature of the problem in high-dimensional feature spaces. To overcome these challenges, we introduce a novel method-of-moments estimator with a tunable bias-variance tradeoff parameterized by an upper bound on bias. We show that this estimator achieves superior performance to standard estimators in simulation and on neural data, particularly in high-dimensional settings. Our theoretical work and estimator thus respectively define and dramatically expand the scope of neural data for which geometric similarity can be accurately measured.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2310.05742},
archivePrefix = {arXiv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://openreview.net/forum?id=kvByNnMERu},
Journal = ICLR24,
year = {2024}
}

Download Citation (BibTex)

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

Rylan Schaeffer, Berivan Isik, Dhruv Bhandarkar Pai, Andres Carranza, Victor Lecomte, Alyssa Unell, Mikail Khona, T. Yerxa, Y. LeCun, S. Chung , Andrey Gromov, Ravid Shwartz-Ziv, Sanmi Koyejo

arXiv:2406.09366

ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024

Export Citation

March 4, 2024

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

@article{Yerxa_Chung_arxiv:2406.09366_2024,
title = {{Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations}},
author = {Rylan Schaeffer and Victor Lecomte and Dhruv Bhandarkar Pai and Andres Carranza and Berivan Isik and Alyssa Unell and Mikail Khona and Thomas Yerxa and Yann LeCun and SueYeon Chung and Andrey Gromov and Ravid Shwartz-Ziv and Sanmi Koyejo},
abstract = {Maximum Manifold Capacity Representations (MMCR) is a recent multi-view self-supervised learning (MVSSL) method that matches or surpasses other leading MVSSL methods. MMCR is intriguing because it does not fit neatly into any of the commonplace MVSSL lineages, instead originating from a statistical mechanical perspective on the linear separability of data manifolds. In this paper, we seek to improve our understanding and our utilization of MMCR. To better understand MMCR, we leverage tools from high dimensional probability to demonstrate that MMCR incentivizes alignment and uniformity of learned embeddings. We then leverage tools from information theory to show that such embeddings maximize a well-known lower bound on mutual information between views, thereby connecting the geometric perspective of MMCR to the information-theoretic perspective commonly discussed in MVSSL. To better utilize MMCR, we mathematically predict and experimentally confirm non-monotonic changes in the pretraining loss akin to double descent but with respect to atypical hyperparameters. We also discover compute scaling laws that enable predicting the pretraining loss as a function of gradients steps, batch size, embedding dimension and number of views. We then show that MMCR, originally applied to image data, is performant on multimodal image-text data. By more deeply understanding the theoretical and empirical behavior of MMCR, our work reveals insights on improving MVSSL methods.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2406.09366},
archivePrefix = {arxiv},
year_posted = {2024},
center = {ccn},
OpenAccessURL = {https://arxiv.org/abs/2406.09366},
Journal = {ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models
(ME-FoMo)},
year = {2024}
}

Download Citation (BibTex)

On the universality of neural encodings in CNNs

F. Guth, Brice Ménard

arXiv:2409.19460

ICLR 2024 Workshop on Representational Alignment

Export Citation

March 2, 2024

On the universality of neural encodings in CNNs

@article{Guth_arxiv:2409.19460_ICLR24_2024,
      title={{On the universality of neural encodings in CNNs}},
      author={\text{Guth, Florentin}} and Brice Ménard},
doi={10.48550/arXiv.2409.19460},
Journal = ICLR24,
abstract={We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.},
      Publisher={ICLR 2024 Workshop on Representational Alignment}, yearposted={2024}, year={2024}, month={Mar},  numpages={13},
      eprint={2409.19460},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2409.19460}, center={CCN}, acrosscenters={no}, external={yes}, highlight={no},
      Open Access Url={https://arxiv.org/abs/2409.19460}
}

Download Citation (BibTex)

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

D. Lipshutz, A. Nejatbakhsh, A. Williams

ICLR 2024 Workshop on Representational Alignment, 2024

Export Citation

March 2, 2024

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

@article{Lipshutz_Williams_ICLR24_2024,
title = {{Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry}},
author = {David Lipshutz AND Amin Nejatbakhsh and Alex H Williams},
abstract = {Uncovering and comparing the dynamical mechanisms that support neural processing remains a key challenge in the analysis of biological and artificial neural systems. However, measures of representational (dis)similarity in neural systems often assume that neural responses are static in time. Here, we show that stochastic shape distances (SSDs; Duong et al., 2023), which were developed to compare noisy neural responses to static inputs and lack an explicit notion of temporal structure, are well equipped to compare noisy dynamics. In two examples, we use SSDs, which interpolate between comparing mean trajectories and secondorder fluctuations about mean trajectories, to disentangle recurrent versus external contributions to noisy dynamics.},
acrosscenters = {no},
external = {no},
highlight = {no},
year_posted = {2024},
center = {CCN},
OpenAccessURL = {https://openreview.net/pdf?id=Fykvxdv2I8},
Journal = {ICLR 2024 Workshop on Representational Alignment},
year = {2024}
}

Download Citation (BibTex)

Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors

C. Chou , Luke Arend, Albert J. Wakhloo, Royoung Kim, Will Slatton, S. Chung

bioRxiv:2024.02.26.582157v1

Export Citation

February 28, 2024

Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors

@article{Chou_Chung_bioRxiv:2024.02.26.582157,
title = {{Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors}},
author = {Chi-Ning Chou and Luke Arend and Albert J. Wakhloo and Royoung Kim and Will Slatton and SueYeon Chung},
year_posted = {2024},
Journal = {bioRxiv e-prints},
center = {CCN},
acrosscenters = {no},
external = {yes},
highlight = {no},
year = {2024},
abstract = {The study of the brain encompasses multiple scales, including temporal, spatial, and functional aspects. To integrate understanding across these different levels and modalities, it requires developing quantification methods and frameworks. Here, we present effective Geometric measures from Correlated Manifold Capacity theory (GCMC) for probing the functional structure in neural representations. We utilize a statistical physics approach to establish analytical connections between neural co-variabilities and downstream read-out efficiency. These effective geometric measures capture both stimulus-driven and behavior-driven structures in neural population activities, while extracting computationally-relevant information from neural data into intuitive and interpretable analysis descriptors. We apply GCMC to a diverse collection of datasets with different recording methods, various model organisms, and multiple task modalities. Specifically, we demonstrate that GCMC enables a wide range of multi-scale data analysis. This includes quantifying the spatial progression of encoding efficiency across brain regions, revealing the temporal dynamics of task-relevant manifold geometry in information processing, and characterizing variances as well as invariances in neural representations throughout learning. Lastly, the effective manifold geometric measures may be viewed as order parameters for phases related to computational efficiency, facilitating data-driven hypothesis generation and latent embedding.},
archivePrefix = {biorxiv},
eprint = {2024.02.26.582157},
OpenAccessURL = {https://www.biorxiv.org/content/10.1101/2024.02.26.582157v1}
}

Download Citation (BibTex)

Neuronal Temporal Filters as Normal Mode Extractors

S. Golkar, J. Berman, D. Lipshutz, Robert Mihai Haret, Tim Gollisch, D. Chklovskii

arXiv:2401.03248

Physical Review Research, 6:013111

Export Citation

January 29, 2024

Neuronal Temporal Filters as Normal Mode Extractors

@article{Lipshutz_Chklovskii_arxiv:2401.03248_PRR_2024,
title = {{Neuronal Temporal Filters as Normal Mode Extractors}},
author = {Golkar, Siavash and Berman, Jules and Lipshutz, David and Haret, Robert Mihai and Gollisch, Tim and Chklovskii, Dmitri B},
abstract = {To generate actions in the face of physiological delays, the brain must predict the future. Here we explore how prediction may lie at the core of brain function by considering a neuron predicting the future of a scalar time series input. Assuming that the dynamics of the lag vector (a vector composed of several consecutive elements of the time series) are locally linear, normal mode decomposition decomposes the dynamics into independently evolving (eigen)modes allowing for straightforward prediction. We propose that a neuron learns the top mode and projects its input onto the associated subspace. Under this interpretation, the temporal filter of a neuron corresponds to the left eigenvector of a generalized eigenvalue problem. We mathematically analyze the operation of such an algorithm on noisy observations of synthetic data generated by a linear system. Interestingly, the shape of the temporal filter varies with the signal-to-noise ratio (SNR): a noisy input yields a monophasic filter and a growing SNR leads to multiphasic filters with progressively greater number of phases. Such variation in the temporal filter with input SNR resembles that observed experimentally in biological neurons.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2401.03248},
archivePrefix = {arXiv},
year_posted = {2024},
Journal = Physical Review Research,
center = {CCN},
OpenAccessURL = {https://link.aps.org/doi/10.1103/PhysRevResearch.6.013111},
year = {2024},
volume = {6},
issue = {1},
pages = {013111},
numpages = {8},
month = {Jan},
publisher = {American Physical Society},
doi = {10.1103/PhysRevResearch.6.013111}
}

Download Citation (BibTex)

Generalization in diffusion models arises from geometry-adaptive harmonic representations

Z. Kadkhodaie, F. Guth, E. P. Simoncelli, S. Mallat

arXiv:2310.02557

The Twelfth International Conference on Learning Representations, 2024

Export Citation

January 16, 2024

Generalization in diffusion models arises from geometry-adaptive harmonic representations

"@article{Kadkhodaie_Simoncelli_arxiv:2310.02557_ICLR24_2024,
      title={{Generalization in diffusion models arises from geometry-adaptive harmonic representations}},
      author={\text{Kadkhodaie, Zahra} and \text{Guth, Florentin} and \text{Simoncelli, Eero} and \text{Mallat, St\'{{e}}phane}},
doi={10.48550/arXiv.2409.19460},
Journal = ICLR24,
abstract={We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.},
      Publisher={ICLR 2024 Workshop on Representational Alignment},
year_posted={2023},
year={2024},
month={Mar},
      eprint={2310.02557},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2310.02557}, center={CCN}, acrosscenters={yes}, external={no}, highlight={no},
      Open Access Url={https://arxiv.org/abs/2310.02557 }
}"

Download Citation (BibTex)

Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities

Benjamin S. H. Lyo, Cristina Savin

bioRxiv:2023.10.03.560701

The Twelfth International Conference on Learning Representations, 2024

Export Citation

January 16, 2024

Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities

@article{Lyo_Savin_bioxriv:2023.10.03.560701_ICLR24_2024,
title = {{Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities}},
author = {Benjamin S. H. Lyo AND Cristina Savin},
abstract = {Despite many successful examples in which probabilistic inference can account for perception, we have little understanding of how the brain represents and uses structured priors that capture the complexity of natural input statistics. Here we construct a recurrent circuit model that can implicitly represent priors over latent variables, and combine them with sensory and contextual sources of information to encode task-specific posteriors. Inspired by the recent success of diffusion models as means of learning and using priors over images, our model uses dendritic nonlinearities optimized for denoising, and stochastic somatic integration with the degree of noise modulated by an oscillating global signal. Combining these elements into a recurrent network yields a stochastic dynamical system that samples from the prior at a rate prescribed by the period of the global oscillator. Additional inputs reflecting sensory or top-down contextual information alter these dynamics to generate samples from the corresponding posterior, with different input gating patterns selecting different inference tasks. We demonstrate that this architecture can sample from low dimensional nonlinear manifolds and multimodal posteriors. Overall, the model provides a new framework for circuit-level representation of probabilistic information, in a format that facilitates flexible inference.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {bioRxiv:2023.10.03.560701},
archivePrefix = {bioRriv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://openreview.net/pdf?id=S5aUhpuyap},
Journal = ICLR24,
year = {2024}
}

Download Citation (BibTex)

Previous Page
Viewing
Next Page