CCN: Publications

Clear Filters

Filter by Group

Computational Vision

Neural Circuits and Algorithms

NeuroAI and Geometric Data Analysis

Statistical Analysis of Neural Data

214 Publications

Disparate nonlinear neural dynamics measured with different techniques in macaque and human V1

J. Zhou, Matt Whitmire, Yuzhi Chen, Eyal Seidemann

Scientific Reports, 14:13193

June 8, 2024

Layerwise complexity-matched learning yields an improved model of cortical area {V2}

N. Parthasarathy, O J Henaff, E. P. Simoncelli

arXiv: 2312.11436

Trans. Machine Learning Research

Export Citation

2024

Layerwise complexity-matched learning yields an improved model of cortical area {V2}

@ARTICLE{Parthasarathy24,
  TITLE=        "Layerwise complexity-matched learning yields an improved model of
                 cortical area {V2}",
  AUTHOR=       "N Parthasarathy and O J H\'{e}naff and E P Simoncelli",
  JOURNAL=      "Trans. Machine Learning Research",
  MONTH=        "Jun",
  YEAR=         2024,
  PUBLISHER=    "https://jmlr.org/tmlr/index.html",
  PDF-URL=      "https://www.cns.nyu.edu/pub/lcv/parthasarathy24.pdf",
  URL=          "https://openreview.net/forum?id=lQBsLfAWhj",
  NOTE=         "Featured Certification",
}

Download Citation (BibTex)

HybridBNN: Joint Training of Deterministic and Stochastic Layers in Bayesian Neural Nets

A. Nejatbakhsh, Julien Boussard

Sixth Symposium on Advances in Approximate Bayesian Inference - Non Archival Track, 2024:1-11

Export Citation

May 27, 2024

HybridBNN: Joint Training of Deterministic and Stochastic Layers in Bayesian Neural Nets

"@article{Nejatbakhsh_AABI24_20224,
title = {{HybridBNN: Joint Training of Deterministic and Stochastic Layers in Bayesian Neural Nets}},
author = {Amin Nejatbakhsh and Julien Boussard},
abstract = {Bayesian Neural Nets are proposed as flexible models that can provide calibrated uncertainty estimates for out-of-distribution data. Due to the high dimensionality of BNN posteriors and the intractability of exact inference, numerous approximate inference techniques have been proposed. However, issues persist. Some approaches lack a proper Bayesian formulation while others result in inexpressive or uncalibrated posteriors, defeating the primary purpose of BNNs. Recently, subspace inference has been proposed to overcome these challenges by running the inference on a lower-dimensional subspace of network parameters. While achieving promising results, these methods are mathematically involved and therefore extending them to general architectures and problems is challenging. Here, we propose a new subspace inference method---called HybridBNN---that divides the network weights into deterministic and stochastic subsets before training. We develop an expectation-maximization algorithm for the joint inference of the posterior over the stochastic weights as well as the optimization of the deterministic ones. HybridBNN achieves competitive prediction and calibration performance on two regression and classification toy datasets and a benchmark dataset for in and out-of-domain distributions. The simplicity and flexibility of HybridBNN make it a favorable candidate for developing generic calibrated models.},
acrosscenters = {no},
external = {yes},
highlight = {no},
center = {CCN},
OpenAccessURL = {https://openreview.net/forum?id=ea3ERGY8bm},
Journal = {Sixth Symposium on Advances in Approximate Bayesian Inference - Non Archival Track},
year = {2024}
}"

Download Citation (BibTex)

Optimal compressed sensing for image reconstruction with diffusion probabilistic models

Ling-Qi Zhang, Z. Kadkhodaie, E. P. Simoncelli, D. H. Brainard

arXiv:2405.17456

Transactions on Machine Learning

Export Citation

May 22, 2024

Optimal compressed sensing for image reconstruction with diffusion probabilistic models

@article{Zhang_Kadkhodaie_Simoncelli_arXiv:2405.17456,
title = {{Optimal compressed sensing for image reconstruction with diffusion probabilistic models}},
author = {Ling-Qi Zhang and Zahra Kadkhodaie and Eero P. Simoncelli and David H. Brainard},
year_posted = {2024},
Journal = {Transactions On Machine Learning},
center = {CCN},
acrosscenters = {no},
external = {yes},
highlight = {no},
year = {2025},
abstract = {We examine the problem of selecting a small set of linear measurements for reconstructing high-dimensional signals. Well-established methods for optimizing such measurements include principal component analysis (PCA), independent component analysis (ICA) and compressed sensing (CS) based on random projections, all of which rely on axis- or subspace-aligned statistical characterization of the signal source. However, many naturally occurring signals, including photographic images, contain richer statistical structure. To exploit such structure, we introduce a general method for obtaining an optimized set of linear measurements for efficient image reconstruction, where the signal statistics are expressed by the prior implicit in a neural network trained to perform denoising (known as a ``diffusion model''). We demonstrate that the optimal measurements derived for two natural image datasets differ from those of PCA, ICA, or CS, and result in substantially lower mean squared reconstruction error. Interestingly, the marginal distributions of the measurement values are asymmetrical (skewed), substantially more so than those of previous methods. We also find that optimizing with respect to perceptual loss, as quantified by structural similarity (SSIM), leads to measurements different from those obtained when optimizing for MSE. Our results highlight the importance of incorporating the specific statistical regularities of natural signals when designing effective linear measurements.},
archivePrefix = {arXiv},
eprint = {2405.17456},
OpenAccessURL = {https://openreview.net/forum?id=lmHh4FmPWZ}
}

Download Citation (BibTex)

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration

James Ryu, A. Nejatbakhsh, Mahdi Torkashvand, Sahana Gangadharan, Maedeh Seyedolmohadesin, Jinmahn Kim, L. Paninski, Vivek Venkatachalam

bioRxiv:2022.07.18.500485v1

PLOS Computational Biology, 20: 1*5

Export Citation

May 20, 2024

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration

"@article{Nejatbakhsh_biorxiv:2022.07.18.500485_PLOSCB_2024,
title = {{Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration}},
author = {Ryu, James AND Nejatbakhsh, Amin AND Torkashvand, Mahdi AND Gangadharan, Sahana AND Seyedolmohadesin, Maedeh AND Kim, Jinmahn AND Paninski, Liam AND Venkatachalam, Vivek},
acrosscenters = {no},
external = {yes},
abstract = {Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results.},
eprint = {https://www.biorxiv.org/content/10.1101/2022.07.18.500485v1.full},
highlight = {no},
archivePrefix = {biorxiv},
year_posted = {2022},
Journal = {PLOS Computational Biology},
center = {CCN},
OpenAccessURL = {https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012075},
year = {2024},
publisher = {Public Library of Science},
month = {05},
    volume = {20},
    number = {5},
    pages = {1-23},
doi = {10.1371/journal.pcbi.1012075}
}"

Download Citation (BibTex)

A unified framework for perceived magnitude and discriminability of sensory stimuli

J. Zhou, L. Duong, E. P. Simoncelli

bioXriv:2022.04.30.490146v3

Proceedings of the National Academy of Sciences of the United States of America, 121: e2312293121

Export Citation

April 25, 2024

A unified framework for perceived magnitude and discriminability of sensory stimuli

@article{Zhou_Simoncelli_biorxiv:2022.04.30.490146_PNAS_2024,
title = {{A unified framework for perceived magnitude and discriminability of sensory stimuli}},
author = {Zhou,  Jingyang and Duong,  Lyndon R. and Simoncelli,  Eero P.},
abstract = {The perception of sensory attributes is often quantified through measurements of sensitivity (the ability to detect small stimulus changes), as well as through direct judgements of appearance or intensity. Despite their ubiquity, the relationship between these two measurements remains controversial and unresolved. Here, we propose a framework in which they arise from different aspects of a common representation. Specifically, we assume that judgements of stimulus intensity (e.g., as measured through rating scales) reflect the mean value of an internal representation, and sensitivity reflects a combination of mean value and noise properties, as quantified by the statistical measure of Fisher Information. Unique identification of these internal representation properties can be achieved by combining measurements of sensitivity and judgments of intensity. As a central example, we show that Weber{\textquoteright}s law of perceptual sensitivity can co-exist with Stevens{\textquoteright} power-law scaling of intensity ratings (for all exponents), when the noise amplitude increases in proportion to the representational mean. We then extend this result beyond the Weber{\textquoteright}s law range by incorporating a more general and physiology-inspired form of noise, and show that the combination of noise properties and sensitivity measurements accurately predicts intensity ratings across a variety of sensory modalities and attributes. Our framework unifies two primary perceptual measurements {\textendash} thresholds for sensitivity and rating scales for intensity {\textendash} and provides a neural interpretation for the underlying representation.Significance Statement Perceptual measurements of sensitivity to stimulus changes and stimulus appearance (intensity) are ubiquitous in the study of perception. However, the relationship between these two seemingly disparate measurements remains unclear. Proposals for unification have been made for over 60 years, but they generally lack support from perceptual or physiological measurements. Here, we provide a framework that offers a unified interpretation of perceptual sensitivity and intensity measurements, and we demonstrate its consistency with experimental measurements across multiple perceptual domains.Competing Interest StatementThe authors have declared no competing interest.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {https://www.biorxiv.org/content/early/2024/05/29/2022.04.30.490146.full.pdf},
archivePrefix = {bioRxiv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://www.pnas.org/doi/10.1073/pnas.2312293121},
Journal = {Proceedings of the National Academy of Sciences of the United States of America},
volume = {121},
ISSN = {1091-6490},
month = jun,
DOI = {10.1073/pnas.2312293121},
number = {25},
publisher = {Proceedings of the National Academy of Sciences},
year = {2024}
}

Download Citation (BibTex)

Estimating Shape Distances on Neural Representations with Limited Samples

Dean A. Pospisil, B. Larsen, S. Harvey, A. Williams

arXiv:2310.05742

The Twelfth International Conference on Learning Representations, 2024

Export Citation

March 15, 2024

Estimating Shape Distances on Neural Representations with Limited Samples

@article{Pospisil_Larsen_Williams_arxiv:2310.05742_ICLR24_2024,
title = {{Estimating Shape Distances on Neural Representations with Limited Samples}},
author = {Dean A Pospisil and Brett W. Larsen and Sarah E Harvey and Alex H Williams},
abstract = {Measuring geometric similarity between high-dimensional network representations is a topic of longstanding interest to neuroscience and deep learning. Although many methods have been proposed, only a few works have rigorously analyzed their statistical efficiency or quantified estimator uncertainty in data-limited regimes. Here, we derive upper and lower bounds on the worst-case convergence of standard estimators of shape distance—a measure of representational dissimilarity proposed by Williams et al. (2021). These bounds reveal the challenging nature of the problem in high-dimensional feature spaces. To overcome these challenges, we introduce a novel method-of-moments estimator with a tunable bias-variance tradeoff parameterized by an upper bound on bias. We show that this estimator achieves superior performance to standard estimators in simulation and on neural data, particularly in high-dimensional settings. Our theoretical work and estimator thus respectively define and dramatically expand the scope of neural data for which geometric similarity can be accurately measured.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2310.05742},
archivePrefix = {arXiv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://openreview.net/forum?id=kvByNnMERu},
Journal = ICLR24,
year = {2024}
}

Download Citation (BibTex)

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

Rylan Schaeffer, Berivan Isik, Dhruv Bhandarkar Pai, Andres Carranza, Victor Lecomte, Alyssa Unell, Mikail Khona, T. Yerxa, Y. LeCun, S. Chung , Andrey Gromov, Ravid Shwartz-Ziv, Sanmi Koyejo

arXiv:2406.09366

ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024

Export Citation

March 4, 2024

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

@article{Yerxa_Chung_arxiv:2406.09366_2024,
title = {{Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations}},
author = {Rylan Schaeffer and Victor Lecomte and Dhruv Bhandarkar Pai and Andres Carranza and Berivan Isik and Alyssa Unell and Mikail Khona and Thomas Yerxa and Yann LeCun and SueYeon Chung and Andrey Gromov and Ravid Shwartz-Ziv and Sanmi Koyejo},
abstract = {Maximum Manifold Capacity Representations (MMCR) is a recent multi-view self-supervised learning (MVSSL) method that matches or surpasses other leading MVSSL methods. MMCR is intriguing because it does not fit neatly into any of the commonplace MVSSL lineages, instead originating from a statistical mechanical perspective on the linear separability of data manifolds. In this paper, we seek to improve our understanding and our utilization of MMCR. To better understand MMCR, we leverage tools from high dimensional probability to demonstrate that MMCR incentivizes alignment and uniformity of learned embeddings. We then leverage tools from information theory to show that such embeddings maximize a well-known lower bound on mutual information between views, thereby connecting the geometric perspective of MMCR to the information-theoretic perspective commonly discussed in MVSSL. To better utilize MMCR, we mathematically predict and experimentally confirm non-monotonic changes in the pretraining loss akin to double descent but with respect to atypical hyperparameters. We also discover compute scaling laws that enable predicting the pretraining loss as a function of gradients steps, batch size, embedding dimension and number of views. We then show that MMCR, originally applied to image data, is performant on multimodal image-text data. By more deeply understanding the theoretical and empirical behavior of MMCR, our work reveals insights on improving MVSSL methods.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2406.09366},
archivePrefix = {arxiv},
year_posted = {2024},
center = {ccn},
OpenAccessURL = {https://arxiv.org/abs/2406.09366},
Journal = {ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models
(ME-FoMo)},
year = {2024}
}

Download Citation (BibTex)

On the universality of neural encodings in CNNs

F. Guth, Brice Ménard

arXiv:2409.19460

ICLR 2024 Workshop on Representational Alignment

Export Citation

March 2, 2024

On the universality of neural encodings in CNNs

@article{Guth_arxiv:2409.19460_ICLR24_2024,
      title={{On the universality of neural encodings in CNNs}},
      author={\text{Guth, Florentin}} and Brice Ménard},
doi={10.48550/arXiv.2409.19460},
Journal = ICLR24,
abstract={We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.},
      Publisher={ICLR 2024 Workshop on Representational Alignment}, yearposted={2024}, year={2024}, month={Mar},  numpages={13},
      eprint={2409.19460},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2409.19460}, center={CCN}, acrosscenters={no}, external={yes}, highlight={no},
      Open Access Url={https://arxiv.org/abs/2409.19460}
}

Download Citation (BibTex)

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

D. Lipshutz, A. Nejatbakhsh, A. Williams

ICLR 2024 Workshop on Representational Alignment, 2024

Export Citation

March 2, 2024

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

@article{Lipshutz_Williams_ICLR24_2024,
title = {{Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry}},
author = {David Lipshutz AND Amin Nejatbakhsh and Alex H Williams},
abstract = {Uncovering and comparing the dynamical mechanisms that support neural processing remains a key challenge in the analysis of biological and artificial neural systems. However, measures of representational (dis)similarity in neural systems often assume that neural responses are static in time. Here, we show that stochastic shape distances (SSDs; Duong et al., 2023), which were developed to compare noisy neural responses to static inputs and lack an explicit notion of temporal structure, are well equipped to compare noisy dynamics. In two examples, we use SSDs, which interpolate between comparing mean trajectories and secondorder fluctuations about mean trajectories, to disentangle recurrent versus external contributions to noisy dynamics.},
acrosscenters = {no},
external = {no},
highlight = {no},
year_posted = {2024},
center = {CCN},
OpenAccessURL = {https://openreview.net/pdf?id=Fykvxdv2I8},
Journal = {ICLR 2024 Workshop on Representational Alignment},
year = {2024}
}

Download Citation (BibTex)

Previous Page
Viewing
Next Page