CCN: Publications

Clear Filters

Filter by Group

Computational Vision

Neural Circuits and Algorithms

NeuroAI and Geometric Data Analysis

Statistical Analysis of Neural Data

207 Publications

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

Rylan Schaeffer, Berivan Isik, Dhruv Bhandarkar Pai, Andres Carranza, Victor Lecomte, Alyssa Unell, Mikail Khona, T. Yerxa, Y. LeCun, S. Chung , Andrey Gromov, Ravid Shwartz-Ziv, Sanmi Koyejo

arXiv:2406.09366

ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024

Export Citation

March 4, 2024

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

@article{Yerxa_Chung_arxiv:2406.09366_2024,
title = {{Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations}},
author = {Rylan Schaeffer and Victor Lecomte and Dhruv Bhandarkar Pai and Andres Carranza and Berivan Isik and Alyssa Unell and Mikail Khona and Thomas Yerxa and Yann LeCun and SueYeon Chung and Andrey Gromov and Ravid Shwartz-Ziv and Sanmi Koyejo},
abstract = {Maximum Manifold Capacity Representations (MMCR) is a recent multi-view self-supervised learning (MVSSL) method that matches or surpasses other leading MVSSL methods. MMCR is intriguing because it does not fit neatly into any of the commonplace MVSSL lineages, instead originating from a statistical mechanical perspective on the linear separability of data manifolds. In this paper, we seek to improve our understanding and our utilization of MMCR. To better understand MMCR, we leverage tools from high dimensional probability to demonstrate that MMCR incentivizes alignment and uniformity of learned embeddings. We then leverage tools from information theory to show that such embeddings maximize a well-known lower bound on mutual information between views, thereby connecting the geometric perspective of MMCR to the information-theoretic perspective commonly discussed in MVSSL. To better utilize MMCR, we mathematically predict and experimentally confirm non-monotonic changes in the pretraining loss akin to double descent but with respect to atypical hyperparameters. We also discover compute scaling laws that enable predicting the pretraining loss as a function of gradients steps, batch size, embedding dimension and number of views. We then show that MMCR, originally applied to image data, is performant on multimodal image-text data. By more deeply understanding the theoretical and empirical behavior of MMCR, our work reveals insights on improving MVSSL methods.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2406.09366},
archivePrefix = {arxiv},
year_posted = {2024},
center = {ccn},
OpenAccessURL = {https://arxiv.org/abs/2406.09366},
Journal = {ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models
(ME-FoMo)},
year = {2024}
}

Download Citation (BibTex)

On the universality of neural encodings in CNNs

F. Guth, Brice Ménard

arXiv:2409.19460

ICLR 2024 Workshop on Representational Alignment

Export Citation

March 2, 2024

On the universality of neural encodings in CNNs

@article{Guth_arxiv:2409.19460_ICLR24_2024,
      title={{On the universality of neural encodings in CNNs}},
      author={\text{Guth, Florentin}} and Brice Ménard},
doi={10.48550/arXiv.2409.19460},
Journal = ICLR24,
abstract={We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.},
      Publisher={ICLR 2024 Workshop on Representational Alignment}, yearposted={2024}, year={2024}, month={Mar},  numpages={13},
      eprint={2409.19460},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2409.19460}, center={CCN}, acrosscenters={no}, external={yes}, highlight={no},
      Open Access Url={https://arxiv.org/abs/2409.19460}
}

Download Citation (BibTex)

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

D. Lipshutz, A. Nejatbakhsh, A. Williams

ICLR 2024 Workshop on Representational Alignment, 2024

Export Citation

March 2, 2024

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

@article{Lipshutz_Williams_ICLR24_2024,
title = {{Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry}},
author = {David Lipshutz AND Amin Nejatbakhsh and Alex H Williams},
abstract = {Uncovering and comparing the dynamical mechanisms that support neural processing remains a key challenge in the analysis of biological and artificial neural systems. However, measures of representational (dis)similarity in neural systems often assume that neural responses are static in time. Here, we show that stochastic shape distances (SSDs; Duong et al., 2023), which were developed to compare noisy neural responses to static inputs and lack an explicit notion of temporal structure, are well equipped to compare noisy dynamics. In two examples, we use SSDs, which interpolate between comparing mean trajectories and secondorder fluctuations about mean trajectories, to disentangle recurrent versus external contributions to noisy dynamics.},
acrosscenters = {no},
external = {no},
highlight = {no},
year_posted = {2024},
center = {CCN},
OpenAccessURL = {https://openreview.net/pdf?id=Fykvxdv2I8},
Journal = {ICLR 2024 Workshop on Representational Alignment},
year = {2024}
}

Download Citation (BibTex)

Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors

C. Chou , Luke Arend, Albert J. Wakhloo, Royoung Kim, Will Slatton, S. Chung

bioRxiv:2024.02.26.582157v1

Export Citation

February 28, 2024

Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors

@article{Chou_Chung_bioRxiv:2024.02.26.582157,
title = {{Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors}},
author = {Chi-Ning Chou and Luke Arend and Albert J. Wakhloo and Royoung Kim and Will Slatton and SueYeon Chung},
year_posted = {2024},
Journal = {bioRxiv e-prints},
center = {CCN},
acrosscenters = {no},
external = {yes},
highlight = {no},
year = {2024},
abstract = {The study of the brain encompasses multiple scales, including temporal, spatial, and functional aspects. To integrate understanding across these different levels and modalities, it requires developing quantification methods and frameworks. Here, we present effective Geometric measures from Correlated Manifold Capacity theory (GCMC) for probing the functional structure in neural representations. We utilize a statistical physics approach to establish analytical connections between neural co-variabilities and downstream read-out efficiency. These effective geometric measures capture both stimulus-driven and behavior-driven structures in neural population activities, while extracting computationally-relevant information from neural data into intuitive and interpretable analysis descriptors. We apply GCMC to a diverse collection of datasets with different recording methods, various model organisms, and multiple task modalities. Specifically, we demonstrate that GCMC enables a wide range of multi-scale data analysis. This includes quantifying the spatial progression of encoding efficiency across brain regions, revealing the temporal dynamics of task-relevant manifold geometry in information processing, and characterizing variances as well as invariances in neural representations throughout learning. Lastly, the effective manifold geometric measures may be viewed as order parameters for phases related to computational efficiency, facilitating data-driven hypothesis generation and latent embedding.},
archivePrefix = {biorxiv},
eprint = {2024.02.26.582157},
OpenAccessURL = {https://www.biorxiv.org/content/10.1101/2024.02.26.582157v1}
}

Download Citation (BibTex)

Neuronal Temporal Filters as Normal Mode Extractors

S. Golkar, J. Berman, D. Lipshutz, Robert Mihai Haret, Tim Gollisch, D. Chklovskii

arXiv:2401.03248

Physical Review Research, 6:013111

Export Citation

January 29, 2024

Neuronal Temporal Filters as Normal Mode Extractors

@article{Lipshutz_Chklovskii_arxiv:2401.03248_PRR_2024,
title = {{Neuronal Temporal Filters as Normal Mode Extractors}},
author = {Golkar, Siavash and Berman, Jules and Lipshutz, David and Haret, Robert Mihai and Gollisch, Tim and Chklovskii, Dmitri B},
abstract = {To generate actions in the face of physiological delays, the brain must predict the future. Here we explore how prediction may lie at the core of brain function by considering a neuron predicting the future of a scalar time series input. Assuming that the dynamics of the lag vector (a vector composed of several consecutive elements of the time series) are locally linear, normal mode decomposition decomposes the dynamics into independently evolving (eigen)modes allowing for straightforward prediction. We propose that a neuron learns the top mode and projects its input onto the associated subspace. Under this interpretation, the temporal filter of a neuron corresponds to the left eigenvector of a generalized eigenvalue problem. We mathematically analyze the operation of such an algorithm on noisy observations of synthetic data generated by a linear system. Interestingly, the shape of the temporal filter varies with the signal-to-noise ratio (SNR): a noisy input yields a monophasic filter and a growing SNR leads to multiphasic filters with progressively greater number of phases. Such variation in the temporal filter with input SNR resembles that observed experimentally in biological neurons.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2401.03248},
archivePrefix = {arXiv},
year_posted = {2024},
Journal = Physical Review Research,
center = {CCN},
OpenAccessURL = {https://link.aps.org/doi/10.1103/PhysRevResearch.6.013111},
year = {2024},
volume = {6},
issue = {1},
pages = {013111},
numpages = {8},
month = {Jan},
publisher = {American Physical Society},
doi = {10.1103/PhysRevResearch.6.013111}
}

Download Citation (BibTex)

Generalization in diffusion models arises from geometry-adaptive harmonic representations

Z. Kadkhodaie, F. Guth, E. P. Simoncelli, S. Mallat

arXiv:2310.02557

The Twelfth International Conference on Learning Representations, 2024

Export Citation

January 16, 2024

Generalization in diffusion models arises from geometry-adaptive harmonic representations

"@article{Kadkhodaie_Simoncelli_arxiv:2310.02557_ICLR24_2024,
      title={{Generalization in diffusion models arises from geometry-adaptive harmonic representations}},
      author={\text{Kadkhodaie, Zahra} and \text{Guth, Florentin} and \text{Simoncelli, Eero} and \text{Mallat, St\'{{e}}phane}},
doi={10.48550/arXiv.2409.19460},
Journal = ICLR24,
abstract={We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.},
      Publisher={ICLR 2024 Workshop on Representational Alignment},
year_posted={2023},
year={2024},
month={Mar},
      eprint={2310.02557},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2310.02557}, center={CCN}, acrosscenters={yes}, external={no}, highlight={no},
      Open Access Url={https://arxiv.org/abs/2310.02557 }
}"

Download Citation (BibTex)

Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities

Benjamin S. H. Lyo, Cristina Savin

bioRxiv:2023.10.03.560701

The Twelfth International Conference on Learning Representations, 2024

Export Citation

January 16, 2024

Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities

@article{Lyo_Savin_bioxriv:2023.10.03.560701_ICLR24_2024,
title = {{Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities}},
author = {Benjamin S. H. Lyo AND Cristina Savin},
abstract = {Despite many successful examples in which probabilistic inference can account for perception, we have little understanding of how the brain represents and uses structured priors that capture the complexity of natural input statistics. Here we construct a recurrent circuit model that can implicitly represent priors over latent variables, and combine them with sensory and contextual sources of information to encode task-specific posteriors. Inspired by the recent success of diffusion models as means of learning and using priors over images, our model uses dendritic nonlinearities optimized for denoising, and stochastic somatic integration with the degree of noise modulated by an oscillating global signal. Combining these elements into a recurrent network yields a stochastic dynamical system that samples from the prior at a rate prescribed by the period of the global oscillator. Additional inputs reflecting sensory or top-down contextual information alter these dynamics to generate samples from the corresponding posterior, with different input gating patterns selecting different inference tasks. We demonstrate that this architecture can sample from low dimensional nonlinear manifolds and multimodal posteriors. Overall, the model provides a new framework for circuit-level representation of probabilistic information, in a format that facilitates flexible inference.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {bioRxiv:2023.10.03.560701},
archivePrefix = {bioRriv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://openreview.net/pdf?id=S5aUhpuyap},
Journal = ICLR24,
year = {2024}
}

Download Citation (BibTex)

Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds

Michael Kuoch, C. Chou , N. Parthasarathy, Joel Dapello, J. DiCarlo, H. Sompolinsky, S. Chung

arXiv:2312.14285

Conference on Parsimony and Learning, PMLR, 234: 395-418

Export Citation

January 6, 2024

Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds

@article{Kuoch_Chou_Chung_arXiv:2312.14285_CPAL24_2024,
title = {{Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds}},
author = {Michael Kuoch and Chou, Chi-Ning and Nikhil Parthasarathy and Joel Dapello and James J. DiCarlo and Haim Sompolinsky and Chung, SueYeon},
editor = {Chi, Yuejie and Dziugaite, Gintare Karolina and Qu, Qing and Wang, Atlas Wang and Zhu, Zhihui},
journal = {Conference on Parsimony and Learning, PMLR},
volume = {234},
pages =	{395-418},
Year_posted = {2023},
year = {2024},
month = Sep,
publisher = {Proceedings of Machine Learning Research},
doi = {10.48550/arXiv.2312.14285},
url = {https://proceedings.mlr.press/v234/kuoch24a.html},
Eprint = {2312.14285},
archivePrefix={arXiv},
OpenAccessURL = {https://arxiv.org/abs/2312.14285},
center = {CCN},
acrosscenters = {no},
external = {yes},
highlight = {no},
Abstract = {In recent years, growth in our understanding of the computations performed in both biological and artificial neural networks has largely been driven by either low-level mechanistic studies or global normative approaches. However, concrete methodologies for bridging the gap between these levels of abstraction remain elusive. In this work, we investigate the internal mechanisms of neural networks through the lens of neural population geometry, aiming to provide understanding at an intermediate level of abstraction, as a way to bridge that gap. Utilizing manifold capacity theory (MCT) from statistical physics and manifold alignment analysis (MAA) from high-dimensional statistics, we probe the underlying organization of task-dependent manifolds in deep neural networks and neural recordings from the macaque visual cortex. Specifically, we quantitatively characterize how different learning objectives lead to differences in the organizational strategies of these models and demonstrate how these geometric analyses are connected to the decodability of task-relevant information. Furthermore, these metrics show that macaque visual cortex data are more similar to unsupervised DNNs in terms of geometrical properties such as manifold position and manifold alignment. These analyses present a strong direction for bridging mechanistic and normative theories in neural networks through neural population geometry, potentially opening up many future research avenues in both machine learning and neuroscience.}
}

Download Citation (BibTex)

Adaptive whitening with fast gain modulation and slow synaptic plasticity

L. Duong, E. P. Simoncelli, D. Chklovskii, D. Lipshutz

Advances in Neural Information Processing Systems, Vol. 36

Export Citation

2023

Adaptive whitening with fast gain modulation and slow synaptic plasticity

@INPROCEEDINGS{Duong23f,
     TITLE= "Adaptive whitening with fast gain modulation and slow synaptic plasticity",
     AUTHOR= "L R Duong and E P Simoncelli and D B Chklovskii and D Lipshutz",
     BOOKTITLE= {Adv. Neural Information Processing Systems (NeurIPS*23)},
     VOLUME= 36,
     MONTH= "Dec",
     YEAR= 2023,
     ADDRESS= "New Orleans",
     PDF-URL= "http://www.cns.nyu.edu/pub/lcv/duong23f.pdf",
     NOTE= "To appear",
}

Download Citation (BibTex)

A polar prediction model for learning to represent visual transformations

P. Fiquet, E. P. Simoncelli

Advances in Neural Information Processing Systems, Vol. 36

Export Citation

2023

A polar prediction model for learning to represent visual transformations

@INPROCEEDINGS{Fiquet23c,
     TITLE= "A polar prediction model for learning to represent visual transformations",
     AUTHOR= "P-E Fiquet and E P Simoncelli",
     BOOKTITLE= {Adv. Neural Information Processing Systems (NeurIPS*23)},
     VOLUME= 36,
     MONTH= "Dec",
     YEAR= 2023,
     ADDRESS= "New Orleans",
     PDF-URL= "http://www.cns.nyu.edu/pub/lcv/fiquet23c.pdf",
     NOTE= "To appear",
}

Download Citation (BibTex)

Previous Page
Viewing
Next Page