CCN: Publications

Filter by Group

Computational Vision

Neural Circuits and Algorithms

NeuroAI and Geometric Data Analysis

Statistical Analysis of Neural Data

181 Publications

Estimating Shape Distances on Neural Representations with Limited Samples

Dean A. Pospisil, B. Larsen, S. Harvey, A. Williams

arXiv:2310.05742

The Twelfth International Conference on Learning Representations, 2024

Export Citation

March 15, 2024

Estimating Shape Distances on Neural Representations with Limited Samples

@article{Pospisil_Larsen_Williams_arxiv:2310.05742_ICLR24_2024,
title = {{Estimating Shape Distances on Neural Representations with Limited Samples}},
author = {Dean A Pospisil and Brett W. Larsen and Sarah E Harvey and Alex H Williams},
abstract = {Measuring geometric similarity between high-dimensional network representations is a topic of longstanding interest to neuroscience and deep learning. Although many methods have been proposed, only a few works have rigorously analyzed their statistical efficiency or quantified estimator uncertainty in data-limited regimes. Here, we derive upper and lower bounds on the worst-case convergence of standard estimators of shape distance—a measure of representational dissimilarity proposed by Williams et al. (2021). These bounds reveal the challenging nature of the problem in high-dimensional feature spaces. To overcome these challenges, we introduce a novel method-of-moments estimator with a tunable bias-variance tradeoff parameterized by an upper bound on bias. We show that this estimator achieves superior performance to standard estimators in simulation and on neural data, particularly in high-dimensional settings. Our theoretical work and estimator thus respectively define and dramatically expand the scope of neural data for which geometric similarity can be accurately measured.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2310.05742},
archivePrefix = {arXiv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://openreview.net/forum?id=kvByNnMERu},
Journal = ICLR24,
year = {2024}
}

Download Citation (BibTex)

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

Rylan Schaeffer, Berivan Isik, Dhruv Bhandarkar Pai, Andres Carranza, Victor Lecomte, Alyssa Unell, Mikail Khona, T. Yerxa, Y. LeCun, S. Chung , Andrey Gromov, Ravid Shwartz-Ziv, Sanmi Koyejo

arXiv:2406.09366

ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024

Export Citation

March 4, 2024

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

@article{Yerxa_Chung_arxiv:2406.09366_2024,
title = {{Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations}},
author = {Rylan Schaeffer and Victor Lecomte and Dhruv Bhandarkar Pai and Andres Carranza and Berivan Isik and Alyssa Unell and Mikail Khona and Thomas Yerxa and Yann LeCun and SueYeon Chung and Andrey Gromov and Ravid Shwartz-Ziv and Sanmi Koyejo},
abstract = {Maximum Manifold Capacity Representations (MMCR) is a recent multi-view self-supervised learning (MVSSL) method that matches or surpasses other leading MVSSL methods. MMCR is intriguing because it does not fit neatly into any of the commonplace MVSSL lineages, instead originating from a statistical mechanical perspective on the linear separability of data manifolds. In this paper, we seek to improve our understanding and our utilization of MMCR. To better understand MMCR, we leverage tools from high dimensional probability to demonstrate that MMCR incentivizes alignment and uniformity of learned embeddings. We then leverage tools from information theory to show that such embeddings maximize a well-known lower bound on mutual information between views, thereby connecting the geometric perspective of MMCR to the information-theoretic perspective commonly discussed in MVSSL. To better utilize MMCR, we mathematically predict and experimentally confirm non-monotonic changes in the pretraining loss akin to double descent but with respect to atypical hyperparameters. We also discover compute scaling laws that enable predicting the pretraining loss as a function of gradients steps, batch size, embedding dimension and number of views. We then show that MMCR, originally applied to image data, is performant on multimodal image-text data. By more deeply understanding the theoretical and empirical behavior of MMCR, our work reveals insights on improving MVSSL methods.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2406.09366},
archivePrefix = {arxiv},
year_posted = {2024},
center = {ccn},
OpenAccessURL = {https://arxiv.org/abs/2406.09366},
Journal = {ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models
(ME-FoMo)},
year = {2024}
}

Download Citation (BibTex)

On the universality of neural encodings in CNNs

F. Guth, Brice Ménard

arXiv:2409.19460

ICLR 2024 Workshop on Representational Alignment

Export Citation

March 2, 2024

On the universality of neural encodings in CNNs

@article{Guth_arxiv:2409.19460_ICLR24_2024,
      title={{On the universality of neural encodings in CNNs}},
      author={\text{Guth, Florentin}} and Brice Ménard},
doi={10.48550/arXiv.2409.19460},
Journal = ICLR24,
abstract={We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.},
      Publisher={ICLR 2024 Workshop on Representational Alignment}, yearposted={2024}, year={2024}, month={Mar},  numpages={13},
      eprint={2409.19460},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2409.19460}, center={CCN}, acrosscenters={no}, external={yes}, highlight={no},
      Open Access Url={https://arxiv.org/abs/2409.19460}
}

Download Citation (BibTex)

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

D. Lipshutz, A. Nejatbakhsh, A. Williams

ICLR 2024 Workshop on Representational Alignment, 2024

Export Citation

March 2, 2024

Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry

@article{Lipshutz_Williams_ICLR24_2024,
title = {{Disentangling Recurrent Neural Dynamics with Stochastic Representational Geometry}},
author = {David Lipshutz AND Amin Nejatbakhsh and Alex H Williams},
abstract = {Uncovering and comparing the dynamical mechanisms that support neural processing remains a key challenge in the analysis of biological and artificial neural systems. However, measures of representational (dis)similarity in neural systems often assume that neural responses are static in time. Here, we show that stochastic shape distances (SSDs; Duong et al., 2023), which were developed to compare noisy neural responses to static inputs and lack an explicit notion of temporal structure, are well equipped to compare noisy dynamics. In two examples, we use SSDs, which interpolate between comparing mean trajectories and secondorder fluctuations about mean trajectories, to disentangle recurrent versus external contributions to noisy dynamics.},
acrosscenters = {no},
external = {no},
highlight = {no},
year_posted = {2024},
center = {CCN},
OpenAccessURL = {https://openreview.net/pdf?id=Fykvxdv2I8},
Journal = {ICLR 2024 Workshop on Representational Alignment},
year = {2024}
}

Download Citation (BibTex)

Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors

C. Chou , Luke Arend, Albert J. Wakhloo, Royoung Kim, Will Slatton, S. Chung

bioRxiv:2024.02.26.582157v1

Export Citation

February 28, 2024

Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors

@article{Chou_Chung_bioRxiv:2024.02.26.582157,
title = {{Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors}},
author = {Chi-Ning Chou and Luke Arend and Albert J. Wakhloo and Royoung Kim and Will Slatton and SueYeon Chung},
year_posted = {2024},
Journal = {bioRxiv e-prints},
center = {CCN},
acrosscenters = {no},
external = {yes},
highlight = {no},
year = {2024},
abstract = {The study of the brain encompasses multiple scales, including temporal, spatial, and functional aspects. To integrate understanding across these different levels and modalities, it requires developing quantification methods and frameworks. Here, we present effective Geometric measures from Correlated Manifold Capacity theory (GCMC) for probing the functional structure in neural representations. We utilize a statistical physics approach to establish analytical connections between neural co-variabilities and downstream read-out efficiency. These effective geometric measures capture both stimulus-driven and behavior-driven structures in neural population activities, while extracting computationally-relevant information from neural data into intuitive and interpretable analysis descriptors. We apply GCMC to a diverse collection of datasets with different recording methods, various model organisms, and multiple task modalities. Specifically, we demonstrate that GCMC enables a wide range of multi-scale data analysis. This includes quantifying the spatial progression of encoding efficiency across brain regions, revealing the temporal dynamics of task-relevant manifold geometry in information processing, and characterizing variances as well as invariances in neural representations throughout learning. Lastly, the effective manifold geometric measures may be viewed as order parameters for phases related to computational efficiency, facilitating data-driven hypothesis generation and latent embedding.},
archivePrefix = {biorxiv},
eprint = {2024.02.26.582157},
OpenAccessURL = {https://www.biorxiv.org/content/10.1101/2024.02.26.582157v1}
}

Download Citation (BibTex)

Neural population geometry and optimal coding of tasks with shared latent structure

Albert J. Wakhloo, Will Slatton, S. Chung

arXiv:2402.16770

Export Citation

February 26, 2024

Neural population geometry and optimal coding of tasks with shared latent structure

@article{Wakhloo_Chung_arxiv:2402.16770,
title = {{Neural population geometry and optimal coding of tasks with shared latent structure}},
author = {Albert J. Wakhloo and Will Slatton and SueYeon Chung},
acrosscenters = {no},
external = {yes},
abstract = {Humans and animals can recognize latent structures in their environment and apply this information to efficiently navigate the world. However, it remains unclear what aspects of neural activity contribute to these computational capabilities. Here, we develop an analytical theory linking the geometry of a neural population's activity to the generalization performance of a linear readout on a set of tasks that depend on a common latent structure. We show that four geometric measures of the activity determine performance across tasks. Using this theory, we find that experimentally observed disentangled representations naturally emerge as an optimal solution to the multi-task learning problem. When data is scarce, these optimal neural codes compress less informative latent variables, and when data is abundant, they expand these variables in the state space. We validate our theory using macaque ventral stream recordings. Our results therefore tie population geometry to multi-task learning.},
eprint = {2402.1677},
highlight = {no},
archivePrefix = {arXiv},
year_posted = {2024},
Journal = {arXiv e-prints},
center = {CCN},
year = {2024},
OpenAccessURL = {https://arxiv.org/abs/2402.16770}
}

Download Citation (BibTex)

Responses of neurons in macaque V4 to object and texture images

Justin D. Lieber, T. D. Oleskiw , E. P. Simoncelli, J. A. Movshon

arXiv:2024.02.20.581273

Export Citation

February 23, 2024

Responses of neurons in macaque V4 to object and texture images

@article{Lieber_Oleskiw_Simoncelli_biorxiv:2024.02.20.581273_2024,
title = {{Responses of neurons in macaque V4 to object and texture images}},
author = {Lieber, Justin D. and Oleskiw, Timothy D. and Simoncelli, Eero P. and Movshon, J. Anthony},
abstract = {Humans and monkeys can effortlessly recognize objects in everyday scenes. This ability relies on neural computations in the ventral stream of visual cortex. The intermediate computations that lead to object selectivity are not well understood, but previous studies implicate V4 as an early site of selectivity for object shape. To explore the mechanisms of this selectivity, we generated a continuum of images between {\textquotedblleft}scrambled{\textquotedblright} textures and photographic images of both natural and manmade environments, using techniques that preserve the local statistics of the original image while discarding information about scene and shape. We measured the responses of single units in awake macaque V4 to these images. On average, V4 neurons were slightly more active in response to photographic images than to their scrambled counterparts. However, responses in V4 varied widely both across different cells and different sets of images. An important determinant of this variation was the effectiveness of image families at driving strong neural responses. Across the full V4 population, a cell{\textquoteright}s average evoked firing rate for a family reliably predicted that family{\textquoteright}s preference for photographic over scrambled images. Accordingly, the cells that respond most strongly to each image family showed a much stronger difference between photographic and scrambled images and a graded level of modulation for images scrambled at intermediate levels. This preference for photographic images was not evident until \~{}50 ms after the onset of neuronal activity and did not peak in strength until 140 ms after activity onset. Finally, V4 neural responses seemed to categorically separate photographic images from all of their scrambled counterparts, despite the fact that the least scrambled images in our set appear similar to the originals. When these same images were analyzed with DISTS (Deep Image Structure and Texture Similarity), an image-computable similarity metric that predicts human judgements of image degradation, this same pattern emerged. This suggests that V4 responses are highly sensitive to small deviations from photographic image structure.Competing Interest StatementThe authors have declared no competing interest.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {https://www.biorxiv.org/content/early/2024/02/23/2024.02.20.581273.full.pdf},
archivePrefix = {bioRxiv},
year_posted = {2024},
center = {CCN},
OpenAccessURL = {https://www.biorxiv.org/content/early/2024/02/23/2024.02.20.581273},
Journal = {bioRxiv e-prints},
year = {2024}
}

Download Citation (BibTex)

Neuronal Temporal Filters as Normal Mode Extractors

S. Golkar, J. Berman, D. Lipshutz, Robert Mihai Haret, Tim Gollisch, D. Chklovskii

arXiv:2401.03248

Physical Review Research, 6:013111

Export Citation

January 29, 2024

Neuronal Temporal Filters as Normal Mode Extractors

@article{Lipshutz_Chklovskii_arxiv:2401.03248_PRR_2024,
title = {{Neuronal Temporal Filters as Normal Mode Extractors}},
author = {Golkar, Siavash and Berman, Jules and Lipshutz, David and Haret, Robert Mihai and Gollisch, Tim and Chklovskii, Dmitri B},
abstract = {To generate actions in the face of physiological delays, the brain must predict the future. Here we explore how prediction may lie at the core of brain function by considering a neuron predicting the future of a scalar time series input. Assuming that the dynamics of the lag vector (a vector composed of several consecutive elements of the time series) are locally linear, normal mode decomposition decomposes the dynamics into independently evolving (eigen)modes allowing for straightforward prediction. We propose that a neuron learns the top mode and projects its input onto the associated subspace. Under this interpretation, the temporal filter of a neuron corresponds to the left eigenvector of a generalized eigenvalue problem. We mathematically analyze the operation of such an algorithm on noisy observations of synthetic data generated by a linear system. Interestingly, the shape of the temporal filter varies with the signal-to-noise ratio (SNR): a noisy input yields a monophasic filter and a growing SNR leads to multiphasic filters with progressively greater number of phases. Such variation in the temporal filter with input SNR resembles that observed experimentally in biological neurons.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {2401.03248},
archivePrefix = {arXiv},
year_posted = {2024},
Journal = Physical Review Research,
center = {CCN},
OpenAccessURL = {https://link.aps.org/doi/10.1103/PhysRevResearch.6.013111},
year = {2024},
volume = {6},
issue = {1},
pages = {013111},
numpages = {8},
month = {Jan},
publisher = {American Physical Society},
doi = {10.1103/PhysRevResearch.6.013111}
}

Download Citation (BibTex)

Generalization in diffusion models arises from geometry-adaptive harmonic representations

Z. Kadkhodaie, F. Guth, E. P. Simoncelli, S. Mallat

arXiv:2310.02557

The Twelfth International Conference on Learning Representations, 2024

Export Citation

January 16, 2024

Generalization in diffusion models arises from geometry-adaptive harmonic representations

"@article{Kadkhodaie_Simoncelli_arxiv:2310.02557_ICLR24_2024,
      title={{Generalization in diffusion models arises from geometry-adaptive harmonic representations}},
      author={\text{Kadkhodaie, Zahra} and \text{Guth, Florentin} and \text{Simoncelli, Eero} and \text{Mallat, St\'{{e}}phane}},
doi={10.48550/arXiv.2409.19460},
Journal = ICLR24,
abstract={We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.},
      Publisher={ICLR 2024 Workshop on Representational Alignment},
year_posted={2023},
year={2024},
month={Mar},
      eprint={2310.02557},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2310.02557}, center={CCN}, acrosscenters={yes}, external={no}, highlight={no},
      Open Access Url={https://arxiv.org/abs/2310.02557 }
}"

Download Citation (BibTex)

Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities

Benjamin S. H. Lyo, Cristina Savin

bioRxiv:2023.10.03.560701

The Twelfth International Conference on Learning Representations, 2024

Export Citation

January 16, 2024

Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities

@article{Lyo_Savin_bioxriv:2023.10.03.560701_ICLR24_2024,
title = {{Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities}},
author = {Benjamin S. H. Lyo AND Cristina Savin},
abstract = {Despite many successful examples in which probabilistic inference can account for perception, we have little understanding of how the brain represents and uses structured priors that capture the complexity of natural input statistics. Here we construct a recurrent circuit model that can implicitly represent priors over latent variables, and combine them with sensory and contextual sources of information to encode task-specific posteriors. Inspired by the recent success of diffusion models as means of learning and using priors over images, our model uses dendritic nonlinearities optimized for denoising, and stochastic somatic integration with the degree of noise modulated by an oscillating global signal. Combining these elements into a recurrent network yields a stochastic dynamical system that samples from the prior at a rate prescribed by the period of the global oscillator. Additional inputs reflecting sensory or top-down contextual information alter these dynamics to generate samples from the corresponding posterior, with different input gating patterns selecting different inference tasks. We demonstrate that this architecture can sample from low dimensional nonlinear manifolds and multimodal posteriors. Overall, the model provides a new framework for circuit-level representation of probabilistic information, in a format that facilitates flexible inference.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {bioRxiv:2023.10.03.560701},
archivePrefix = {bioRriv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://openreview.net/pdf?id=S5aUhpuyap},
Journal = ICLR24,
year = {2024}
}

Download Citation (BibTex)

Previous Page
Viewing
Next Page