CCN: Publications

Clear Filters

Filter by Group

Computational Vision

Neural Circuits and Algorithms

NeuroAI and Geometric Data Analysis

NeuroRSE

Statistical Analysis of Neural Data

220 Publications

Equivalence between representational similarity analysis, centered kernel alignment, and canonical correlations analysis

A. Williams

bioXriv:2024.10.23.619871

UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models, 2024

Export Citation

October 10, 2024

Equivalence between representational similarity analysis, centered kernel alignment, and canonical correlations analysis

"@article{Williams_biorxiv:2024.10.23.619871_UR2_2024,
title = {{Equivalence between representational similarity analysis, centered kernel alignment, and canonical correlations analysis}},
author = {Williams, Alex H.},
abstract = {Centered kernel alignment (CKA) and representational similarity analysis (RSA) of dissimilarity matrices are two popular methods for comparing neural systems in terms of representational geometry. Although they follow a conceptually similar approach, typical implementations of CKA and RSA tend to result in numerically different outcomes. Here, I show that these two approaches are largely equivalent once one incorporates a mean-centering step into RSA. This equivalence holds for both linear and nonlinear variants of these methods. These connections are simple to derive, but appear to have been thus far overlooked in the context of comparing neural representations. By unifying these measures, this paper hopes to simplify a complex and fragmented literature on this subject.Competing Interest StatementThe authors have declared no competing interest.},
acrosscenters = {no},
external = {no},
highlight = {no},
eprint = {https://www.biorxiv.org/content/early/2024/10/24/2024.10.23.619871.full.pdf},
archivePrefix = {bioRxiv},
year_posted = {2024},
Journal = {UR2},
center = {CCN},
OpenAccessURL = {https://www.biorxiv.org/content/early/2024/10/24/2024.10.23.619871},
year = {2024},
doi = {10.1101/2024.10.23.619871},
publisher = {Cold Spring Harbor Laboratory}
}"

Download Citation (BibTex)

What Representational Similarity Measures Imply about Decodable Information

S. Harvey, D. Lipshutz, A. Williams

arXiv:2411.08197

UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models

Export Citation

October 10, 2024

What Representational Similarity Measures Imply about Decodable Information

@article{Harvey_Williams_arXiv:2411.08197_UR2_2024,
title = {{What Representational Similarity Measures Imply about Decodable Information}},
author = {Sarah E. Harvey and David Lipshutz and Alex H. Williams},
year_posted = {2024},
Journal = UR2,
center = {CCN},
acrosscenters = {no},
external = {no},
highlight = {no},
year = {2024},
abstract = {Neural responses encode information that is useful for a variety of downstream tasks. A common approach to understand these systems is to build regression models or ``decoders'' that reconstruct features of the stimulus from neural responses. Popular neural network similarity measures like centered kernel alignment (CKA), canonical correlation analysis (CCA), and Procrustes shape distance, do not explicitly leverage this perspective and instead highlight geometric invariances to orthogonal or affine transformations when comparing representations. Here, we show that many of these measures can, in fact, be equivalently motivated from a decoding perspective. Specifically, measures like CKA and CCA quantify the average alignment between optimal linear readouts across a distribution of decoding tasks. We also show that the Procrustes shape distance upper bounds the distance between optimal linear readouts and that the converse holds for representations with low participation ratio. Overall, our work demonstrates a tight link between the geometry of neural representations and the ability to linearly decode information. This perspective suggests new ways of measuring similarity between neural systems and also provides novel, unifying interpretations of existing measures.},
archivePrefix = {arXiv},
eprint = {2411.08197},
OpenAccessURL = {https://arxiv.org/abs/2411.08197}
}

Download Citation (BibTex)

Classification-Denoising Networks

Louis Thiry, F. Guth

arXiv:2410.03505

Export Citation

October 4, 2024

Classification-Denoising Networks

@article{Thiry_Guth_arXiv:2410.03505,
      title={{Classification-Denoising Networks}},
      author={Louis Thiry and Florentin Guth},
      year={2024},
    publisher={Cornell Tech},
    journal = {arXiv e-prints},
    doi={10.48550/arXiv.2410.03505},
    abstract={Image classification and denoising suffer from complementary issues of lack of robustness or partially ignoring conditioning information. We argue that they can be alleviated by unifying both tasks through a model of the joint probability of (noisy) images and class labels. Classification is performed with a forward pass followed by conditioning. Using the Tweedie-Miyasawa formula, we evaluate the denoising function with the score, which can be computed by marginalization and back-propagation. The training objective is then a combination of cross-entropy loss and denoising score matching loss integrated over noise levels. Numerical experiments on CIFAR-10 and ImageNet show competitive classification and denoising performance compared to reference deep convolutional classifiers/denoisers, and significantly improves efficiency compared to previous joint approaches. Our model shows an increased robustness to adversarial perturbations compared to a standard discriminative classifier, and allows for a novel interpretation of adversarial gradients as a difference of denoisers.},
      eprint={2410.03505},
      archivePrefix={arXiv},
      OpenAccessUrl={https://arxiv.org/abs/2410.03505}
}

Download Citation (BibTex)

A Rainbow in Deep Network Black Boxes

F. Guth, Brice Ménard, MD, Gaspar Rochette, S. Mallat

arXiv:2305.18512

Journal of Machine Learning Research, 25: 1-59

Export Citation

2024

A Rainbow in Deep Network Black Boxes

@article{Guth_Mallat_arxiv:2305.18512_JMLR_2024,
      title={{A Rainbow in Deep Network Black Boxes}},
      author={\text{Guth, Florentin} and M\'{e}nard, Brice and Rochette, Gaspar and \text{Mallat, St\'{{e}}phane}},
doi={10.48550/arXiv.2305.18512},
Journal = {Journal of Machine Learning Research},
Publisher= {Journal of Machine Learning Research},
abstract={A central question in deep learning is to understand the functions learned by deep networks. What is their approximation class? Do the learned weights and representations depend on initialization? Previous empirical work has evidenced that kernels defined by network acti- vations are similar across initializations. For shallow networks, this has been theoretically studied with random feature models, but an extension to deep networks has remained elu- sive. Here, we provide a deep extension of such random feature models, which we call the rainbow model. We prove that rainbow networks define deterministic (hierarchical) kernels in the infinite-width limit. The resulting functions thus belong to a data-dependent RKHS which does not depend on the weight randomness. We also verify numerically our mod- eling assumptions on deep CNNs trained on image classification tasks, and show that the trained networks approximately satisfy the rainbow hypothesis. In particular, rainbow net- works sampled from the corresponding random feature model achieve similar performance as the trained networks. Our results highlight the central role played by the covariances of network weights at each layer, which are observed to be low-rank as a result of feature learning.},
    Volume={25},
Issue={350},
year_posted={2023},
year={2024},
month={Oct},
      eprint={2305.18512},
pages={1-59},
     url= {http://jmlr.org/papers/v25/23-1573.html},
      archivePrefix={arXiv},
      Open Access Url={https://arxiv.org/abs/2305.18512}, center={CCN}, acrosscenters={yes}, external={yes}, highlight={no}
}

Download Citation (BibTex)

Vocal Call Locator Benchmark (VCL) for localizing rodent vocalizations from multi-channel audio

R. Peterson, A. Tanelus, Christopher Ick, Bartul Mimica, Niegil Francis, Violet J Ivan, A. Choudhri, A. Falkner, M. Murthy, David M Schneider, Dan Sanes, A. Williams

bioRxiv:2024.09.20.613758

The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

Export Citation

2024

Vocal Call Locator Benchmark (VCL) for localizing rodent vocalizations from multi-channel audio

@article {Peterson_Tanelus_Williams_biorxiv:2024.09.20.613758_NEURIPS24_2024,
author = {Peterson, Ralph E and Tanelus, Aramis and Ick, Christopher and Mimica, Bartul and Francis, Niegil and Ivan, Violet J and Choudhri, Aman and Falkner, Annegret L and Murthy, Mala and Schneider, David M and Sanes, Dan H and Williams, Alex H.},
title = {{Vocal Call Locator Benchmark (VCL) for localizing rodent vocalizations from multi-channel audio}},
year = {2024},
doi = {10.1101/2024.09.20.613758},
journal = NEURIPS24,
publisher = {Cold Spring Harbor Laboratory},
abstract = {Understanding the behavioral and neural dynamics of social interactions is a goal of contemporary neuroscience. Many machine learning methods have emerged in recent years to make sense of complex video and neurophysiological data that result from these experiments. Less focus has been placed on understanding how animals process acoustic information, including social vocalizations. A critical step to bridge this gap is determining the senders and receivers of acoustic information in social interactions. While sound source localization (SSL) is a classic problem in signal processing, existing approaches are limited in their ability to localize animal-generated sounds in standard laboratory environments. Advances in deep learning methods for SSL are likely to help address these limitations, however there are currently no publicly available models, datasets, or benchmarks to systematically evaluate SSL algorithms in the domain of bioacoustics. Here, we present the VCL Benchmark: the first large-scale dataset for benchmarking SSL algorithms in rodents. We acquired synchronized video and multi-channel audio recordings of 767,295 sounds with annotated ground truth sources across 9 conditions. The dataset provides benchmarks which evaluate SSL performance on real data, simulated acoustic data, and a mixture of real and simulated data. We intend for this benchmark to facilitate knowledge transfer between the neuroscience and acoustic machine learning communities, which have had limited overlap.Competing Interest StatementThe authors have declared no competing interest.},
eprint = {https://www.biorxiv.org/content/early/2024/09/21/2024.09.20.613758.full.pdf},
URL = {https://proceedings.neurips.cc/paper_files/paper/2024/hash/c00d37d6b04d73b870b963a4d70051c1-Abstract-Datasets_and_Benchmarks_Track.html},
}

Download Citation (BibTex)

Perpetual step-like restructuring of hippocampal circuit dynamics

S. Zheng, Roman Huszár, Thomas Hainmueller, Marlene Bartos, A. Williams, György Buzsáki

bioRxiv:2024.04.22.590576v1

Cell Reports, 43:114702

Export Citation

September 24, 2024

Perpetual step-like restructuring of hippocampal circuit dynamics

@article{Zheng_WIlliams_CRS_2024.114702,
Archiveprefix = {CRS},
Author = {Zheng, Zheyang Sam and Huszár, Roman and Hainmueller, Thomas and Bartos, Marlene and \text{Williams, Alex H.} and Buzsáki, György},
Doi = {10.1016/j.celrep.2024.114702},
Eprint = {doi: 10.1016/j.celrep.2024.114702},
Journal = CRS,
Issue = {9},
month = {Sep},
Pages = {114702},
Publisher = {Elsevier},
Title = {{Perpetual step-like restructuring of hippocampal circuit dynamics}},
Url = {https://doi.org/10.1016/j.celrep.2024.114702},
OpenAccessURL = {https://doi.org/10.1016/j.celrep.2024.114702},
Volume = {43},
Year = {2024},
year_posted = {2024},
center = {CCN},
acrosscenters = {no},
external = {yes},
highlight = {no},
abstract = {Representation of the environment by hippocampal populations is known to drift even within a familiar environment, which could reflect gradual changes in single-cell activity or result from averaging across discrete switches of single neurons. Disambiguating these possibilities is crucial, as they each imply distinct mechanisms. Leveraging change point detection and model comparison, we find that CA1 population vectors decorrelate gradually within a session. In contrast, individual neurons exhibit predominantly step-like emergence and disappearance of place fields or sustained changes in within-field firing. The changes are not restricted to particular parts of the maze or trials and do not require apparent behavioral changes. The same place fields emerge, disappear, and reappear across days, suggesting that the hippocampus reuses pre-existing assemblies, rather than forming new fields de novo. Our results suggest an internally driven perpetual step-like reorganization of the neuronal assemblies.}
}

Download Citation (BibTex)

Mapping models of V1 and V2 selectivity with local spectral reverse correlation

T. D. Oleskiw , Ramanujan Raghavan, Justin Lieber, E. P. Simoncelli, J. A. Movshon

Journal of Vision, 24: 1126-1126

Export Citation

September 15, 2024

Mapping models of V1 and V2 selectivity with local spectral reverse correlation

@article{Simoncelli_JOV_2024,
title = {{Mapping models of V1 and V2 selectivity with local spectral reverse correlation}},
author = {Oleskiw, Timothy and Raghavan, Ramanujan and Lieber, Justin and Simoncelli, Eero and Movshon, J. Anthony},
abstract = {Neurons in macaque area V2 respond selectively to higher-order visual features, such as the quasi-periodic structure of natural texture, but it is unknown how selectivity for these features is built from V1 inputs tuned more simply for orientation and spatial frequency. We have recently developed an image-computable two-layer linear-nonlinear network that captures higher-order tuning from a sparse combination of subunits tuned in orientation and scale. This model can be independently trained to predict data from single-unit recordings of V1 and V2 neurons from the awake macaque, responding to a stimulus set comprised of multiple superimposed grating patches that localize oriented contrast energy. These optimized models accurately predict neural responses to other stimuli, including gratings and synthetic textures with higher-order features common to natural images. However, the high-dimensional parameter space of these models makes them difficult to interpret. A systematic comparison of V1 and V2 selectivity is therefore elusive. To address this limitation, we investigate neural tuning across our population of models in silico using local spectral reverse correlation (LSRC; Nishimoto et al., 2006). Here, we present thousands of ternary white noise stimuli to fitted models, computing a response-weighted windowed frequency spectrum across image coordinates. LSRC can effectively characterize the tuning properties of our model neurons, thereby estimating the linear and nonlinear components of model receptive fields. These estimates are qualitatively similar to our direct LSRC measurements made using identical methods from single-unit V1 and V2 recordings collected in separate experiments.},
acrosscenters = {no},
external = {yes},
highlight = {no},
center = {CCN},
OpenAccessURL = {https://jov.arvojournals.org/article.aspx?articleid=2801954},
Journal = JOV,
year = {2024},
ISSN = {1534-7362},
month = Sep,
pages = {1126},
publisher = {Association for Research in Vision and Ophthalmology (ARVO)},
volume = {24},
number = {10},
doi = {10.1167/jov.24.10.1126},
pages = {1126}
}

Download Citation (BibTex)

Fixational eye movements enhance the precision of visual information transmitted by the primate retina

Eric G. Wu, Nora Brackbill, Colleen Rhoades, Alexandra Kling, Alex R. Gogliettino, Nishal P. Shah, Alexander Sher, Alan M. Litke, E. P. Simoncelli, E. J. Chichilnisky

bioRxiv:2023.08.12.552902

Nature Communications, 15: 7964

Export Citation

September 11, 2024

Fixational eye movements enhance the precision of visual information transmitted by the primate retina

@article{Wu_Simoncelli_biorxiv:08.12.552902_NATCOM_2024,
title = {{Fixational eye movements enhance the precision of visual information transmitted by the primate retina}},
author = {Wu, Eric G. and Brackbill, Nora and Rhoades, Colleen and Kling, Alexandra and Gogliettino, Alex R. and Shah, Nishal P. and Sher, Alexander and Litke, Alan M. and Simoncelli, Eero P. and Chichilnisky, E. J.},
abstract = {Fixational eye movements alter the number and timing of spikes transmitted from the retina to the brain, but whether these changes enhance or degrade the retinal signal is unclear. To quantify this, we developed a Bayesian method for reconstructing natural images from the recorded spikes of hundreds of retinal ganglion cells (RGCs) in the macaque retina (male), combining a likelihood model for RGC light responses with the natural image prior implicitly embedded in an artificial neural network optimized for denoising. The method matched or surpassed the performance of previous reconstruction algorithms, and provides an interpretable framework for characterizing the retinal signal. Reconstructions were improved with artificial stimulus jitter that emulated fixational eye movements, even when the eye movement trajectory was assumed to be unknown and had to be inferred from retinal spikes. Reconstructions were degraded by small artificial perturbations of spike times, revealing more precise temporal encoding than suggested by previous studies. Finally, reconstructions were substantially degraded when derived from a model that ignored cell-to-cell interactions, indicating the importance of stimulus-evoked correlations. Thus, fixational eye movements enhance the precision of the retinal representation.},
acrosscenters = {no},
external = {yes},
highlight = {no},
year_posted = {2024},
center = {CCN},
Archiveprefix = {bioRxiv},
doi = {10.1038/s41467-024-52304-7},
Eprint = {10.1101/2023.08.12.552902},
OpenAccessURL = {https://www.nature.com/articles/s41467-024-52304-7},
Journal = NATCOM,
Volume = {15},
pages = {7964},
year = {2024}
}

Download Citation (BibTex)

Geometric changes in monkey V4 and IT neural responses during visual category learning

J. Feather, Long Sha, Gouki Okazawa, N. Yu Lo, S. Chung , R. Kiani

Journal of Vision, 24: 1430

Export Citation

September 1, 2024

Geometric changes in monkey V4 and IT neural responses during visual category learning

@article{Feather_Chung_JOV_2024,
title = {{Geometric changes in monkey V4 and IT neural responses during visual category learning}},
author = {Feather,  Jenelle and Sha,  Long and Okazawa,  Gouki and Lo,  Nga Yu and Chung,  SueYeon and Kiani,  Roozbeh},
abstract = {Behavior changes over the course of learning a task. This behavioral change is due to shifts in neural responses that support improved performance. Here, we investigated how the underlying representational geometries in visual areas V4 and IT of the Macaque visual system change during a categorization learning task. Visual stimuli varied in two independent attributes, and monkeys learned to categorize them based on a category boundary in the stimulus space that was defined by a combination of the attributes. Chronic neural population recordings were obtained from V4 and IT over multiple days of training while a monkey learned the task through receiving correct/incorrect feedback. Additionally, we recorded from the same neural populations while a monkey performed a fixation task viewing the same sets of stimuli. In all eight analyzed tasks, the monkey’s performance on the categorization task improved with training. To link this behavioral improvement to the underlying population responses, we investigated how the geometry of neural population activity changed over the course of learning. We treated population responses to all stimuli in each of the two categories as manifold-like representations, and analyzed the geometric properties of these representations using mean-field theoretic manifold capacity analysis. As the monkey learned the task, we observed that the representations in both V4 and IT for the two classes became more separable as measured by an increase in manifold capacity. This increase in capacity was associated with a characteristic geometric change in the neural population response geometry. Our results suggest that both V4 and IT responses actively change during category learning in ways that directly lead to increased separability and improved readouts for downstream neural areas, and point towards future work linking these population-level geometric changes to local changes at the single-neuron level.},
acrosscenters = {no},
external = {yes},
highlight = {no},
center = {CCN},
OpenAccessURL = {https://jov.arvojournals.org/article.aspx?articleid=2801667},
Journal = JOV,
volume = {24},
ISSN = {1534-7362},
DOI = {10.1167/jov.24.10.1430},
number = {10},
publisher = {Association for Research in Vision and Ophthalmology (ARVO)},
month = sep,
pages = {1430},
year = {2024}
}

Download Citation (BibTex)

Visual Temporal Prediction: Representation, Estimation, and Modeling

P. Fiquet

New York University ProQuest Dissertations & Theses, 31489428

Export Citation

September 1, 2024

Visual Temporal Prediction: Representation, Estimation, and Modeling

@PHDTHESIS{Fiquet-phd,
     TITLE= "Visual Temporal Prediction: {Representation}, Estimation, and Modeling",
     AUTHOR= "Pierre-Etienne Fiquet",
     ADVISOR= "Eero P. Simoncelli",
     SCHOOL= "Center for Neural Science, New York University",
     ADDRESS= "New York, NY",
     YEAR= 2024,
     MONTH= "Sep",
     PDF-URL= "https://www.cns.nyu.edu/pub/lcv/fiquet-phd.pdf",
}

Download Citation (BibTex)

Previous Page
Viewing
Next Page