CCN: Publications

Filter by Group

Computational Vision

Neural Circuits and Algorithms

133 Publications

Mapping models of V1 and V2 selectivity with local spectral reverse correlation

T. D. Oleskiw , Ramanujan Raghavan, Justin Lieber, E. P. Simoncelli, J. A. Movshon

Journal of Vision, 24: 1126-1126

Export Citation

September 15, 2024

Mapping models of V1 and V2 selectivity with local spectral reverse correlation

@article{Simoncelli_JOV_2024,
title = {{Mapping models of V1 and V2 selectivity with local spectral reverse correlation}},
author = {Oleskiw, Timothy and Raghavan, Ramanujan and Lieber, Justin and Simoncelli, Eero and Movshon, J. Anthony},
abstract = {Neurons in macaque area V2 respond selectively to higher-order visual features, such as the quasi-periodic structure of natural texture, but it is unknown how selectivity for these features is built from V1 inputs tuned more simply for orientation and spatial frequency. We have recently developed an image-computable two-layer linear-nonlinear network that captures higher-order tuning from a sparse combination of subunits tuned in orientation and scale. This model can be independently trained to predict data from single-unit recordings of V1 and V2 neurons from the awake macaque, responding to a stimulus set comprised of multiple superimposed grating patches that localize oriented contrast energy. These optimized models accurately predict neural responses to other stimuli, including gratings and synthetic textures with higher-order features common to natural images. However, the high-dimensional parameter space of these models makes them difficult to interpret. A systematic comparison of V1 and V2 selectivity is therefore elusive. To address this limitation, we investigate neural tuning across our population of models in silico using local spectral reverse correlation (LSRC; Nishimoto et al., 2006). Here, we present thousands of ternary white noise stimuli to fitted models, computing a response-weighted windowed frequency spectrum across image coordinates. LSRC can effectively characterize the tuning properties of our model neurons, thereby estimating the linear and nonlinear components of model receptive fields. These estimates are qualitatively similar to our direct LSRC measurements made using identical methods from single-unit V1 and V2 recordings collected in separate experiments.},
acrosscenters = {no},
external = {yes},
highlight = {no},
center = {CCN},
OpenAccessURL = {https://jov.arvojournals.org/article.aspx?articleid=2801954},
Journal = JOV,
year = {2024},
ISSN = {1534-7362},
month = Sep,
pages = {1126},
publisher = {Association for Research in Vision and Ophthalmology (ARVO)},
volume = {24},
number = {10},
doi = {10.1167/jov.24.10.1126},
pages = {1126}
}

Download Citation (BibTex)

Detecting Moving Objects During Self-motion

H. Lutwak, Bas Rokers, E. P. Simoncelli

Journal of Vision, 24: 761

Export Citation

2024

Detecting Moving Objects During Self-motion

@article{Simoncelli_JOV_2024,
title = {{Detecting Moving Objects During Self-motion}},
author = {Hope Lutwak and Bas Rokers and Eero Simoncelli},
abstract = {As we move through the world, the pattern of light projected on our eyes is complex and dynamic, yet we are still able to distinguish moving and stationary objects. One might hypothesize that this is achieved by detecting discontinuities in the spatial pattern of velocities, however this computation is also sensitive to velocity discontinuities at boundaries of stationary objects. We instead propose that humans make use of the specific constraints that self-motion imposes on retinal velocities. When an eye translates and rotates within a rigid 3D world, the velocity at each location on the retina is constrained to a line segment in the 2D space of retinal velocities (Longuet, Higgins, Prazdny 1980). The slope and intercept of this segment is determined by the eye’s translation and rotation, and the position along the segment is determined by depth of the scene. Since all possible velocities arising from a rigid world must lie on this segment, velocities not on the segment must correspond to moving objects. We hypothesize that humans make use of these constraints, by partially inferring self-motion based on the global pattern of retinal velocities, and using deviations of local velocity from the resulting constraint lines to detect moving objects. Using a head-mounted virtual reality device we simulated a translation forward in different virtual environments: one consisting of textured cubes above a textured ground plane, and one of scattered depth-matched dots. Participants had to determine if a cued cube/dot moved relative to the scene. Consistent with the hypothesis, we found that performance depended on the deviation of the object velocity from the constraint segment, not on the difference between retinal velocities of the object and its surround. Our findings contrast with previous inconclusive results, that relied on an impoverished stimulus with a limited field of view.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {},
archivePrefix = {},
year_posted = {},
center = {CCN},
OpenAccessURL = {http://dx.doi.org/10.1167/jov.24.10.761},
Journal = {Journal of Vision},
year = {2024},
pages = {761},
ISSN = {1534-7362},
volume = {24},
Number = {10},
month = {Sep},
publisher = {Association for Research in Vision and Ophthalmology (ARVO)}
DOI = {10.1167/jov.24.10.761}
}

Download Citation (BibTex)

Neuronal and behavioral responses to naturalistic texture images in macaque monkeys

C M Ziemba, R L T Goris, G M Stine, R K Perez, E. P. Simoncelli, J A Movshon

bioRxiv 2024.02.22.581645

The Journal of Neuroscience, 44: 42

Export Citation

2024

Neuronal and behavioral responses to naturalistic texture images in macaque monkeys

@ARTICLE{Ziemba24,
  TITLE=        "Neuronal and behavioral responses to naturalistic texture images in macaque monkeys",
  AUTHOR=       "C M Ziemba and R L T Goris and G M Stine and R K Perez and E P
                 Simoncelli and J A Movshon",
  JOURNAL=      "The Journal of Neuroscience",
  VOLUME=       44,
  ISSUE=        42,
  MONTH=        "Aug",
  YEAR=         "2024",
%  PDF-URL=     "https://www.cns.nyu.edu/pub/lcv/ziemba24-preprint.pdf",
  PDF-URL=      "https://www.cns.nyu.edu/pub/lcv/ziemba24.pdf",
  URL=          "https://www.jneurosci.org/content/early/2024/08/26/JNEUROSCI.0349-24.2024.long",
  DOI=          "10.1523/JNEUROSCI.0349-24.2024",
  PMID=         "39197942",
}

Download Citation (BibTex)

Open Data In Neurophysiology: Advancements, Solutions & Challenges

Colleen J Gillon, Cody Baker, Ryan Ly, E. Balzani, Bingni W Brunton, Manuel Schottdorf, Satrajit Ghosh, Noma Dehghani

PMCID: PMC11247910

Export Citation

July 1, 2024

Open Data In Neurophysiology: Advancements, Solutions & Challenges

@article{Gillon_Balzani_arXiv:2407.00976v1,
Archiveprefix = {arXiv},
Author = {Colleen J. Gillon and Cody Baker and Ryan Ly and Edoardo Balzani and Bingni W. Brunton and Manuel Schottdorf and Satrajit Ghosh and Nima Dehghani},
Eprint = {2407.00976},
Journal = {arXiv e-prints},
Title = {{Open Data In Neurophysiology: Advancements, Solutions \& Challenges.}},
OpenAccessURL = {https://pmc.ncbi.nlm.nih.gov/articles/PMC11247910/},
year_posted = {2024},
center = {CCN},
acrosscenters = {no},
external = {yes},
highlight = {no},
year = {2024},
abstract = {Across the life sciences, an ongoing effort over the last 50 years has made data and methods more reproducible and transparent. This openness has led to transformative insights and vastly accelerated scientific progress(1,2). For example, structural biology(3) and genomics(4,5) have undertaken systematic collection and publication of protein sequences and structures over the past half-century, and these data have led to scientific breakthroughs that were unthinkable when data collection first began (e.g.(6)). We believe that neuroscience is poised to follow the same path, and that principles of open data and open science will transform our understanding of the nervous system in ways that are impossible to predict at the moment. To this end, new social structures along with active and open scientific communities are essential(7) to facilitate and expand the still limited adoption of open science practices in our field(8). Unified by shared values of openness, we set out to organize a symposium for Open Data in Neuroscience (ODIN) to strengthen our community and facilitate transformative neuroscience research at large. In this report, we share what we learned during this first ODIN event. We also lay out plans for how to grow this movement, document emerging conversations, and propose a path toward a better and more transparent science of tomorrow.}
}

Download Citation (BibTex)

The neuron as a direct data-driven controller

J. Moore, A. Genkin, Magnus Tournoy, J. Pughe-Sanford, Rob R. de Ruyter van Steveninck, D. Chklovskii

arXiv:2401.01489

Proceedings of the National Academy of Sciences of the United States of America, 121: e2311893121

Export Citation

June 24, 2024

The neuron as a direct data-driven controller

@article{Moore_Chklovskii_arxiv:2401.01489_PNAS_2024,
title = {{The neuron as a direct data-driven controller}},
author = {Jason J. Moore  and Alexander Genkin  and Magnus Tournoy  and Joshua L. Pughe-Sanford  and Rob R. de Ruyter van Steveninck  and Dmitri B. Chklovskii},
acrosscenters = {no},
external = {yes},
abstract = {Building upon the efficient coding and predictive information theories, we present a perspective that neurons not only predict but may also actively influence their future inputs through their outputs. We model neurons as feedback controllers of their environments, a role traditionally considered computationally demanding, particularly when the dynamical system characterizing the environment is unknown. By harnessing an advanced data-driven control framework, we illustrate the feasibility of biological neurons functioning as effective feedback controllers. This innovative approach enables us to coherently explain various experimental findings that previously seemed unrelated. Our research has multiple potential implications, from the modeling of neuronal circuits to enabling biologically inspired artificial intelligence systems. In the quest to model neuronal function amid gaps in physiological data, a promising strategy is to develop a normative theory that interprets neuronal physiology as optimizing a computational objective. This study extends current normative models, which primarily optimize prediction, by conceptualizing neurons as optimal feedback controllers. We posit that neurons, especially those beyond early sensory areas, steer their environment toward a specific desired state through their output. This environment comprises both synaptically interlinked neurons and external motor sensory feedback loops, enabling neurons to evaluate the effectiveness of their control via synaptic feedback. To model neurons as biologically feasible controllers which implicitly identify loop dynamics, infer latent states, and optimize control we utilize the contemporary direct data-driven control (DD-DC) framework. Our DD-DC neuron model explains various neurophysiological phenomena: the shift from potentiation to depression in spike-timing-dependent plasticity with its asymmetry, the duration and adaptive nature of feedforward and feedback neuronal filters, the imprecision in spike generation under constant stimulation, and the characteristic operational variability and noise in the brain. Our model presents a significant departure from the traditional, feedforward, instant-response McCulloch–Pitts–Rosenblatt neuron, offering a modern, biologically informed fundamental unit for constructing neural networks.},
eprint = {2401.01489},
highlight = {no},
archivePrefix = {arXiv},
year_posted = {2024},
Journal = {Proceedings of the National Academy of Sciences of the United States of America},
center = {CCN},
OpenAccessURL = {https://www.pnas.org/doi/10.1073/pnas.2311893121},
year = {2024},
volume = {121},
number = {27},
pages = {e2311893121},
doi = {10.1073/pnas.2311893121}
}

Download Citation (BibTex)

Layerwise complexity-matched learning yields an improved model of cortical area {V2}

N. Parthasarathy, O J Henaff, E. P. Simoncelli

arXiv: 2312.11436

Trans. Machine Learning Research

Export Citation

2024

Layerwise complexity-matched learning yields an improved model of cortical area {V2}

@ARTICLE{Parthasarathy24,
  TITLE=        "Layerwise complexity-matched learning yields an improved model of
                 cortical area {V2}",
  AUTHOR=       "N Parthasarathy and O J H\'{e}naff and E P Simoncelli",
  JOURNAL=      "Trans. Machine Learning Research",
  MONTH=        "Jun",
  YEAR=         2024,
  PUBLISHER=    "https://jmlr.org/tmlr/index.html",
  PDF-URL=      "https://www.cns.nyu.edu/pub/lcv/parthasarathy24.pdf",
  URL=          "https://openreview.net/forum?id=lQBsLfAWhj",
  NOTE=         "Featured Certification",
}

Download Citation (BibTex)

A unified framework for perceived magnitude and dicriminability of sensory stimuli

J Y Zhou, L R Duong, E. P. Simoncelli

Proceedings of the National Academy of Sciences of the United States of America, 121: 25

Export Citation

2024

A unified framework for perceived magnitude and dicriminability of sensory stimuli

@ARTICLE{Zhou24,
  TITLE=        "A unified framework for perceived magnitude and dicriminability of
                 sensory stimuli",
  AUTHOR=       "J Y Zhou and L R Duong and E P Simoncelli",
  JOURNAL=      "Proc. Nat'l Academy of Sciences",
  VOLUME=       "121",
  NUMBER=       "25",
  MONTH=        "Jun",
  YEAR=         2024,
  DOI=          "10.1073/pnas.2312293121",
  URL=          "https://www.pnas.org/doi/10.1073/pnas.2312293121",
  PDF-URL=      "https://www.cns.nyu.edu/pub/lcv/zhou24.pdf",
}

Download Citation (BibTex)

WASPSYN: A Challenge for Domain Adaptive Synapse Detection in Microwasp Brain Connectomes

Yicong Li, Wanhua Li, Qi chen, Wei Huang, Yuda Zou, Xin Xiao, K. Shinomiya, P. Gunn, Nishika Gupta, Alexey Polilov, Yongchao Xu, Yueyi Zhang, Zhiwei Xiong, Hanspeter Pfister, Donglai Wei, J. Wu

IEEE Transactions on Medical Imaging, 43: 3719-3730

Export Citation

May 13, 2024

WASPSYN: A Challenge for Domain Adaptive Synapse Detection in Microwasp Brain Connectomes

@article{Shinomiya_Wu_arxiv:2302.00545_IEEEMI_2024,
title = {{WASPSYN: A Challenge for Domain Adaptive Synapse Detection in Microwasp Brain Connectomes}},
author = {Li, Yicong and Li, Wanhua and Chen, Qi and Huang, Wei and Zou, Yuda and Xiao, Xin and Shinomiya, Kazunori and Gunn, Pat and Gupta, Nishika and Polilov, Alexey and Xu, Yongchao and Zhang, Yueyi and Xiong, Zhiwei and Pfister, Hanspeter and Wei, Donglai and Wu, Jingpeng},
acrosscenters = {no},
external = {yes},
abstract = {The size of image volumes in connectomics studies now reaches terabyte and often petabyte scales with a great diversity of appearance due to different sample preparation procedures. However, manual annotation of neuronal structures (e.g., synapses) in these huge image volumes is time-consuming, leading to limited labeled training data often smaller than 0.001% of the large-scale image volumes in application. Methods that can utilize in-domain labeled data and generalize to out-of-domain unlabeled data are in urgent need. Although many domain adaptation approaches are proposed to address such issues in the natural image domain, few of them have been evaluated on connectomics data due to a lack of domain adaptation benchmarks. Therefore, to enable developments of domain adaptive synapse detection methods for large-scale connectomics applications, we annotated 14 image volumes from a biologically diverse set of Megaphragma viggianii brain regions originating from three different whole-brain datasets and organized the WASPSYN challenge at ISBI 2023. The annotations include coordinates of pre-synapses and post-synapses in the 3D space, together with their one-to-many connectivity information. This paper describes the dataset, the tasks, the proposed baseline, the evaluation method, and the results of the challenge. Limitations of the challenge and the impact on neuroscience research are also discussed. The challenge is and will continue to be available at https://codalab.lisn.upsaclay.fr/competitions/9169. Successful algorithms that emerge from our challenge may potentially revolutionize real-world connectomics research and further the cause that aims to unravel the complexity of brain structure and function.},
eprint = {2302.00545},
highlight = {no},
archivePrefix = {arXiv},
year_posted = {2023},
Journal = {IEEE Transactions on Medical Imaging},
center = {CCN},
OpenAccessURL = {https://ieeexplore.ieee.org/document/10530168/authors#authors},
year = {2024},
volume={43},
number={11},
pages={3719-3730},
doi={10.1109/TMI.2024.3400276}
}

Download Citation (BibTex)

Nonlinear classification of neural manifolds with contextual information

F. Mignacco, C. Chou , S. Chung

arXiv:2405.06851

Export Citation

May 10, 2024

Nonlinear classification of neural manifolds with contextual information

@article{Mignacco_Chung_arXiv:2405.06851,
archivePrefix={arXiv},
author={Francesca Mignacco and Chi-Ning Chou and SueYeon Chung},
eprint={2405.06851},
journal = {arXiv e-prints},
title={{Nonlinear classification of neural manifolds with contextual information}},
OpenAccessUrl={https://arxiv.org/abs/2405.06851},
year_posted={2024},
Center = {CCN},
accrosscenters = {no},
external = {yes},
highlight = {no},
year = {2024},
abstract = {Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learning. Recent approaches analyze the statistical and geometrical attributes of neural representations as population-level mechanistic descriptors of task implementation. In particular, manifold capacity has emerged as a promising framework linking population geometry to the separability of neural manifolds. However, this metric has been limited to linear readouts. Here, we propose a theoretical framework that overcomes this limitation by leveraging contextual input information. We derive an exact formula for the context-dependent capacity that depends on manifold geometry and context correlations, and validate it on synthetic and real data. Our framework's increased expressivity captures representation untanglement in deep networks at early stages of the layer hierarchy, previously inaccessible to analysis. As context-dependent nonlinearity is ubiquitous in neural systems, our data-driven and theoretically grounded approach promises to elucidate context-dependent computation across scales, datasets, and models.}
}

Download Citation (BibTex)

A unified framework for perceived magnitude and discriminability of sensory stimuli

J. Zhou, L. Duong, E. P. Simoncelli

bioXriv:2022.04.30.490146v3

Proceedings of the National Academy of Sciences of the United States of America, 121: e2312293121

Export Citation

April 25, 2024

A unified framework for perceived magnitude and discriminability of sensory stimuli

@article{Zhou_Simoncelli_biorxiv:2022.04.30.490146_PNAS_2024,
title = {{A unified framework for perceived magnitude and discriminability of sensory stimuli}},
author = {Zhou,  Jingyang and Duong,  Lyndon R. and Simoncelli,  Eero P.},
abstract = {The perception of sensory attributes is often quantified through measurements of sensitivity (the ability to detect small stimulus changes), as well as through direct judgements of appearance or intensity. Despite their ubiquity, the relationship between these two measurements remains controversial and unresolved. Here, we propose a framework in which they arise from different aspects of a common representation. Specifically, we assume that judgements of stimulus intensity (e.g., as measured through rating scales) reflect the mean value of an internal representation, and sensitivity reflects a combination of mean value and noise properties, as quantified by the statistical measure of Fisher Information. Unique identification of these internal representation properties can be achieved by combining measurements of sensitivity and judgments of intensity. As a central example, we show that Weber{\textquoteright}s law of perceptual sensitivity can co-exist with Stevens{\textquoteright} power-law scaling of intensity ratings (for all exponents), when the noise amplitude increases in proportion to the representational mean. We then extend this result beyond the Weber{\textquoteright}s law range by incorporating a more general and physiology-inspired form of noise, and show that the combination of noise properties and sensitivity measurements accurately predicts intensity ratings across a variety of sensory modalities and attributes. Our framework unifies two primary perceptual measurements {\textendash} thresholds for sensitivity and rating scales for intensity {\textendash} and provides a neural interpretation for the underlying representation.Significance Statement Perceptual measurements of sensitivity to stimulus changes and stimulus appearance (intensity) are ubiquitous in the study of perception. However, the relationship between these two seemingly disparate measurements remains unclear. Proposals for unification have been made for over 60 years, but they generally lack support from perceptual or physiological measurements. Here, we provide a framework that offers a unified interpretation of perceptual sensitivity and intensity measurements, and we demonstrate its consistency with experimental measurements across multiple perceptual domains.Competing Interest StatementThe authors have declared no competing interest.},
acrosscenters = {no},
external = {yes},
highlight = {no},
eprint = {https://www.biorxiv.org/content/early/2024/05/29/2022.04.30.490146.full.pdf},
archivePrefix = {bioRxiv},
year_posted = {2023},
center = {CCN},
OpenAccessURL = {https://www.pnas.org/doi/10.1073/pnas.2312293121},
Journal = {Proceedings of the National Academy of Sciences of the United States of America},
volume = {121},
ISSN = {1091-6490},
month = jun,
DOI = {10.1073/pnas.2312293121},
number = {25},
publisher = {Proceedings of the National Academy of Sciences},
year = {2024}
}

Download Citation (BibTex)

Previous Page
Viewing
Next Page