Search

What are you looking for?

  1. Two Speeds of Learning: A Representation-Readout Decomposition of Grokking and Double Descent

    Publication
    Training loss and accuracy are the standard signals used to monitor generalization during deep neural network training. Two well-documented phenomena complicate this picture: in grokking, train loss …

  2. Linear Readout of Neural Manifolds with Continuous Variables

    Publication
    Brains and artificial neural networks compute with continuous variables such as object position or stimulus orientation. However, the complex variability in neural responses makes it difficult to link …

  3. Diagnosing Generalization Failures from Representational Geometry Markers

    Publication
    Generalization, the ability to perform well beyond the training context, is a hallmark of biological and artificial intelligence, yet anticipating unseen failures remains a central challenge. Conventional approaches …

  4. Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry

    Publication
    Integrating task-relevant information into neural representations is a fundamental ability of both biological and artificial intelligence systems. Recent theories have categorized learning into two regimes: the rich …

  5. The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models

    Publication
    Decoder-only language models have the ability to dynamically switch between various computational tasks based on input prompts. Despite many successful applications of prompting, there is very limited …

  6. Geometry Linked to Untangling Efficiency Reveals Structure and Computation in Neural Populations

    Publication
    From an eagle spotting a fish in shimmering water to a scientist extracting patterns from noisy data, many cognitive tasks require untangling overlapping signals. Neural circuits achieve this …

  7. Nonlinear classification of neural manifolds with contextual information

    Publication
    Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learning. Recent approaches analyze the statistical and …

  8. Representational learning by optimization of neural manifolds in an olfactory memory network

    Publication
    Higher brain functions depend on experience-dependent representations of relevant information that may be organized by attractor dynamics or by geometrical modifications of continuous “neural manifolds”. To explore …

  9. Task-Relevant Covariance from Manifold Capacity Theory Improves Robustness in Deep Networks

    Publication
    Analysis of high-dimensional representations in neuroscience and deep learning traditionally places equal importance on all points in a representation, potentially leading to significant information loss. Recent advances …

  10. Neural Manifold Capacity Captures Representation Geometry, Correlations, and Task-Efficiency Across Species and Behaviors

    Publication
    The study of the brain encompasses multiple scales, including temporal, spatial, and functional aspects. To integrate understanding across these different levels and modalities, it requires developing quantification methods …

  11. Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds

    Publication
    In recent years, growth in our understanding of the computations performed in both biological and artificial neural networks has largely been driven by either low-level mechanistic studies …