Vikram Gadagkar is a postdoctoral fellow with Jesse Goldberg at the Department of Neurobiology and Behavior, Cornell University. He has a B.S. in physics, chemistry and mathematics from Bangalore University, an M.S. in physics from the Indian Institute of Science, and a Ph.D. in physics from Cornell University. For his graduate research, he worked with Séamus Davis to investigate the existence of a new state of matter – the putative supersolid state. Solid helium taken close to absolute zero of temperature was proposed to be a supersolid, simultaneously a solid and a superfluid. Using a Superconducting Quantum Interference Device (SQUID)-based torsional oscillator and ultra-low vibration techniques, Gadagkar and his colleagues demonstrated that solid helium, while exhibiting many interesting properties, is not a supersolid. He then developed a model based on lattice defects (dislocations) to explain the experimental observations. During his graduate years, he became interested in how networks of neurons in the brain produce behavior, akin to how networks of dislocations produce emergent physical phenomena. He found a kindred spirit in Jesse Goldberg, who also saw computation as the key intermediate link between neural circuits and behavior. After graduating, he teamed up with Jesse to help set up a brand-new systems neuroscience lab at Cornell. Using a combination of awake-behaving electrophysiology, advanced cellular-resolution imaging and network models, Gadagkar aims to identify computational principles underlying trial-and-error learning. Dopamine neurons are known to mediate trial-and-error learning in animals seeking rewards by encoding reward prediction error – they are activated by better-than-predicted reward outcomes and suppressed by worse-than-predicted ones. Do dopamine neurons also play a role in learning to speak or play an instrument, skills that are not learned for external rewards? He is investigating the generality of neural-reinforcement mechanisms by recording dopamine neurons in singing birds to test if performance-prediction error is encoded like reward-prediction error.
“How is Performance Evaluation Encoded in the Brain?”
How do organisms associate actions with rewards or punishments? In classic experiments from neuroscience, rats explore an enclosure that has a lever located on one of its walls. When the lever is pressed, a food pellet is dispensed. Soon, through trial and error, the rat learns to press the lever and get the food. In this simple case, the action-reward relationship is clear: press the lever; get the food. But what about in cases in which the reward isn’t so immediately clear, such as learning to speak or play a musical instrument? These skills are not learned for immediate rewards but instead by matching ongoing performance to internal goals. Neuroscientists have discovered a possible mechanism for simple reward learning, like the rat in the box. In the simple model, neurons from a brain area called the “ventral tegmental area,” or VTA, release the neurotransmitter dopamine in response to a reward. The twist is that these neurons don’t just fire in response to the reward itself, but rather encode “reward prediction error.” This means that the dopamine neurons are activated by better-than-predicted rewards and suppressed by worse-than-predicted rewards. It’s this error signal that appears to drive trial-and-error learning. But whether this error signal exists, and how it might be generated in more complex learning such as speech or music, is unclear. We study this question in the songbird, which learns to sing by trial and error. “Performance prediction error” in this case is defined by how much the young bird’s song matches the song it has heard from older birds. To compute this error, the songbird must compare its own motor performance (singing) to auditory feedback (how close the song sounds to the ideal version). We record the activity of dopamine neurons in the songbird’s VTA while we distort syllables in the bird’s auditory feedback. Remarkably, we find that these VTA neurons are suppressed immediately after distortions, indicating a worse-than-predicted outcome, and activated at the precise moment in the song when a predicted distortion would have occurred but did not, indicating a better-than-predicted outcome. This one-dimensional error signal (i.e., better or worse performance) must be computed from auditory evaluation of a complex, high-dimensional motor output (i.e., the song). To investigate how this signal is produced, we will image populations of dopamine neurons and their inputs at cellular resolution. Our work will advance our understanding of learning beyond just a handful of dopamine neurons in the songbird, offering insights into how distributed activity across many brain regions computes the performance prediction error signal fundamental to learning complex skills.