What Do Animals Really Learn? Adventures of Reinforcement Learning in the Real World

Animals and humans alike can quickly learn to associate different stimuli in the environment with rewards or punishments. In recent years, ideas from the computational field of reinforcement learning have revolutionized the study of learning in the brain, providing new, precise theories of how such associations are formed. However, although these learning algorithms work well in simplified laboratory scenarios, they are known to suffer from the “curse of dimensionality” that makes learning in complex, multidimensional scenarios infeasible. How does the brain scale reinforcement learning to realistic tasks?

In this lecture, Yael Niv will argue that the key to learning efficiently in real-world scenarios is to use a simplified representation of the task that includes only those dimensions of the environment that are relevant to obtaining reward. This, however, raises the new question of how such task representations are learned. She will first demonstrate, using behavioral experiments, that animals and humans learn the causal, often hidden structure of a task, thus forming a concise task representation through experience. Dr. Niv will then suggest that these task representations reside in the orbitofrontal cortex, and show how we can visualize these mental maps of task space and how these maps are related to behavioral performance.