Come back soon for a full list of talks and abstracts!
(This list updates dynamically)
(This list updates dynamically)
Lin Tian
University of California, Davis
Multiplex interrogation of dopamine signaling with genetically encoded indicators
In this talk, I will discuss our recent progress into develop and apply a color-palette of genetically encoded indicators of neuromodulators, including dopamine, norepinephrine and serotonin. In combination with calcium, glutamate imaging and optogenetics, these sensors are well poised to permit direct functional analysis of how the spatiotemporal coding of neural input signaling mediates the plasticity and function of target circuits.
University of California, Davis
Multiplex interrogation of dopamine signaling with genetically encoded indicators
In this talk, I will discuss our recent progress into develop and apply a color-palette of genetically encoded indicators of neuromodulators, including dopamine, norepinephrine and serotonin. In combination with calcium, glutamate imaging and optogenetics, these sensors are well poised to permit direct functional analysis of how the spatiotemporal coding of neural input signaling mediates the plasticity and function of target circuits.
Louis-Eric Trudeau
University of Montreal
Exploring the synaptic and non-synaptic connectivity of dopamine neurons
Dopamine and other neuromodulatory neurons in the brain appear to be quite distinct in their connectivity compared to the larger contingent of more classic, fast-signaling glutamate and GABA neurons. Dopamine neurons are endowed with a particularly broad and arborized axonal domain containing an extremely large number of release sites. The discovery in the 1980s that most axonal varicosities established by these neurons do not display a classical synaptic structure is a long-standing mystery. We have been recently exploring the topography of dopaminergic release sites using a co-culture system of postnatal mouse dopamine neurons together with striatal medium spiny neurons. We find that contrarily to cortical or striatal neurons, dopamine neurons have a surprising propensity to establish axonal varicosities that mostly ignore target cells. These axonal varicosities contain classical vesicular proteins such as synaptotagmin 1 and VMAT2. Only a very small subset of dopaminergic terminals are in close proximity to postsynaptic domains containing dopamine D2 receptors or postsynaptic organizers such as PSD95 or gephyrin. We find this small subset of “synaptic” release sites can be distinguished from the larger subset of “non-synaptic” terminals by their differential expression of active zone proteins. In stark contrast, the vast majority of release sites established by cortical or striatal neurons establish tight links with postsynaptic domains. Whether synaptic or not in structure, we find that in this co-culture system, most dopaminergic axonal varicosities appear to be functional as revealed by the activity-dependent uptake and release of synaptic vesicle recycling markers. A better understanding of the unique connectivity of neuromodulatory neurons is bound to provide new insights into their normal physiological roles and their impairment and vulnerability is diseases such as Parkinson’s, drug abuse and schizophrenia.
University of Montreal
Exploring the synaptic and non-synaptic connectivity of dopamine neurons
Dopamine and other neuromodulatory neurons in the brain appear to be quite distinct in their connectivity compared to the larger contingent of more classic, fast-signaling glutamate and GABA neurons. Dopamine neurons are endowed with a particularly broad and arborized axonal domain containing an extremely large number of release sites. The discovery in the 1980s that most axonal varicosities established by these neurons do not display a classical synaptic structure is a long-standing mystery. We have been recently exploring the topography of dopaminergic release sites using a co-culture system of postnatal mouse dopamine neurons together with striatal medium spiny neurons. We find that contrarily to cortical or striatal neurons, dopamine neurons have a surprising propensity to establish axonal varicosities that mostly ignore target cells. These axonal varicosities contain classical vesicular proteins such as synaptotagmin 1 and VMAT2. Only a very small subset of dopaminergic terminals are in close proximity to postsynaptic domains containing dopamine D2 receptors or postsynaptic organizers such as PSD95 or gephyrin. We find this small subset of “synaptic” release sites can be distinguished from the larger subset of “non-synaptic” terminals by their differential expression of active zone proteins. In stark contrast, the vast majority of release sites established by cortical or striatal neurons establish tight links with postsynaptic domains. Whether synaptic or not in structure, we find that in this co-culture system, most dopaminergic axonal varicosities appear to be functional as revealed by the activity-dependent uptake and release of synaptic vesicle recycling markers. A better understanding of the unique connectivity of neuromodulatory neurons is bound to provide new insights into their normal physiological roles and their impairment and vulnerability is diseases such as Parkinson’s, drug abuse and schizophrenia.
Carmen Canavier
LSU Health Sciences Center New Orleans
What Constrains the Maximum Firing Rate in Midbrain Dopamine Neurons?
The electrical activity of midbrain dopamine (DA) neurons was thought to homogeneously signal reward prediction errors. However, distinctions between DA subpopulations are now becoming increasingly clear. Subpopulations of dopamine neurons are characterized in various ways, including electrophysiology, projection target, anatomical location and gene expression. Lammel et al. 2008 identified two subpopulations based on electrophysiology: conventional, slow firing dopamine neurons in the substantia nigra plus a subset of VTA neurons projecting to the lateral shell of the nucleus accumbens versus atypical, faster firing dopamine neurons in the remainder of the VTA. A key difference was the dynamic range of the subpopulations; the atypical population was able to sustain much higher firing frequencies (~20-25 Hz versus ~10 Hz). Another key difference was the mode of entry into depolarization block that limits the firing frequency. The slow firing population fails abruptly, whereas the fast firing population fails via decrementing action potential height. A single compartment model with a Markov model of the sodium channel that included both fast and slow activation states was able to capture these differences very well. The fast firing population was modeled with the known differences of a slower recovery from inactivation for Kv4.3, a smaller surface area, a weak or absent H current, and a shallower AHP. These differences did not account for the difference in maximal firing rate. However, increasing the tendency to enter the slow inactivated state did account for both the difference in maximal rate and the different mode of entry into depolarization block.
LSU Health Sciences Center New Orleans
What Constrains the Maximum Firing Rate in Midbrain Dopamine Neurons?
The electrical activity of midbrain dopamine (DA) neurons was thought to homogeneously signal reward prediction errors. However, distinctions between DA subpopulations are now becoming increasingly clear. Subpopulations of dopamine neurons are characterized in various ways, including electrophysiology, projection target, anatomical location and gene expression. Lammel et al. 2008 identified two subpopulations based on electrophysiology: conventional, slow firing dopamine neurons in the substantia nigra plus a subset of VTA neurons projecting to the lateral shell of the nucleus accumbens versus atypical, faster firing dopamine neurons in the remainder of the VTA. A key difference was the dynamic range of the subpopulations; the atypical population was able to sustain much higher firing frequencies (~20-25 Hz versus ~10 Hz). Another key difference was the mode of entry into depolarization block that limits the firing frequency. The slow firing population fails abruptly, whereas the fast firing population fails via decrementing action potential height. A single compartment model with a Markov model of the sodium channel that included both fast and slow activation states was able to capture these differences very well. The fast firing population was modeled with the known differences of a slower recovery from inactivation for Kv4.3, a smaller surface area, a weak or absent H current, and a shallower AHP. These differences did not account for the difference in maximal firing rate. However, increasing the tendency to enter the slow inactivated state did account for both the difference in maximal rate and the different mode of entry into depolarization block.
David Sulzer
Columbia University
Dopamine synapses and the regulation of synapses during behavior
The output neurons of the striatum (spiny projection neurons, SPNs) are intrinsically silent and are activated only in response to excitation by inputs from the cortex and thalamus. The SPNs in early life are highly excitable, but as learning during critical periods ends, they become far less excitable. We discuss new results (from Ori Lieberman) that this is due to effects of dopamine and macroautophagy on SPN ion channel expression. When dopamine is released simultaneously with cortical input, it acts at D2 receptors to select specific corticostriatal synapses by decreasing release from the less active corticostriatal synapses, a form of synaptic filtering that may underlie learning as animals interact with their environment (from Nigel Bamford and Minerva Wong). This synaptic selection may occur in a very short defined time frame during operant learning when an animal must coordinate a stimulus with a learned behavior (from Avery McGuirt).
Columbia University
Dopamine synapses and the regulation of synapses during behavior
The output neurons of the striatum (spiny projection neurons, SPNs) are intrinsically silent and are activated only in response to excitation by inputs from the cortex and thalamus. The SPNs in early life are highly excitable, but as learning during critical periods ends, they become far less excitable. We discuss new results (from Ori Lieberman) that this is due to effects of dopamine and macroautophagy on SPN ion channel expression. When dopamine is released simultaneously with cortical input, it acts at D2 receptors to select specific corticostriatal synapses by decreasing release from the less active corticostriatal synapses, a form of synaptic filtering that may underlie learning as animals interact with their environment (from Nigel Bamford and Minerva Wong). This synaptic selection may occur in a very short defined time frame during operant learning when an animal must coordinate a stimulus with a learned behavior (from Avery McGuirt).
Wolfram Schultz
University of Cambridge
Dopamine: from movement via reward to rational choice
Given that the phasic dopamine reward prediction error signal is suitable for updating neuronal choice signals, we investigated its properties using economic formalisms. Before starting, I should mention that reward coding is not the only phasic dopamine change; dopamine neurons shows also separate, slower, lower and heterogeneous changes related to what can be broadly described as behavioral activation. In our experimental economics studies on the dopamine reward signal, we estimated formal utility functions from choice under risk (Von Neumann-Morgenstern utility). These choices were rational in following first, second and third order stochastic dominance (reflecting value, variance risk, skewness risk). Utility was coded in dopamine neurones as utility prediction error (which incorporates risk into subjective value). Consistent with this neuronal signal, the dopamine response followed first- and second-order stochastic dominance. These data unite concepts from animal learning theory and economic decision theory at the level of single reward neurons.
University of Cambridge
Dopamine: from movement via reward to rational choice
Given that the phasic dopamine reward prediction error signal is suitable for updating neuronal choice signals, we investigated its properties using economic formalisms. Before starting, I should mention that reward coding is not the only phasic dopamine change; dopamine neurons shows also separate, slower, lower and heterogeneous changes related to what can be broadly described as behavioral activation. In our experimental economics studies on the dopamine reward signal, we estimated formal utility functions from choice under risk (Von Neumann-Morgenstern utility). These choices were rational in following first, second and third order stochastic dominance (reflecting value, variance risk, skewness risk). Utility was coded in dopamine neurones as utility prediction error (which incorporates risk into subjective value). Consistent with this neuronal signal, the dopamine response followed first- and second-order stochastic dominance. These data unite concepts from animal learning theory and economic decision theory at the level of single reward neurons.
Stephanie Cragg
University of Oxford
Axonal gating of striatal dopamine transmission
Dopamine (DA) in the mammalian striatum plays critical roles in motivation, action and reinforcement learning, and DA dysregulation underlies psychomotor disorders that include Parkinson’s disease and addictions. DA neurons in substantia nigra and ventral tegmental area generate action potentials at a range of instantaneous frequencies, but mechanisms operating on the colossal branched arbours of DA axons will govern whether and how dynamic activity in DA neurons is relayed into DA release. We have identified a diverse range of processes that gate DA output that include axonal inputs and neuromodulators from other networks of striatal neurons (cholinergic interneurons, GABA tone and regulators) and extend to non-neuronal cells (astrocytes), as well as intrinsic regulators of short-term presynaptic plasticity including the DA transporter, and proteins associated with Parkinson’s. I will highlight some of our recent findings in mouse striatum to illustrate how diverse processes operating on DA axons powerfully filter DA output.
University of Oxford
Axonal gating of striatal dopamine transmission
Dopamine (DA) in the mammalian striatum plays critical roles in motivation, action and reinforcement learning, and DA dysregulation underlies psychomotor disorders that include Parkinson’s disease and addictions. DA neurons in substantia nigra and ventral tegmental area generate action potentials at a range of instantaneous frequencies, but mechanisms operating on the colossal branched arbours of DA axons will govern whether and how dynamic activity in DA neurons is relayed into DA release. We have identified a diverse range of processes that gate DA output that include axonal inputs and neuromodulators from other networks of striatal neurons (cholinergic interneurons, GABA tone and regulators) and extend to non-neuronal cells (astrocytes), as well as intrinsic regulators of short-term presynaptic plasticity including the DA transporter, and proteins associated with Parkinson’s. I will highlight some of our recent findings in mouse striatum to illustrate how diverse processes operating on DA axons powerfully filter DA output.
Michael Frank
Brown University
Striatal dopamine computations in learning about agency
The basal ganglia and dopaminergic systems are well studied for their roles in reinforcement learning and reward-based decision making. Much work focuses on "reward prediction error" (RPE) signals conveyed by dopamine and used for learning. Computational considerations suggest that such signals may be enriched beyond the classical global and scalar RPE computation, to support more structured learning in distinct sub-circuits ("vector RPEs"). Such signals allow an agent to assign credit to the level of action selection most likely responsible for the outcomes, and hence to enhance learning depending on the generative task statistics. I will present studies from rodents in tasks involving learning about their agency in controlling reward-predictive sensory events while activity from dopamine terminals across a wide range of dorsal striatum is monitored. Findings indicate that spatiotemporal dynamics of striatal dopamine are influenced by instrumental task demands, and that they are used to enhance credit assignment to differentially reinforce the underlying circuits.
Brown University
Striatal dopamine computations in learning about agency
The basal ganglia and dopaminergic systems are well studied for their roles in reinforcement learning and reward-based decision making. Much work focuses on "reward prediction error" (RPE) signals conveyed by dopamine and used for learning. Computational considerations suggest that such signals may be enriched beyond the classical global and scalar RPE computation, to support more structured learning in distinct sub-circuits ("vector RPEs"). Such signals allow an agent to assign credit to the level of action selection most likely responsible for the outcomes, and hence to enhance learning depending on the generative task statistics. I will present studies from rodents in tasks involving learning about their agency in controlling reward-predictive sensory events while activity from dopamine terminals across a wide range of dorsal striatum is monitored. Findings indicate that spatiotemporal dynamics of striatal dopamine are influenced by instrumental task demands, and that they are used to enhance credit assignment to differentially reinforce the underlying circuits.
Ashok Litwin-Kumar
Columbia University
Heterogeneous dopamine signaling in the Drosophila mushroom body
The Drosophila mushroom body exhibits dopamine (DA) dependent synaptic plasticity that underlies the acquisition and retrieval of associative memories. Classic studies have recorded DA activity in this system and identified signals related to external reinforcement such as reward and punishment. However, recent studies have found that other factors including locomotion, novelty, reward expectation, and internal state also modulate DA neurons. I will discuss recent work analyzing the circuits upstream of DA neurons in adult and larval Drosophila as well as a modeling framework we have developed to account for the heterogeneity of signals observed in recordings.
Columbia University
Heterogeneous dopamine signaling in the Drosophila mushroom body
The Drosophila mushroom body exhibits dopamine (DA) dependent synaptic plasticity that underlies the acquisition and retrieval of associative memories. Classic studies have recorded DA activity in this system and identified signals related to external reinforcement such as reward and punishment. However, recent studies have found that other factors including locomotion, novelty, reward expectation, and internal state also modulate DA neurons. I will discuss recent work analyzing the circuits upstream of DA neurons in adult and larval Drosophila as well as a modeling framework we have developed to account for the heterogeneity of signals observed in recordings.
Josh Berke
UCSF
Dopamine firing versus dopamine release during motivated behavior.
We have reported that the spiking of identified VTA dopamine neurons does not account for dopamine release within the NAc, at least under some behavioral conditions. For example, many labs have observed fast ramps in dopamine concentration as rats approach rewards, but these are not accompanied by increased dopamine cell spiking. This work has received substantial interest but also criticism. I will briefly describe our main findings and some key points of controversy, and then present new results on the mechanisms governing dopamine release during reward anticipation.
UCSF
Dopamine firing versus dopamine release during motivated behavior.
We have reported that the spiking of identified VTA dopamine neurons does not account for dopamine release within the NAc, at least under some behavioral conditions. For example, many labs have observed fast ramps in dopamine concentration as rats approach rewards, but these are not accompanied by increased dopamine cell spiking. This work has received substantial interest but also criticism. I will briefly describe our main findings and some key points of controversy, and then present new results on the mechanisms governing dopamine release during reward anticipation.
Joshua Dudman
Janelia Research Campus, HHMI
Dopamine modulates policy learning during associative conditioning
Animals and artificial agents form a subjective description of their environment to guide the planning and control of actions to achieve their goals. There are two critical components of the subjective representation: the “value function” which captures an agent’s expectation of future rewards given its current state; and the “policy” which determines what action to take given its current state. In value learning, the agent uses prior experience of when goals were achieved to update its (subjective) estimate of the value function. In policy learning, the agent updates its policy to keep those changes that lead to more rapid or reliable attainment of a goal. Both approaches are guaranteed to converge on an optimal solution, but often differ in the trajectory of learning. The activity of midbrain dopamine neurons (mDA) has long been associated with updates predicted by value learning models - reward prediction errors. However, there are many well known discrepancies between the predictions of value learning and observed mDA activity. One particular recent observation is that mDA activity tends to be closely linked to initiation of actions which is not a component of influential value learning models. Here we consider a parsimonious alternative: perhaps mDA activity reflects action because initial learning of associative tasks is better described by a policy learning model. I will describe our recent work evaluating this hypothesis using a new dataset including detailed continuous monitoring of orofacial and whole body behaviors in naive mice during initial learning. We use closed-loop perturbation experiments and a model of policy learning to argue that behavior and mDA activity during initial learning are well explained from a policy learning perspective. I will conclude by discussing some interesting implications of our particular formulation of a policy learning model for thinking about representations that underlie reinforcement learning models more broadly.
Janelia Research Campus, HHMI
Dopamine modulates policy learning during associative conditioning
Animals and artificial agents form a subjective description of their environment to guide the planning and control of actions to achieve their goals. There are two critical components of the subjective representation: the “value function” which captures an agent’s expectation of future rewards given its current state; and the “policy” which determines what action to take given its current state. In value learning, the agent uses prior experience of when goals were achieved to update its (subjective) estimate of the value function. In policy learning, the agent updates its policy to keep those changes that lead to more rapid or reliable attainment of a goal. Both approaches are guaranteed to converge on an optimal solution, but often differ in the trajectory of learning. The activity of midbrain dopamine neurons (mDA) has long been associated with updates predicted by value learning models - reward prediction errors. However, there are many well known discrepancies between the predictions of value learning and observed mDA activity. One particular recent observation is that mDA activity tends to be closely linked to initiation of actions which is not a component of influential value learning models. Here we consider a parsimonious alternative: perhaps mDA activity reflects action because initial learning of associative tasks is better described by a policy learning model. I will describe our recent work evaluating this hypothesis using a new dataset including detailed continuous monitoring of orofacial and whole body behaviors in naive mice during initial learning. We use closed-loop perturbation experiments and a model of policy learning to argue that behavior and mDA activity during initial learning are well explained from a policy learning perspective. I will conclude by discussing some interesting implications of our particular formulation of a policy learning model for thinking about representations that underlie reinforcement learning models more broadly.
Talia Lerner
Northwestern University
Dorsal Striatal Dopamine Circuits for Habit and Compulsion
As animals learn to connect patterns of external reinforcement with their own behavior, they exhibit goal-directed behavior, executing actions that lead to desired outcomes. However, with extended training behavior can become inflexible, with animals continuing to perform actions even if the outcomes are no longer desired. Habits and compulsions are modes of behavioral control that lead to inflexible responding. Habitual control makes animals less able to respond to changes in action-outcome contingency or outcome devaluation; compulsivity makes animals less able to inhibit their actions in the face of emerging negative consequences. Both habits and compulsions are hypothesized to be involved in a range of psychiatric disorders, such as drug addiction and obsessive-compulsive disorder (OCD), but habits and compulsions are distinct behaviors that may rely on different brain circuitries. We developed an experimental paradigm which allows us to track the development of both habits and compulsions in individual animals while recording neural activity. We performed fiber photometry measurements of dopamine axon activity in the dorsal striatum while mice engaged in reinforcement learning on a random interval (RI60) and found that compulsive responding during a probe session was predicted by the dopamine signal in the dorsomedial striatum (DMS) during RI60 performance. By mimicking this DMS dopamine signal from early in training using optogenetics, we accelerated animals' transitions to compulsive responding, irrespective of habit formation. These results indicate that DMS dopamine signaling specifically plays an important role in establishing compulsions, and provides a foundation to test how alterations of this circuitry participate in drug addiction and OCD.
Northwestern University
Dorsal Striatal Dopamine Circuits for Habit and Compulsion
As animals learn to connect patterns of external reinforcement with their own behavior, they exhibit goal-directed behavior, executing actions that lead to desired outcomes. However, with extended training behavior can become inflexible, with animals continuing to perform actions even if the outcomes are no longer desired. Habits and compulsions are modes of behavioral control that lead to inflexible responding. Habitual control makes animals less able to respond to changes in action-outcome contingency or outcome devaluation; compulsivity makes animals less able to inhibit their actions in the face of emerging negative consequences. Both habits and compulsions are hypothesized to be involved in a range of psychiatric disorders, such as drug addiction and obsessive-compulsive disorder (OCD), but habits and compulsions are distinct behaviors that may rely on different brain circuitries. We developed an experimental paradigm which allows us to track the development of both habits and compulsions in individual animals while recording neural activity. We performed fiber photometry measurements of dopamine axon activity in the dorsal striatum while mice engaged in reinforcement learning on a random interval (RI60) and found that compulsive responding during a probe session was predicted by the dopamine signal in the dorsomedial striatum (DMS) during RI60 performance. By mimicking this DMS dopamine signal from early in training using optogenetics, we accelerated animals' transitions to compulsive responding, irrespective of habit formation. These results indicate that DMS dopamine signaling specifically plays an important role in establishing compulsions, and provides a foundation to test how alterations of this circuitry participate in drug addiction and OCD.
Pascal Kaeser
Harvard University
Mechanisms for fast dopamine signaling
Dopamine is a neuromodulator that codes information for a wide range of brain functions and across broad time domains. Remarkably, however, specific knowledge on the release mechanisms of dopamine is sparse, and it is unclear how the dopamine secretory apparatus has evolved to allow for signaling across multiple time scales. We have recently identified molecular components of the secretory machinery in dopamine axons that accounts for rapid and precise dopamine secretion in the vertebrate striatum. This machinery likely enables fast dopamine coding. I will discuss our progress on the identification of fast release mechanisms for dopamine, present recent data on triggering mechanisms for fast dopamine release, and discuss our work on how dopamine axons interact with other striatal elements. My laboratory uses a combination of mouse genetics, super-resolution microscopy, electrophysiology and imaging to assess the architecture and function of striatal dopaminergic microcircuits. Our hope it that dissecting the molecular, cellular and microcircuit architecture of dopamine signaling will allow for a better understanding of how dopamine controls behavior across various time scales and of neurological diseases that are associated with a breakdown of dopamine signaling.
Harvard University
Mechanisms for fast dopamine signaling
Dopamine is a neuromodulator that codes information for a wide range of brain functions and across broad time domains. Remarkably, however, specific knowledge on the release mechanisms of dopamine is sparse, and it is unclear how the dopamine secretory apparatus has evolved to allow for signaling across multiple time scales. We have recently identified molecular components of the secretory machinery in dopamine axons that accounts for rapid and precise dopamine secretion in the vertebrate striatum. This machinery likely enables fast dopamine coding. I will discuss our progress on the identification of fast release mechanisms for dopamine, present recent data on triggering mechanisms for fast dopamine release, and discuss our work on how dopamine axons interact with other striatal elements. My laboratory uses a combination of mouse genetics, super-resolution microscopy, electrophysiology and imaging to assess the architecture and function of striatal dopaminergic microcircuits. Our hope it that dissecting the molecular, cellular and microcircuit architecture of dopamine signaling will allow for a better understanding of how dopamine controls behavior across various time scales and of neurological diseases that are associated with a breakdown of dopamine signaling.
Veronica A Alvarez
Intramural Research Program, National Institutes of Health
Dopamine modulation of striatal microcircuitry and its behavioral implications
Dopamine is involved in a wide-range of critical striatal functions including reinforcement learning, motivation and vigor. Dopamine participates in these functions via its modulatory effects on striatal neurons and their synaptic connections. Thus, understanding the cellular and synaptic mechanisms underlying dopamine actions in the striatum is key to advancing our knowledge of these critical striatal functions. The data presented will dissect the roles of D1 and D2 receptors, which are the two main types of dopamine receptors expressed in the striatum. While the focus of most studies thus far has been on the action of dopamine on glutamatergic inputs to the striatum, we have found that dopamine also potently suppresses local inhibitory synapses among striatal projection neurons (aka, medium spiny neurons). These local inhibitory synapses mediate lateral inhibition among striatal projection neurons. Therefore, suppression of the collateral inhibition by dopamine can promote the activation of D1 receptor expressing striatal projection neurons when it coincides with activation of D1 receptors and/or excitatory inputs. A combination of in vitro and in vivo studies show that dopamine actions on lateral inhibition provide a circuit mechanism that explains the poorly understood synergistic effect of D1-type and D2-type dopamine receptors on behavior. I will also describe other studies that explore the synaptic and circuit mechanism of alcohol and other drugs of abuse. Most behavioral effects of these drugs rely on an increase of dopamine in the striatum. Mice with targeted deletion of the D2 receptor gene on either indirect-pathway (D2 receptor) projection neurons or midbrain dopamine neuron terminals in the striatum show unique alterations in the cellular and behavioral responses to substances of abuse including cocaine and alcohol. As such, these experiments shed further light on the unique functions of dopamine and D2Rs expressed across the basal ganglia circuitry and the consequent behavioral implications of these circuit effects.
Intramural Research Program, National Institutes of Health
Dopamine modulation of striatal microcircuitry and its behavioral implications
Dopamine is involved in a wide-range of critical striatal functions including reinforcement learning, motivation and vigor. Dopamine participates in these functions via its modulatory effects on striatal neurons and their synaptic connections. Thus, understanding the cellular and synaptic mechanisms underlying dopamine actions in the striatum is key to advancing our knowledge of these critical striatal functions. The data presented will dissect the roles of D1 and D2 receptors, which are the two main types of dopamine receptors expressed in the striatum. While the focus of most studies thus far has been on the action of dopamine on glutamatergic inputs to the striatum, we have found that dopamine also potently suppresses local inhibitory synapses among striatal projection neurons (aka, medium spiny neurons). These local inhibitory synapses mediate lateral inhibition among striatal projection neurons. Therefore, suppression of the collateral inhibition by dopamine can promote the activation of D1 receptor expressing striatal projection neurons when it coincides with activation of D1 receptors and/or excitatory inputs. A combination of in vitro and in vivo studies show that dopamine actions on lateral inhibition provide a circuit mechanism that explains the poorly understood synergistic effect of D1-type and D2-type dopamine receptors on behavior. I will also describe other studies that explore the synaptic and circuit mechanism of alcohol and other drugs of abuse. Most behavioral effects of these drugs rely on an increase of dopamine in the striatum. Mice with targeted deletion of the D2 receptor gene on either indirect-pathway (D2 receptor) projection neurons or midbrain dopamine neuron terminals in the striatum show unique alterations in the cellular and behavioral responses to substances of abuse including cocaine and alcohol. As such, these experiments shed further light on the unique functions of dopamine and D2Rs expressed across the basal ganglia circuitry and the consequent behavioral implications of these circuit effects.
Vanessa Ruta
Rockefeller University
Dissecting the diverse roles of dopaminergic modulation in Drosophila
Dopamine plays a central role in motivation and reinforcement learning, allowing animals to take advantage of their current circumstances to optimize both present and future behavior. Yet reconciling the diverse roles of dopamine has remained a challenge, in part due to the difficulty of understanding how a single neuromodulator can convey distinct signals to its cellular targets in different behavioral contexts. We have been using simple architecture of the Drosophila mushroom body to gain insight into the dual role of dopaminergic signaling at the molecular, synaptic, and circuit level. Dopaminergic neurons richly innervate the mushroom body where they have been canonically studied for their role in instructing olfactory learning. However, recording from mushroom body while a fly actively navigates in a virtual olfactory environment has revealed how rewards and goal-directed actions are encoded through comparable patterns of dopaminergic neuron activity and dopamine release. Together, our observations reveal how the same dopaminergic pathways can coordinately contribute to multiple forms of behavioral modulation over different timescales—conveying motivational signals to rapidly shape ongoing behavior as well as instructive signals to modify future behavior through learning.
Rockefeller University
Dissecting the diverse roles of dopaminergic modulation in Drosophila
Dopamine plays a central role in motivation and reinforcement learning, allowing animals to take advantage of their current circumstances to optimize both present and future behavior. Yet reconciling the diverse roles of dopamine has remained a challenge, in part due to the difficulty of understanding how a single neuromodulator can convey distinct signals to its cellular targets in different behavioral contexts. We have been using simple architecture of the Drosophila mushroom body to gain insight into the dual role of dopaminergic signaling at the molecular, synaptic, and circuit level. Dopaminergic neurons richly innervate the mushroom body where they have been canonically studied for their role in instructing olfactory learning. However, recording from mushroom body while a fly actively navigates in a virtual olfactory environment has revealed how rewards and goal-directed actions are encoded through comparable patterns of dopaminergic neuron activity and dopamine release. Together, our observations reveal how the same dopaminergic pathways can coordinately contribute to multiple forms of behavioral modulation over different timescales—conveying motivational signals to rapidly shape ongoing behavior as well as instructive signals to modify future behavior through learning.
Yael Niv
Princeton University
Model-based predictions for dopamine
Phasic dopamine responses are thought to encode a prediction-error signal consistent with model-free reinforcement learning theories. However, a number of recent findings highlight the influence of model-based computations on dopamine responses, and suggest that dopamine prediction errors reflect more dimensions of an expected outcome than scalar reward value. In this talk I will focus on these challenges to the scalar prediction-error theory of dopamine, and to the strict dichotomy between model-based and model-free learning, suggesting that these may better be viewed as a set of intertwined computations rather than two alternative systems. Alas, phasic dopamine signals, until recently a beacon of computationally-interpretable brain activity, may not be as simple as we once hoped they were.
Princeton University
Model-based predictions for dopamine
Phasic dopamine responses are thought to encode a prediction-error signal consistent with model-free reinforcement learning theories. However, a number of recent findings highlight the influence of model-based computations on dopamine responses, and suggest that dopamine prediction errors reflect more dimensions of an expected outcome than scalar reward value. In this talk I will focus on these challenges to the scalar prediction-error theory of dopamine, and to the strict dichotomy between model-based and model-free learning, suggesting that these may better be viewed as a set of intertwined computations rather than two alternative systems. Alas, phasic dopamine signals, until recently a beacon of computationally-interpretable brain activity, may not be as simple as we once hoped they were.
Naoshige Uchida
Harvard University
Dissociating reward prediction error and value in dopamine signals
Previous studies have revealed an exceptional correspondence between the activity of midbrain dopamine neurons and a ‘teaching signal’ in reinforcement learning algorithms. In particular, the reward prediction error (RPE) used in the temporal difference (TD) learning algorithm captures aspects of phasic dopamine responses. However, this idea has been challenged by recent observations that dopamine signals ramp up gradually over the timescale of seconds as animals approach a reward location. It has been argued that these slow fluctuations of dopamine are inconsistent with the RPE model, and instead represent the state value, which gradually increases toward a reward location. Whether these slowly fluctuating dopamine signals represent value or RPE, and under what conditions a dopamine ramp occurs, remain elusive. As originally formulated, the TD RPE approximates the derivative of the value function. Based on this core property, we developed a set of experimental paradigms that dissociate RPE from value. We employed visual virtual reality in mice to manipulate the location of the animal and the speed of scene movement independent of the animal’s locomotion. We found that the manipulation of scene movement – teleport and speed manipulations – caused dopamine responses in the ventral striatum that were consistent with TD RPEs but inconsistent with state values. Furthermore, we found that a more abstract, non-navigational stimulus that indicates temporal proximity to reward is sufficient to cause a dopamine ramp. These results indicate that the RPE account of dopamine responses can be extended to slowly fluctuating dopamine signals in addition to phasic dopamine responses, and support the previously untested central tenet of TD RPEs that dopamine neurons signal RPEs through a derivative-like computation over value on a moment-by-moment basis.
Harvard University
Dissociating reward prediction error and value in dopamine signals
Previous studies have revealed an exceptional correspondence between the activity of midbrain dopamine neurons and a ‘teaching signal’ in reinforcement learning algorithms. In particular, the reward prediction error (RPE) used in the temporal difference (TD) learning algorithm captures aspects of phasic dopamine responses. However, this idea has been challenged by recent observations that dopamine signals ramp up gradually over the timescale of seconds as animals approach a reward location. It has been argued that these slow fluctuations of dopamine are inconsistent with the RPE model, and instead represent the state value, which gradually increases toward a reward location. Whether these slowly fluctuating dopamine signals represent value or RPE, and under what conditions a dopamine ramp occurs, remain elusive. As originally formulated, the TD RPE approximates the derivative of the value function. Based on this core property, we developed a set of experimental paradigms that dissociate RPE from value. We employed visual virtual reality in mice to manipulate the location of the animal and the speed of scene movement independent of the animal’s locomotion. We found that the manipulation of scene movement – teleport and speed manipulations – caused dopamine responses in the ventral striatum that were consistent with TD RPEs but inconsistent with state values. Furthermore, we found that a more abstract, non-navigational stimulus that indicates temporal proximity to reward is sufficient to cause a dopamine ramp. These results indicate that the RPE account of dopamine responses can be extended to slowly fluctuating dopamine signals in addition to phasic dopamine responses, and support the previously untested central tenet of TD RPEs that dopamine neurons signal RPEs through a derivative-like computation over value on a moment-by-moment basis.
Ilana Witten
Princeton University
Inferring spikes from calcium signals in dopamine neurons
Recent work using calcium indicators has revealed unexpected functional heterogeneity in the dopamine system. We use a new spike inference algorithm to examine the correspondence between calcium signals and spiking in dopamine neurons, to help interpret this recent data.
Princeton University
Inferring spikes from calcium signals in dopamine neurons
Recent work using calcium indicators has revealed unexpected functional heterogeneity in the dopamine system. We use a new spike inference algorithm to examine the correspondence between calcium signals and spiking in dopamine neurons, to help interpret this recent data.
Nathaniel Daw
Princeton University
Population codes and prediction errors
Standard theoretical accounts envision that the midbrain dopamine system broadcasts a uniform prediction error signal through diffuse ascending connections to forebrain. However, I review mounting evidence that now clearly demonstrates that the dopamine response is instead heterogeneous from target area to target area and even from neuron to neuron. I present a new model that suggests that this heterogeneity is inherited from, and ultimately may help to shed light on, the population codes for ongoing task events in afferent cortical and striatal areas. This may provide an unexpected window on a core problem for decision and learning models of all sorts, known as the problem of state: how the brain represents the relevant (and ignores the irrelevant) features of a task at each step.
Princeton University
Population codes and prediction errors
Standard theoretical accounts envision that the midbrain dopamine system broadcasts a uniform prediction error signal through diffuse ascending connections to forebrain. However, I review mounting evidence that now clearly demonstrates that the dopamine response is instead heterogeneous from target area to target area and even from neuron to neuron. I present a new model that suggests that this heterogeneity is inherited from, and ultimately may help to shed light on, the population codes for ongoing task events in afferent cortical and striatal areas. This may provide an unexpected window on a core problem for decision and learning models of all sorts, known as the problem of state: how the brain represents the relevant (and ignores the irrelevant) features of a task at each step.
Alexandra Nelson
UCSF
Striatal Mechanisms of Levodopa-Induced Dyskinesia
In Parkinson’s Disease, motor symptoms are often treated with the dopamine precursor, levodopa. Levodopa is highly effective in most cases, but over time, treatment is complicated by the development of abnormal involuntary movements, or levodopa-induced dyskinesia (LID). There are few effective medications to manage LID, and it has been very difficult to disentangle the therapeutic and dyskinetic effects of levodopa, as both depend on dopamine signaling. To examine whether these two behavioral effects of levodopa might be mediated by distinct neural ensembles, we have used an activity-dependent mouse line to label neurons brain-wide that are active during LID. We have identified a subset of striatal direct pathway neurons that achieve exceptionally high firing rates during LID, and whose activation is sufficient, in the absence of levodopa, to evoke dyskinesia. To determine how these LID-associated neurons differ from other neighboring direct pathway neurons, we have used Cre-dependent rabies tracing and ex vivo electrophysiology to investigate the distribution and function of their excitatory inputs, as well as their intrinsic properties. At the anatomical level, we find that LID-associated striatal neurons receive a similar distribution of excitatory inputs from the cortex and thalamus as their direct pathway neighbors. However, at the functional level, they receive stronger excitatory input from both primary motor cortex and thalamus. In addition, their intrinsic excitability, especially spike generation, is more strongly regulated by dopamine than neighboring direct pathway neurons. These findings identify a distinct subpopulation of striatal direct pathway neurons that are causally involved in LID, but further provide a cellular and synaptic mechanism for their distinct properties during dyskinesia in vivo. We hope these findings will promote the development of more targeted therapies for the motor symptoms of Parkinson’s Disease.
UCSF
Striatal Mechanisms of Levodopa-Induced Dyskinesia
In Parkinson’s Disease, motor symptoms are often treated with the dopamine precursor, levodopa. Levodopa is highly effective in most cases, but over time, treatment is complicated by the development of abnormal involuntary movements, or levodopa-induced dyskinesia (LID). There are few effective medications to manage LID, and it has been very difficult to disentangle the therapeutic and dyskinetic effects of levodopa, as both depend on dopamine signaling. To examine whether these two behavioral effects of levodopa might be mediated by distinct neural ensembles, we have used an activity-dependent mouse line to label neurons brain-wide that are active during LID. We have identified a subset of striatal direct pathway neurons that achieve exceptionally high firing rates during LID, and whose activation is sufficient, in the absence of levodopa, to evoke dyskinesia. To determine how these LID-associated neurons differ from other neighboring direct pathway neurons, we have used Cre-dependent rabies tracing and ex vivo electrophysiology to investigate the distribution and function of their excitatory inputs, as well as their intrinsic properties. At the anatomical level, we find that LID-associated striatal neurons receive a similar distribution of excitatory inputs from the cortex and thalamus as their direct pathway neighbors. However, at the functional level, they receive stronger excitatory input from both primary motor cortex and thalamus. In addition, their intrinsic excitability, especially spike generation, is more strongly regulated by dopamine than neighboring direct pathway neurons. These findings identify a distinct subpopulation of striatal direct pathway neurons that are causally involved in LID, but further provide a cellular and synaptic mechanism for their distinct properties during dyskinesia in vivo. We hope these findings will promote the development of more targeted therapies for the motor symptoms of Parkinson’s Disease.
Zayd Khaliq
NINDS, NIH
GABAA receptor mediated control of spiking and transmitter release from dopaminergic neuron axons
Axons of midbrain dopaminergic neurons innervate the striatum where they contribute to movement and reinforcement learning. In this talk, I will discuss our work in which we use whole-cell and perforated-patch recordings to test for GABA-A receptors on the main dopaminergic neuron axons and branching processes within striatum. We found that application of GABA decreased the amplitude of axonal spikes, limited propagation and reduced striatal dopamine release. I will discuss the mechanism of this GABA mediated inhibition. I will also discuss effects of the broad-spectrum modulator, diazepam, which enhanced GABA-A currents on dopaminergic neuron axons but also likely works indirectly through elements within the local striatal circuit. We believe that these findings provide insight into the actions of benzodiazepines within the striatum.
NINDS, NIH
GABAA receptor mediated control of spiking and transmitter release from dopaminergic neuron axons
Axons of midbrain dopaminergic neurons innervate the striatum where they contribute to movement and reinforcement learning. In this talk, I will discuss our work in which we use whole-cell and perforated-patch recordings to test for GABA-A receptors on the main dopaminergic neuron axons and branching processes within striatum. We found that application of GABA decreased the amplitude of axonal spikes, limited propagation and reduced striatal dopamine release. I will discuss the mechanism of this GABA mediated inhibition. I will also discuss effects of the broad-spectrum modulator, diazepam, which enhanced GABA-A currents on dopaminergic neuron axons but also likely works indirectly through elements within the local striatal circuit. We believe that these findings provide insight into the actions of benzodiazepines within the striatum.
Margaret Rice
NYU Grossman School of Medicine
Somatodendritic Dopamine Release: New Insights into Long-Standing Questions
Somatodendritic dopamine (DA) release is a pivotal component of midbrain DA neuron physiology, and contributes importantly to DA transmission. Somatodendritic DA release, first reported in the 1970’s, was shown in those and subsequent studies to be exocytotic and calcium dependent. Among other functions, locally released DA from midbrain DA neurons activates D2 autoreceptors that inhibit neuronal firing, thereby influencing axonal DA release in target regions. Despite decades of investigation, fundamental questions about the release process remain unresolved. Among these is the source of autoregulatory DA, with evidence for dendro-dendritic synapses, as well as diffusion-based volume transmission reported in the literature. Also, the calcium-dependence of somatodendritic vs. axon terminal DA release differs: somatodendritic release persists at calcium concentrations that are too low to support axonal release. We have addressed these questions using whole-cell recording to monitor D2 inhibitory currents (D2ICs) in substantia nigra pars compacta (SNc) DA neurons in ex vivo midbrain slices as an index of locally evoked somatodendritic DA release (a technique introduced by Beckstead, Williams, and colleagues). Our studies show that a SNc DA neuron is autoregulated primarily by its own DA release, rather than via synaptic DA release or volume transmission from neighboring DA neurons. We extended this result by using intracellular antibodies and transgenic mice to identify specific synaptotagmin isoforms that contribute to the enhanced calcium sensitivity of somatodendritic DA release. Our finding that D2 autoreceptors on a given SNc DA neuron are activated primarily by DA released from that same cell provides new insight into the autoregulatory role of somatodendritic DA release. In addition, this result enables the use single-cell application of antibodies and other agents to DA neurons to address long-standing questions about the mechanism of somatodendritic release.
NYU Grossman School of Medicine
Somatodendritic Dopamine Release: New Insights into Long-Standing Questions
Somatodendritic dopamine (DA) release is a pivotal component of midbrain DA neuron physiology, and contributes importantly to DA transmission. Somatodendritic DA release, first reported in the 1970’s, was shown in those and subsequent studies to be exocytotic and calcium dependent. Among other functions, locally released DA from midbrain DA neurons activates D2 autoreceptors that inhibit neuronal firing, thereby influencing axonal DA release in target regions. Despite decades of investigation, fundamental questions about the release process remain unresolved. Among these is the source of autoregulatory DA, with evidence for dendro-dendritic synapses, as well as diffusion-based volume transmission reported in the literature. Also, the calcium-dependence of somatodendritic vs. axon terminal DA release differs: somatodendritic release persists at calcium concentrations that are too low to support axonal release. We have addressed these questions using whole-cell recording to monitor D2 inhibitory currents (D2ICs) in substantia nigra pars compacta (SNc) DA neurons in ex vivo midbrain slices as an index of locally evoked somatodendritic DA release (a technique introduced by Beckstead, Williams, and colleagues). Our studies show that a SNc DA neuron is autoregulated primarily by its own DA release, rather than via synaptic DA release or volume transmission from neighboring DA neurons. We extended this result by using intracellular antibodies and transgenic mice to identify specific synaptotagmin isoforms that contribute to the enhanced calcium sensitivity of somatodendritic DA release. Our finding that D2 autoreceptors on a given SNc DA neuron are activated primarily by DA released from that same cell provides new insight into the autoregulatory role of somatodendritic DA release. In addition, this result enables the use single-cell application of antibodies and other agents to DA neurons to address long-standing questions about the mechanism of somatodendritic release.
Jesse Goldberg
Cornell University
Male songbirds turn off their self evaluation system when they perform for females
Attending to mistakes while practicing alone provides opportunities for learning, but self-evaluation during audience-directed performance could distract from ongoing execution. It remains unknown how animals switch between practice and performance modes, and how evaluation systems process errors across distinct performance contexts. We recorded from striatal-projecting dopamine (DA) neurons as male songbirds transitioned from singing alone to singing female-directed courtship song. In the presence of the female, singing-related performance error signals were reduced or gated off and DA neurons were instead phasically activated by female vocalizations. DA neurons can thus dynamically change their tuning with changes in social context.
Cornell University
Male songbirds turn off their self evaluation system when they perform for females
Attending to mistakes while practicing alone provides opportunities for learning, but self-evaluation during audience-directed performance could distract from ongoing execution. It remains unknown how animals switch between practice and performance modes, and how evaluation systems process errors across distinct performance contexts. We recorded from striatal-projecting dopamine (DA) neurons as male songbirds transitioned from singing alone to singing female-directed courtship song. In the presence of the female, singing-related performance error signals were reduced or gated off and DA neurons were instead phasically activated by female vocalizations. DA neurons can thus dynamically change their tuning with changes in social context.
Jennifer Whistler
UC Davis
Post-endocytic sorting of D2 dopamine receptors underlies drug induced changes in synaptic and behavioral plasticity
Low D2 dopamine receptor availability is a key hallmark of substance use disorder for all drugs of abuse. Low D2 receptor both increases the risk of drug seeking, and occurs as a consequence of drug use. Selective loss of D2 receptors alters the balance of excitatory and inhibitory G protein coupled receptor (GPCR) dopamine signaling, as the D1-like dopamine receptors are Gs-coupled whereas the D2-like receptors are Gi-coupled GPCRs. Loss of balance in GPCR-mediated dopamine signaling, in turn, is likely to underlie many of the alterations in synaptic and behavioral plasticity that occur with prolonged drug use. However, the molecular mechanisms underlying drug-induced loss of D2 receptors remain understudied. We have found that while both the D1 the D2 dopamine receptors are endocytosed after activation by dopamine, the D1 receptors are rapidly recycled whereas the D2 receptors are targeted for degradation in the lysosome. We have identified a sorting protein, GPCR-associated sorting protein 1 (GASP1) that binds to D2 and D3 dopamine receptors and is necessary for their dopamine-induced degradation. Mice with a disruption in GASP1 (GASP1-KO) do not downregulate D2 receptors in response to drug of abuse, unlike wild type mice that show profound loss of D2 receptors. In addition, GASP1-KO mice do not show sensitization to the locomotor activating effects of drug nor do they show changes in behavioral flexibility in response to drug. Furthermore, drug-induced alterations in glutamatergic plasticity in dopamine neurons in the ventral tegmental area (VTA) that occur in wild type mice are absent in GASP1-KO mice. These data suggest that post-endocytic targeting of D2-like receptors to the lysosome underlies changes in both behavioral and synaptic plasticity that occur as a consequence of drug use. They also suggest that drugs that can activate the D2 receptor without promoting its endocytosis and degradation could be used to prevent or restore these drug-induced changes in plasticity. In support of this hypothesis, we have found that administration of aripiprazole, a high affinity G protein biased agonist at the D2 receptor that does not promote receptor endocytosis prevents drug-induced changes in synaptic plasticity in the VTA as well as the drug-induced changes in behavioral flexibility. Hence, we have identified a molecular mechanism that underlies the drug-induced loss of D2 receptor, and we have leveraged this finding to identify potential therapeutic interventions that can restore the balance in dopamine signaling.
UC Davis
Post-endocytic sorting of D2 dopamine receptors underlies drug induced changes in synaptic and behavioral plasticity
Low D2 dopamine receptor availability is a key hallmark of substance use disorder for all drugs of abuse. Low D2 receptor both increases the risk of drug seeking, and occurs as a consequence of drug use. Selective loss of D2 receptors alters the balance of excitatory and inhibitory G protein coupled receptor (GPCR) dopamine signaling, as the D1-like dopamine receptors are Gs-coupled whereas the D2-like receptors are Gi-coupled GPCRs. Loss of balance in GPCR-mediated dopamine signaling, in turn, is likely to underlie many of the alterations in synaptic and behavioral plasticity that occur with prolonged drug use. However, the molecular mechanisms underlying drug-induced loss of D2 receptors remain understudied. We have found that while both the D1 the D2 dopamine receptors are endocytosed after activation by dopamine, the D1 receptors are rapidly recycled whereas the D2 receptors are targeted for degradation in the lysosome. We have identified a sorting protein, GPCR-associated sorting protein 1 (GASP1) that binds to D2 and D3 dopamine receptors and is necessary for their dopamine-induced degradation. Mice with a disruption in GASP1 (GASP1-KO) do not downregulate D2 receptors in response to drug of abuse, unlike wild type mice that show profound loss of D2 receptors. In addition, GASP1-KO mice do not show sensitization to the locomotor activating effects of drug nor do they show changes in behavioral flexibility in response to drug. Furthermore, drug-induced alterations in glutamatergic plasticity in dopamine neurons in the ventral tegmental area (VTA) that occur in wild type mice are absent in GASP1-KO mice. These data suggest that post-endocytic targeting of D2-like receptors to the lysosome underlies changes in both behavioral and synaptic plasticity that occur as a consequence of drug use. They also suggest that drugs that can activate the D2 receptor without promoting its endocytosis and degradation could be used to prevent or restore these drug-induced changes in plasticity. In support of this hypothesis, we have found that administration of aripiprazole, a high affinity G protein biased agonist at the D2 receptor that does not promote receptor endocytosis prevents drug-induced changes in synaptic plasticity in the VTA as well as the drug-induced changes in behavioral flexibility. Hence, we have identified a molecular mechanism that underlies the drug-induced loss of D2 receptor, and we have leveraged this finding to identify potential therapeutic interventions that can restore the balance in dopamine signaling.
David Kleinfeld
UCSD
Reinforcement learning will link spontaneous dopamine transients to a reward
How does neuromodulation form a link backward in time, the "distal reward problem", from a neutral clue or conditioning stimulus (CS) to a reward or unconditioned stimulus (US)? A classic proposal is the formation of a memory of the CS, i.e., an eligibility trace. In contrast, we conjecture that spontaneous spikes in neuromodulator concentration can initially bridge the temporal interval from the CS to US. By imaging cell-based molecular indicators implanted in frontal cortex of mice, we observe such spontaneous events for dopamine. Using real-time feedback and reinforcement learning, we show that mice can volitionally initiate dopamine spikes and establish a neuromodulatory link to a reward without need for a CS. Neuromodulatory kinetics track an increasing threshold for reward by an increase in basal concentration. A link to a reward could also be established for norepinephrine but not for acetylcholine, consistent with acetylcholine's role in attention but not learning per se. The findings for dopamine provide a mechanism to transform a Reward signal to an Expectation signal.
UCSD
Reinforcement learning will link spontaneous dopamine transients to a reward
How does neuromodulation form a link backward in time, the "distal reward problem", from a neutral clue or conditioning stimulus (CS) to a reward or unconditioned stimulus (US)? A classic proposal is the formation of a memory of the CS, i.e., an eligibility trace. In contrast, we conjecture that spontaneous spikes in neuromodulator concentration can initially bridge the temporal interval from the CS to US. By imaging cell-based molecular indicators implanted in frontal cortex of mice, we observe such spontaneous events for dopamine. Using real-time feedback and reinforcement learning, we show that mice can volitionally initiate dopamine spikes and establish a neuromodulatory link to a reward without need for a CS. Neuromodulatory kinetics track an increasing threshold for reward by an increase in basal concentration. A link to a reward could also be established for norepinephrine but not for acetylcholine, consistent with acetylcholine's role in attention but not learning per se. The findings for dopamine provide a mechanism to transform a Reward signal to an Expectation signal.
