-
Translational Psychiatry May 2022Avoidance and heightened responses to perceived threats are key features of anxiety disorders. These disorders are characterised by inflexibility in dynamically updating...
Avoidance and heightened responses to perceived threats are key features of anxiety disorders. These disorders are characterised by inflexibility in dynamically updating behavioural and physiological responses to aversively conditioned cues or environmental contexts which are no longer objectively threatening, often manifesting in perseverative avoidance. However, less is known about how anxiety disorders might differ in adjusting to threat and safety shifts in the environment or how idiosyncratic avoidance responses are learned and persist. Twenty-eight patients with generalised anxiety disorder (GAD), without DSM co-morbidities, and 27 matched healthy controls were administered two previously established paradigms: Pavlovian threat reversal and shock avoidance habits through overtraining (assessed following devaluation with measures of perseverative responding). For both tasks we used subjective report scales and skin conductance responses (SCR). In the Pavlovian threat reversal task, patients with GAD showed a significantly overall higher SCR as well as a reduced differential SCR response compared to controls in the early but not late reversal phase. During the test of habitual avoidance responding, GAD patients did not differ from controls in task performance, habitual active avoidance responses during devaluation, or corresponding SCR during trials, but showed a trend toward more abstract confirmatory subjective justifications for continued avoidance following the task. GAD patients exhibited significantly greater skin conductance responses to signals of threat than controls, but did not exhibit the major deficits in reversal and safety signal learning shown previously by patients with OCD. Moreover, this patient group, again unlike OCD patients, did not show evidence of altered active avoidance learning or enhanced instrumental avoidance habits. Overall, these findings indicate no deficits in instrumental active avoidance or persistent avoidance habits, despite enhanced responses to Pavlovian threat cues in GAD. They suggest that GAD is characterised by passive, and not excessively rigid, avoidance styles.
Topics: Anxiety Disorders; Avoidance Learning; Cues; Habits; Humans; Reversal Learning
PubMed: 35641488
DOI: 10.1038/s41398-022-01981-3 -
Learning & Memory (Cold Spring Harbor,... Nov 2019Adjusting behavior to changed environmental contingencies is critical for survival, and reversal learning provides an experimental handle on such cognitive flexibility....
Adjusting behavior to changed environmental contingencies is critical for survival, and reversal learning provides an experimental handle on such cognitive flexibility. Here, we investigate reversal learning in larval Using odor-taste associations, we establish olfactory reversal learning in the appetitive and the aversive domain, using either fructose as a reward or high-concentration sodium chloride as a punishment, respectively. Reversal learning is demonstrated both in differential and in absolute conditioning, in either valence domain. In differential conditioning, the animals are first trained such that an odor A is paired, for example, with the reward whereas odor B is not (A+/B); this is followed by a second training phase with reversed contingencies (A/B+). In absolute conditioning, odor B is omitted, such that the animals are first trained with paired presentations of A and reward, followed by unpaired training in the second training phase. Our results reveal "true" reversal learning in that the opposite associative effects of both the first and the second training phase are detectable after reversed-contingency training. In what is a surprisingly quick, one-trial contingency adjustment in the larva, the present study establishes a simple and genetically easy accessible study case of cognitive flexibility.
Topics: Animals; Appetitive Behavior; Association Learning; Avoidance Learning; Behavior, Animal; Conditioning, Psychological; Drosophila; Larva; Olfactory Perception; Reversal Learning; Reward; Taste Perception
PubMed: 31615854
DOI: 10.1101/lm.049510.119 -
Psychopharmacology Jan 2019This review is concerned with methods for assessing the processing of unrewarded responses in experimental animals and the mechanisms underlying performance of these... (Review)
Review
This review is concerned with methods for assessing the processing of unrewarded responses in experimental animals and the mechanisms underlying performance of these tasks. A number of clinical populations, including Parkinson's disease, depression, compulsive disorders, and schizophrenia demonstrate either abnormal processing or learning from non-rewarded responses in laboratory-based reinforcement learning tasks. These effects are hypothesized to result from disturbances in modulatory neurotransmitter systems, including dopamine and serotonin. Parallel work in experimental animals has revealed consistent behavioral patterns associated with non-reward and, consistent with the human literature, modulatory roles for specific neurotransmitters. Classical tests involving an important reward omission component include appetitive extinction, ratio schedules of responding, reversal learning, and delay and probability discounting procedures. In addition, innovative behavioral tests have recently been developed leverage probabilistic feedback to specifically assay accommodation of, and learning from, non-rewarded responses. These procedures will be described and reviewed with discussion of the behavioral and neural determinants of performance. A final section focusses specifically on the benefits of trial-by-trial analysis of responding during such tasks, and the implications of such analyses for the translation of findings to clinical studies.
Topics: Animals; Association Learning; Brain; Conditioning, Classical; Conditioning, Operant; Delay Discounting; Disease Models, Animal; Dopamine; Humans; Male; Motivation; Neurotransmitter Agents; Reinforcement, Psychology; Reversal Learning; Reward; Schizophrenia; Schizophrenic Psychology; Serotonin; Translational Research, Biomedical
PubMed: 30306228
DOI: 10.1007/s00213-018-5062-x -
Psychopharmacology Jan 2012Our ability to measure the cognitive components of complex decision-making across species has greatly facilitated our understanding of its neurobiological mechanisms.... (Review)
Review
BACKGROUND
Our ability to measure the cognitive components of complex decision-making across species has greatly facilitated our understanding of its neurobiological mechanisms. One task in particular, reversal learning, has proven valuable in assessing the inhibitory processes that are central to executive control. Reversal learning measures the ability to actively suppress reward-related responding and to disengage from ongoing behavior, phenomena that are biologically and descriptively related to impulsivity and compulsivity. Consequently, reversal learning could index vulnerability for disorders characterized by impulsivity such as proclivity for initial substance abuse as well as the compulsive aspects of dependence.
OBJECTIVE
Though we describe common variants and similar tasks, we pay particular attention to discrimination reversal learning, its supporting neural circuitry, neuropharmacology and genetic determinants. We also review the utility of this task in measuring impulsivity and compulsivity in addictions.
METHODS
We restrict our review to instrumental, reward-related reversal learning studies as they are most germane to addiction.
CONCLUSION
The research reviewed here suggests that discrimination reversal learning may be used as a diagnostic tool for investigating the neural mechanisms that mediate impulsive and compulsive aspects of pathological reward-seeking and -taking behaviors. Two interrelated mechanisms are posited for the neuroadaptations in addiction that often translate to poor reversal learning: frontocorticostriatal circuitry dysregulation and poor dopamine (D2 receptor) modulation of this circuitry. These data suggest new approaches to targeting inhibitory control mechanisms in addictions.
Topics: Behavior, Addictive; Biogenic Monoamines; Brain; Compulsive Behavior; Decision Making; Humans; Impulsive Behavior; Inhibition, Psychological; Models, Neurological; Neural Pathways; Reversal Learning
PubMed: 22134477
DOI: 10.1007/s00213-011-2579-7 -
PloS One 2021Although several studies showed adverse neurotoxic effects of melamine on hippocampus (HPC)-dependent learning and reversal learning, the evidence for this mechanism is...
Although several studies showed adverse neurotoxic effects of melamine on hippocampus (HPC)-dependent learning and reversal learning, the evidence for this mechanism is still unknown. We recently demonstrated that intra-hippocampal melamine injection affected the induction of long-term depression, which is associated with novelty acquisition and memory consolidation. Here, we infused melamine into the HPC of rats, and employed behavioral tests, immunoblotting, immunocytochemistry and electrophysiological methods to sought evidence for its effects on cognitive flexibility. Rats with intra-hippocampal infusion of melamine displayed dose-dependent increase in trials to the criterion in reversal learning, with no locomotion or motivation defect. Compared with controls, melamine-treated rats avoided HPC-dependent place strategy. Meanwhile, the learning-induced BDNF level in the HPC neurons was significantly reduced. Importantly, bilateral intra-hippocampal BDNF infusion could effectively mitigate the suppressive effects of melamine on neural correlate with reversal performance, and rescue the strategy bias and reversal learning deficits. Our findings provide first evidence for the effect of melamine on cognitive flexibility and suggest that the reversal learning deficit is due to the inability to use place strategy. Furthermore, the suppressive effects of melamine on BDNF-mediated neural activity could be the mechanism, thus advancing the understanding of compulsive behavior in melamine-induced and other neuropsychiatric disorders.
Topics: Animals; Brain-Derived Neurotrophic Factor; Hippocampus; Male; Maze Learning; Neurons; Rats, Sprague-Dawley; Reversal Learning; Spatial Learning; Triazines; Rats
PubMed: 33428671
DOI: 10.1371/journal.pone.0245326 -
Current Biology : CB Aug 2022Being able to let go of behaviors that are no longer valuable and adopt actions that achieve the same outcome is fundamental for animal survival. A new study offers...
Being able to let go of behaviors that are no longer valuable and adopt actions that achieve the same outcome is fundamental for animal survival. A new study offers clues on the neural mechanisms that allow animals to reverse their behavior as needed.
Topics: Reversal Learning
PubMed: 35944488
DOI: 10.1016/j.cub.2022.06.045 -
NeuroImage. Clinical 2021Reversal learning reflects an individual's capacity to adapt to a dynamic environment with changing stimulus-reward contingencies. This study focuses on the potential...
BACKGROUND
Reversal learning reflects an individual's capacity to adapt to a dynamic environment with changing stimulus-reward contingencies. This study focuses on the potential influence of anxiety on reversal learning skills.
METHODS
We asked 40 participants with a high level of trait anxiety (HTA) and 40 counterparts with a low anxiety level (LTA) to finish a probabilistic reversal learning task with event-related potential (ERP) recording, during which stimulus-reward contingencies are reversed after players have learned the optimal choice.
RESULTS
We found that compared to their LTA counterparts, the HTA participants showed worse learning performance and were less likely to make lose-shift choices. The FRN amplitude might help interpret these behavioral results, which is suggested to be associated with punishment sensitivity and was positively correlated with the number of lose-shift in this study. Seeing that anxiety level predicted the FRN amplitude for lose-shift, we explain that anxious individuals' inflexible behavioral responses to losses are due to their impaired sensitivity to negative feedback.
CONCLUSIONS
A higher level of anxiety is associated with weaker reversal learning performance, possibly because of abnormal sensitivity to negative outcomes. These findings have implications for the understanding of behavioral symptoms in anxiety.
Topics: Anxiety; Anxiety Disorders; Evoked Potentials; Humans; Reversal Learning; Reward
PubMed: 34242887
DOI: 10.1016/j.nicl.2021.102751 -
Behavioral Neuroscience Feb 2022Reversal learning depends on cognitive flexibility. Many reversal learning studies assess cognitive flexibility based on the number of reversals that occur over a test...
Reversal learning depends on cognitive flexibility. Many reversal learning studies assess cognitive flexibility based on the number of reversals that occur over a test session. Reversals occur when an option is repeatedly chosen, e.g., eight times in a row. This design feature encourages win-stay behavior and thus makes it difficult to understand how win-stay decisions influence reversal performance. We used an alternative design, reversals over blocks of trials independent of performance, to study how perturbations of the medial orbital cortex and the noradrenergic system influence reversal learning. We found that choice accuracy varies independently of win-stay behavior and the noradrenergic system controls sensitivity to positive feedback during reversal learning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Topics: Prefrontal Cortex; Reversal Learning; Reward
PubMed: 34647770
DOI: 10.1037/bne0000495 -
Nature Neuroscience Dec 2023The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new...
The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.
Topics: Mice; Animals; Reward; Reinforcement, Psychology; Prefrontal Cortex; Reversal Learning
PubMed: 37957318
DOI: 10.1038/s41593-023-01485-3 -
Psychophysiology Mar 2022The process of learning allows organisms to develop predictions about outcomes in the environment, and learning is sensitive to both simple associations and higher order...
The process of learning allows organisms to develop predictions about outcomes in the environment, and learning is sensitive to both simple associations and higher order knowledge. However, it is unknown whether consciously attending to expectations shapes the learning process itself. Here, we directly tested whether rating expectations shapes arousal during classical conditioning. Participants performed an aversive learning paradigm wherein one image (CS+) was paired with shock on 50% of trials, while a second image (CS-) was never paired with shock. Halfway through the task, contingencies reversed. One group of participants rated the probability of upcoming shock on each trial, while the other group made no online ratings. We measured skin conductance response (SCR) evoked in response to the CS and used traditional analyses as well as quantitative models of reinforcement learning to test whether rating expectations influenced arousal and aversive reversal learning. Participants who provided online expectancy ratings displayed slower learning based on a hybrid model of adaptive learning and reduced reversal of SCR relative to those who did not rate expectations. Mediation analysis revealed that the effect of associative learning on SCR could be fully explained through its effects on subjective expectancy within the group who provided ratings. This suggests that the act of rating expectations reduces the speed of learning, likely through changes in attention, and that expectations directly influence arousal. Our findings indicate that higher order expectancy judgments can alter associative learning.
Topics: Adult; Arousal; Attention; Conditioning, Classical; Female; Galvanic Skin Response; Humans; Male; Reversal Learning; Young Adult
PubMed: 34837385
DOI: 10.1111/psyp.13979