-
Annual Review of Psychology 2003Operant behavior is behavior "controlled" by its consequences. In practice, operant conditioning is the study of reversible behavior maintained by reinforcement... (Review)
Review
Operant behavior is behavior "controlled" by its consequences. In practice, operant conditioning is the study of reversible behavior maintained by reinforcement schedules. We review empirical studies and theoretical approaches to two large classes of operant behavior: interval timing and choice. We discuss cognitive versus behavioral approaches to timing, the "gap" experiment and its implications, proportional timing and Weber's law, temporal dynamics and linear waiting, and the problem of simple chain-interval schedules. We review the long history of research on operant choice: the matching law, its extensions and problems, concurrent chain schedules, and self-control. We point out how linear waiting may be involved in timing, choice, and reinforcement schedules generally. There are prospects for a unified approach to all these areas.
Topics: Attention; Choice Behavior; Conditioning, Operant; Humans; Mental Recall; Motivation; Problem Solving; Reinforcement Schedule
PubMed: 12415075
DOI: 10.1146/annurev.psych.54.101601.145124 -
Behavioural Processes Jan 2014Contingency, and more particularly temporal contingency, has often figured in thinking about the nature of learning. However, it has never been formally defined in such... (Review)
Review
Contingency, and more particularly temporal contingency, has often figured in thinking about the nature of learning. However, it has never been formally defined in such a way as to make it a measure that can be applied to most animal learning protocols. We use elementary information theory to define contingency in such a way as to make it a measurable property of almost any conditioning protocol. We discuss how making it a measurable construct enables the exploration of the role of different contingencies in the acquisition and performance of classically and operantly conditioned behavior.
Topics: Animals; Association Learning; Conditioning, Classical; Conditioning, Operant; Reinforcement Schedule; Time Factors
PubMed: 23994260
DOI: 10.1016/j.beproc.2013.08.012 -
Behavioural Processes Apr 2024It is generally believed that termites can't learn and are not "intelligent". This study aimed to test whether termites could have any form of memory. A Y-shaped test...
It is generally believed that termites can't learn and are not "intelligent". This study aimed to test whether termites could have any form of memory. A Y-shaped test device with one release chamber and two identical test chambers was designed and constructed by 3D printing. A colony of damp wood termites was harvested from the wild. Worker termites were randomly selected for experiment. Repellent odors that could mimic the alarm pheromone for termites were first identified. Among all substances tested, a tea tree oil and lemon juice were found to contain repellent odors for the tested termites, as they significantly reduced the time that termites spent in the chamber treated with these substances. As control, a trail pheromone was found to be attractive. Subsequently, a second cohort of termites were operant conditioned by punishment using both tea tree oil and lemon juice, and then tested for their ability to remember the path that could lead to the repellant odors. The test device was thoroughly cleaned between trials. It was found that conditioned termites displayed a reduced tendency to choose the path that led to expectant punishment as compared with naïve termites. Thus, it is concluded that damp wood termites are capable of learning and forming "fear memory", indicative of "intelligence" in termites. This result challenges established presumption about termites' intelligence.
Topics: Isoptera; Animals; Odorants; Conditioning, Operant; Pheromones; Memory; Learning; Tea Tree Oil; Citrus; Insect Repellents; Behavior, Animal; Punishment
PubMed: 38493970
DOI: 10.1016/j.beproc.2024.105012 -
Behavioural Processes Mar 2023Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this... (Review)
Review
Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.
Topics: Animals; Conditioning, Operant; Extinction, Psychological; Learning; Punishment; Inhibition, Psychological
PubMed: 36702436
DOI: 10.1016/j.beproc.2023.104830 -
The Journal of Neuroscience : the... Feb 2022Despite numerous studies examining the mechanisms of operant conditioning (OC), the diversity of OC plasticity loci and their synergism have not been examined...
Despite numerous studies examining the mechanisms of operant conditioning (OC), the diversity of OC plasticity loci and their synergism have not been examined sufficiently. In the well-characterized feeding neural circuit of , and appetitive OC increases neuronal excitability and electrical coupling among several neurons leading to an increase in expression of ingestive behavior. Here, we used the analog of OC to investigate whether OC reduces the excitability of a neuron, B4, whose inhibitory connections decrease expression of ingestive behavior. We found OC decreased the excitability of B4. This change appeared intrinsic to B4 because it could be replicated with an analog of OC in isolated cultures of B4 neurons. In addition to changes in B4 excitability, OC decreased the strength of B4's inhibitory connection to a key decision-making neuron, B51. The OC-induced changes were specific without affecting the excitability of another neuron critical for feeding behavior, B8, or the B4-to-B8 inhibitory connection. A conductance-based circuit model indicated that reducing the B4-to-B51 synapse, or increasing B51 excitability, mediated the OC phenotype more effectively than did decreasing B4 excitability. We combined these modifications to examine whether they could act synergistically. Combinations including B51 synergistically enhanced feeding. Taken together, these results suggest modifications of diverse loci work synergistically to mediate OC and that some neurons are well suited to work synergistically with plasticity in other loci. The ways in which synergism of diverse plasticity loci mediate the change in motor patterns in operant conditioning (OC) are poorly understood. Here, we found that OC was in part mediated by decreasing the intrinsic excitability of a critical neuron of feeding behavior, and specifically reducing the strength of one of its inhibitory connections that targets a key decision-making neuron. A conductance-based computational model indicated that the known plasticity loci showed a surprising level of synergism to mediate the behavioral changes associated with OC. These results highlight the importance of understanding the diversity, specificity and synergy among different types of plasticity that encode memory. Also, because OC in is mediated by dopamine (DA), the present study provides insights into specific and synergistic mechanisms of DA-mediated reinforcement of behaviors.
Topics: Animals; Aplysia; Computer Simulation; Conditioning, Operant; Models, Neurological; Neuronal Plasticity; Neurons
PubMed: 34992131
DOI: 10.1523/JNEUROSCI.1722-21.2021 -
ENeuro 2022Operant chambers are widely used in animal research to study cognition, motivation, and learning processes. Paired with the rapidly developing technologies for brain...
Operant chambers are widely used in animal research to study cognition, motivation, and learning processes. Paired with the rapidly developing technologies for brain imaging and manipulations of brain activity, operant conditioning chambers are a powerful tool for neuroscience research. The behavioral testing and imaging setups that are commercially available are often quite costly. Here, we present a custom-built operant chamber that can be constructed in a few days by an unexperienced user with relatively inexpensive, widely available materials. The advantages of our operant setup compared with other open-source and closed-source solutions are its relatively low cost, its support of complex behavioral tasks, its user-friendly setup, and its validated functionality with video imaging of behavior and calcium imaging using the UCLA Miniscope. Using this setup, we replicate our previously published findings showing that mice exposed to social defeat stress in adolescence have inhibitory control impairments in the Go/No-Go task when they reach adulthood. We also present calcium imaging data of medial prefrontal cortex (mPFC) neuronal activity acquired during Go/No-Go testing in freely moving mice and show that neuronal population activity increases from day 1 to day 14 of the task. We propose that our operant chamber is a cheaper alternative to its commercially available counterparts and offers a better balance between versatility and user-friendly setup than other open-source alternatives.
Topics: Animals; Calcium; Cognition; Conditioning, Operant; Learning; Mice; Neuropsychological Tests
PubMed: 35105659
DOI: 10.1523/ENEURO.0430-21.2022 -
ELife Oct 2022Learning which stimuli (classical conditioning) or which actions (operant conditioning) predict rewards or punishments can improve chances of survival. However, the...
Learning which stimuli (classical conditioning) or which actions (operant conditioning) predict rewards or punishments can improve chances of survival. However, the circuit mechanisms that underlie distinct types of associative learning are still not fully understood. Automated, high-throughput paradigms for studying different types of associative learning, combined with manipulation of specific neurons in freely behaving animals, can help advance this field. The larva is a tractable model system for studying the circuit basis of behaviour, but many forms of associative learning have not yet been demonstrated in this animal. Here, we developed a high-throughput (i.e. multi-larva) training system that combines real-time behaviour detection of freely moving larvae with targeted opto- and thermogenetic stimulation of tracked animals. Both stimuli are controlled in either open- or closed-loop, and delivered with high temporal and spatial precision. Using this tracker, we show for the first time that larvae can perform classical conditioning with no overlap between sensory stimuli (i.e. trace conditioning). We also demonstrate that larvae are capable of operant conditioning by inducing a bend direction preference through optogenetic activation of reward-encoding serotonergic neurons. Our results extend the known associative learning capacities of larvae. Our automated training rig will facilitate the study of many different forms of associative learning and the identification of the neural circuits that underpin them.
Topics: Animals; Conditioning, Operant; Drosophila; Larva; Drosophila melanogaster; Conditioning, Classical
PubMed: 36305588
DOI: 10.7554/eLife.70015 -
Neurotherapeutics : the Journal of the... Jul 2018Neurological disorders, such as spinal cord injury, stroke, traumatic brain injury, cerebral palsy, and multiple sclerosis cause motor impairments that are a huge burden... (Review)
Review
Neurological disorders, such as spinal cord injury, stroke, traumatic brain injury, cerebral palsy, and multiple sclerosis cause motor impairments that are a huge burden at the individual, family, and societal levels. Spinal reflex abnormalities contribute to these impairments. Spinal reflex measurements play important roles in characterizing and monitoring neurological disorders and their associated motor impairments, such as spasticity, which affects nearly half of those with neurological disorders. Spinal reflexes can also serve as therapeutic targets themselves. Operant conditioning protocols can target beneficial plasticity to key reflex pathways; they can thereby trigger wider plasticity that improves impaired motor skills, such as locomotion. These protocols may complement standard therapies such as locomotor training and enhance functional recovery. This paper reviews the value of spinal reflexes and the therapeutic promise of spinal reflex operant conditioning protocols; it also considers the complex process of translating this promise into clinical reality.
Topics: Animals; Conditioning, Operant; Humans; Neuronal Plasticity; Reflex; Spinal Cord Injuries
PubMed: 29987761
DOI: 10.1007/s13311-018-0643-2 -
Scientific Reports Feb 2021Altered functioning of GABAergic interneurons expressing parvalbumin (PV) in the basal ganglia-thalamo-cortical circuit are likely to be involved in several human...
Altered functioning of GABAergic interneurons expressing parvalbumin (PV) in the basal ganglia-thalamo-cortical circuit are likely to be involved in several human psychiatric disorders characterized by deficits in attention and sensory gating with dysfunctional decision-making behavior. However, the contribution of these interneurons in the ability to acquire demanding learning tasks remains unclear. Here, we combine an operant conditioning task with local field potentials simultaneously recorded in several nuclei involved in reward circuits of wild-type (WT) and PV-deficient (PVKO) mice, which are characterized by changes in firing activity of PV-expressing interneurons. In comparison with WT mice, PVKO animals presented significant deficits in the acquisition of the selected learning task. Recordings from prefrontal cortex, nucleus accumbens (NAc) and hippocampus showed significant decreases of the spectral power in beta and gamma bands in PVKO compared with WT mice particularly during the performance of the operant conditioning task. From the first to the last session, at all frequency bands the spectral power in NAc tended to increase in WT and to decrease in PVKO. Results indicate that PV deficiency impairs signaling necessary for instrumental learning and the recognition of natural rewards.
Topics: Animals; Conditioning, Operant; GABAergic Neurons; Interneurons; Male; Mice; Mice, Knockout; Models, Animal; Parvalbumins; Reward; Sensory Gating
PubMed: 33536607
DOI: 10.1038/s41598-021-82519-3 -
Journal of Visualized Experiments : JoVE Aug 2022The Evoked Potential Operant Conditioning System (EPOCS) is a software tool that implements protocols for operantly conditioning stimulus-triggered muscle responses in...
The Evoked Potential Operant Conditioning System (EPOCS) is a software tool that implements protocols for operantly conditioning stimulus-triggered muscle responses in people with neuromuscular disorders, which in turn can improve sensorimotor function when applied appropriately. EPOCS monitors the state of specific target muscles-e.g., from surface electromyography (EMG) while standing, or from gait cycle measurements while walking on a treadmill-and automatically triggers calibrated stimulation when pre-defined conditions are met. It provides two forms of feedback that enable a person to learn to modulate the targeted pathway's excitability. First, it continuously monitors ongoing EMG activity in the target muscle, guiding the person to produce a consistent level of activity suitable for conditioning. Second, it provides immediate feedback of the response size following each stimulation and indicates whether it has reached the target value. To illustrate its use, this article describes a protocol through which a person can learn to decrease the size of the Hoffmann reflex-the electrically-elicited analog of the spinal stretch reflex-in the soleus muscle. Down-conditioning this pathway's excitability can improve walking in people with spastic gait due to incomplete spinal cord injury. The article demonstrates how to set up the equipment; how to place stimulating and recording electrodes; and how to use the free software to optimize electrode placement, measure the recruitment curve of direct motor and reflex responses, measure the response without operant conditioning, condition the reflex, and analyze the resulting data. It illustrates how the reflex changes over multiple sessions and how walking improves. It also discusses how the system can be applied to other kinds of evoked responses and to other kinds of stimulation, e.g., motor evoked potentials to transcranial magnetic stimulation; how it can address various clinical problems; and how it can support research studies of sensorimotor function in health and disease.
Topics: Chronic Disease; Conditioning, Operant; Electromyography; Evoked Potentials; H-Reflex; Humans; Neuromuscular Diseases; Spinal Cord Injuries
PubMed: 36094287
DOI: 10.3791/63736