Download the Free Unbound MEDLINE PubMed App to your smartphone or tablet.
Available for iPhone, iPad, iPod touch, and Android.
operant conditioning [keywords]
- What learning theories can teach us in designing neurofeedback treatments. [JOURNAL ARTICLE]
- Front Hum Neurosci 2014.:894.
Popular definitions of neurofeedback point out that neurofeedback is a process of operant conditioning which leads to self-regulation of brain activity. Self-regulation of brain activity is considered to be a skill. The aim of this paper is to clarify that not only operant conditioning plays a role in the acquisition of this skill. In order to design the learning process additional references have to be derived from classical conditioning, two-process-theory and in particular from skill learning and research into motivational aspects. The impact of learning by trial and error, cueing of behavior, feedback, reinforcement, and knowledge of results as well as transfer of self-regulation skills into everyday life will be analyzed in this paper. In addition to these learning theory basics this paper tries to summarize the knowledge about acquisition of self-regulation from neurofeedback studies with a main emphasis on clinical populations. As a conclusion it is hypothesized that learning to self-regulate has to be offered in a psychotherapeutic, i.e., behavior therapy framework.
- Altered motivation masks appetitive learning potential of obese mice. [Journal Article]
- Front Behav Neurosci 2014.:377.
Eating depends strongly on learning processes which, in turn, depend on motivation. Conditioned learning, where individuals associate environmental cues with receipt of a reward, forms an important part of hedonic mechanisms; the latter contribute to the development of human overweight and obesity by driving excessive eating in what may become a vicious cycle. Although mice are commonly used to explore the regulation of human appetite, it is not known whether their conditioned learning of food rewards varies as a function of body mass. To address this, groups of adult male mice of differing body weights were tested two appetitive conditioning paradigms (pavlovian and operant) as well as in food retrieval and hedonic preference tests in an attempt to dissect the respective roles of learning/motivation and energy state in the regulation of feeding behavior. We found that (i) the rate of pavlovian conditioning to an appetitive reward develops as an inverse function of body weight; (ii) higher body weight associates with increased latency to collect food reward; and (iii) mice with lower body weights are more motivated to work for a food reward, as compared to animals with higher body weights. Interestingly, as compared to controls, overweight and obese mice consumed smaller amounts of palatable foods (isocaloric milk or sucrose, in either the presence or absence of their respective maintenance diets: standard, low fat-high carbohydrate or high fat-high carbohydrate). Notably, however, all groups adjusted their consumption of the different food types, such that their body weight-corrected daily intake of calories remained constant. Thus, overeating in mice does not reflect a reward deficiency syndrome and, in contrast to humans, mice regulate their caloric intake according to metabolic status rather than to the hedonic properties of a particular food. Together, these observations demonstrate that excess weight masks the capacity for appetitive learning in the mouse.
- Relative potency of varenicline or fluvoxamine to reduce responding for ethanol versus food depends on the presence or absence of concurrently earned food. [Journal Article, Research Support, N.I.H., Extramural, Research Support, Non-U.S. Gov't]
- Alcohol Clin Exp Res 2014 Mar; 38(3):860-70.
Varenicline, a nicotinic partial agonist, selectively reduces ethanol (EtOH)- versus sucrose-maintained behavior when tested in separate groups, yet like the indirect agonist fluvoxamine, this selectively inverts when EtOH and food are concurrently available.Here, we extend these findings by examining varenicline and fluvoxamine effects under a multiple concurrent schedule where food and EtOH are concurrently available in different components: Component 1 where the food fixed-ratio was 25 and Component 2 where the food fixed-ratio was 75. The EtOH fixed-ratio was always 5. Food-maintained responding predominated in Component 1, while EtOH-maintained responding predominated in Component 2. In a second experiment, varenicline effects were assessed under a multiple schedule where food, then EtOH, then again food were available in separate 5-minute components with fixed-ratios of 5 for each reinforcement.In the multiple concurrent schedule, varenicline was more potent at reducing food- versus EtOH-maintained responding in both components and reduced EtOH-maintained responding more potently during Component 1 (when food was almost never earned) than in Component 2 (where food was often earned). Fluvoxamine was similarly potent at reducing food- and EtOH-maintained responding. Under the multiple schedule, varenicline, like fluvoxamine, more potently decreases EtOH- versus food maintained responding when only food or EtOH is available in separate components.These results demonstrate that selective effects on drug- versus alternative-maintained behavior depend on the schedule arrangement, and assays in which EtOH or an alternative is the only programmed reinforcement may overestimate the selectivity of treatments to decrease EtOH self-administration. Thus selective effects obtained under one assay may not generalize to another. Better understanding the behavioral mechanisms responsible for these results may help to guide pharmaco-therapeutic development for substance use disorders.
- Volitional Weight-Lifting in Rats Promotes Adaptation via Performance and Muscle Morphology prior to Gains in Muscle Mass. [Journal Article]
- Environ Health Insights 2014; 8(Suppl 1):1-9.
Investigation of volitional animal models of resistance training has been instrumental in our understanding of adaptive training. However, these studies have lacked reactive force measurements, a precise performance measure, and morphological analysis at a distinct phase of training - when initial strength gains precede muscle hypertrophy. Our aim was to expose rats to one month of training (70 or 700 g load) on a custom-designed weight-lifting apparatus for analysis of reactive forces and muscle morphology prior to muscle hypertrophy. Exclusively following 700 g load training, forces increased by 21% whereas muscle masses remained unaltered. For soleus (SOL) and tibialis anterior (TA) muscles, 700 g load training increased muscle fiber number per unit area by ∼20% and decreased muscle fiber area by ∼20%. Additionally, number of muscle fibers per section increased by 18% for SOL muscles. These results establish that distinct morphological alterations accompany early strength gains in a volitional animal model of load-dependent adaptive resistance training.
- Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control. [Journal Article]
- Front Neural Circuits 2014.:126.
Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.
- Differential effects of clozapine, metoclopramide, haloperidol and risperidone on acquisition and performance of operant responding in rats. [JOURNAL ARTICLE]
- Psychopharmacology (Berl) 2014 Nov 11.
Prior research has not systematically investigated the effects of systemic antipsychotic drugs on operant response acquisition, specifically their behavioural microstructure, reinforcement blunting and relative potency in acquisition compared to performance once operant responding has stabilized.This study aims to systematically investigate the effects of systemically administered clozapine, metoclopramide, haloperidol and risperidone during free operant response acquisition and performance.Following magazine training, food-restricted male Wistar rats lever pressed for food reward in 15 min daily operant conditioning sessions.All drugs suppressed operant response acquisition and performance. Risperidone and metoclopramide, but not clozapine or haloperidol, suppressed operant responding more potently during acquisition than performance. The dopamine D2-like receptor antagonists haloperidol and metoclopramide that affect the ventral and dorsal striatum blunted reinforcement and decreased inactive lever presses in acquisition. In contrast, the atypical antipsychotics clozapine and risperidone that affect the ventral striatum and prefrontal cortex failed to decrease inactive lever presses during acquisition, suggesting a possible decision-making deficit. Haloperidol decreased active lever pressing over performance days. The drugs did not appear to affect rats' sensitivity to active lever press outcome, even though they suppressed active lever pressing.Results suggest that reinforcement impact during operant acquisition is dependent on dopamine D2 receptors while drugs affecting, among other areas, the prefrontal cortex produce a deficit in ability to suppress inactive lever press responses.
- Trained Quantity Abilities in Horses (Equus caballus): A Preliminary Investigation. [Journal Article]
- Behav Sci (Basel) 2014 Sep; 4(3):213-25.
Once believed to be a human prerogative, the capacity to discriminate between quantities now has also been reported in several vertebrates. To date, only two studies investigated numerical abilities in horses (Equus caballus) but reported contrasting data. To assess whether horses can be trained to discriminate between quantities, I have set up a new experimental protocol using operant conditioning. One adult female was trained to discriminate between 1 and 4 (Test 1) in three different conditions: non-controlled continuous variables (numerical and continuous quantities that co-vary with number are simultaneously available), 50% controlled continuous variables (intermediate condition), and 100% controlled continuous variables (only numerical information available). The subject learned the discrimination in all conditions, showing the capacity to process numerical information. When presented with a higher numerical ratio (2 vs. 4, Test 2), the subject still discriminated between the quantities but its performance was statistically significant only in the non-controlled condition, suggesting that the subject used multiple cues in presence of a more difficult discrimination. On the whole, the results here reported encourage the use of this experimental protocol as a valid tool to investigate the capacity to process numerical and continuous quantities in horses in future research.
- Stand-up exercise training facilitates muscle recovery from disuse atrophy by stimulating myogenic satellite cell proliferation in mice. [Journal Article]
- Physiol Rep 2014 Nov 1; 2(11)
Determining the cellular and molecular recovery processes in inactivity - or unloading -induced atrophied muscles should improve rehabilitation strategies. We assessed the effects of stand-up exercise (SE) training on the recovery of atrophied skeletal muscles in male mice. Mice were trained to stand up and press an elevated lever in response to a light-tone cue preceding an electric foot shock and then subjected to tail suspension (TS) for 2 weeks to induce disuse atrophy in hind limb muscles. After release from TS, mice were divided into SE-trained (SE cues: 25 times per set, two sets per day) and non-SE-trained groups. Seven days after the training, average myofiber cross-sectional area (CSA) of the soleus muscle was significantly greater in the SE-trained group than in the non-SE-trained group (1843 ± 194 μm(2) vs. 1315 ± 153 μm(2)). Mean soleus muscle CSA in the SE trained group was not different from that in the CON group subjected to neither TS nor SE training (2005 ± 196 μm(2)), indicating that SE training caused nearly complete recovery from muscle atrophy. The number of myonuclei per myofiber was increased by ~60% in the SE-trained group compared with the non-SE-trained and CON groups (0.92 ± 0.03 vs. 0.57 ± 0.03 and 0.56 ± 0.11, respectively). The number of proliferating myonuclei, identified by 5-ethynyl-2'-deoxyuridine staining, increased within the first few days of SE training. Thus, it is highly likely that myogenic satellite cells proliferated rapidly in atrophied muscles in response to SE training and fused with existing myofibers to reestablish muscle mass.
- Too difficult to stop: mechanisms facilitating relapse in alcohol dependence. [Journal Article]
- Neuropsychobiology 2014; 70(2):103-10.
In alcohol and other substance dependencies, patients often suffer relapse despite better knowledge and their intention to remain abstinent. A variety of neurotransmitter systems and their respective alterations due to the chronic drug intake are involved in mechanisms that facilitate relapse. It has been postulated that these neurotransmitter systems are related to changes in motivational and learning mechanisms, and engender a shift from goal-directed to habitual behavior in dependent patients that facilitates drug-seeking behavior.We review learning mechanisms facilitating relapse, as identified and tested to date. We focus on studies examining the interaction between alcohol-related changes in monoaminergic neurotransmission and their respective effects on pavlovian and operant learning mechanisms in alcohol dependence.Animal experiments and first human studies suggest that chronic alcohol intake impairs goal-directed behavior and facilitates habitual drug intake. Key symptoms of alcohol dependence such as tolerance development, withdrawal, craving and reduced control of alcohol intake can be explained by alcohol-induced alteration of dopaminergic neurotransmission and its GABAergic and glutamatergic modulation and their respective effects on pavlovian and operant conditioning as well as pavlovian-to-instrumental transfer.Chronic alcohol intake impairs neurotransmitter systems that regulate prefrontal-striatal circuits and interfere with goal-directed decision-making and the acquisition of new, non-drug-related behavior patterns. Alcohol craving induced by pavlovian conditioned cues can facilitate habitual drug intake. Such learning mechanisms and their alterations by chronic alcohol intake might be targeted by specific interventions. © 2014 S. Karger AG, Basel.
- Comparison of anterior cingulate vs. insular cortex as targets for real-time fMRI regulation during pain stimulation. [Journal Article]
- Front Behav Neurosci 2014.:350.
Real-time functional magnetic resonance imaging (rt-fMRI) neurofeedback allows learning voluntary control over specific brain areas by means of operant conditioning and has been shown to decrease pain perception. To further increase the effect of rt-fMRI neurofeedback on pain, we directly compared two different target regions of the pain network, notably the anterior insular cortex (AIC) and the anterior cingulate cortex (ACC). Participants for this prospective study were randomly assigned to two age-matched groups of 14 participants each (7 females per group) for AIC and ACC feedback. First, a functional localizer using block-design heat pain stimulation was performed to define the pain-sensitive target region within the AIC or ACC. Second, subjects were asked to down-regulate the BOLD activation in four neurofeedback runs during identical pain stimulation. Data analysis included task-related and functional connectivity analysis. At the behavioral level, pain ratings significantly decreased during feedback vs. localizer runs, but there was no difference between AIC and ACC groups. Concerning neuroimaging, ACC and AIC showed consistent involvement of the caudate nucleus for subjects that learned down-regulation (17/28) in both task-related and functional connectivity analysis. The functional connectivity toward the caudate nucleus is stronger for the ACC while the AIC is more heavily connected to the ventrolateral prefrontal cortex. Consequently, the ACC and AIC are suitable targets for real-time fMRI neurofeedback during pain perception as they both affect the caudate nucleus, although functional connectivity indicates that the direct connection seems to be stronger with the ACC. Additionally, the caudate, an important area involved in pain perception and suppression, could be a good rt-fMRI target itself. Future studies are needed to identify parameters characterizing successful regulators and to assess the effect of repeated rt-fMRI neurofeedback on pain perception.