Individual differences in proactive interference in rats (Rattus Norvegicus)

Individual differences in behaviors are seen across many species, and investigations have focused on traits linked to aggression, risk taking, emotionality, coping styles, and differences in cognitive systems. The current study investigated whether there were individual differences in proactive interference tasks in rats (Rattus Norvegicus), and tested hypotheses suggesting that these tasks should load onto a single factor and there should be clusters of rats who perform well or poorly on these tasks. The performance of 39 rats was tested across three learning tasks that all involved disengagement from an irrelevant previously learned stimulus to a relevant stimulus: latent inhibition (LI), partial reinforcement extinction effect (PREE), and reversal learning (RL). An exploratory factor analysis revealed the existence of one factor underlying performance. A cluster analysis revealed the existence of sets of rats displaying either weak LI and strong PREE and RL effects, or vice versa. These findings suggest that proactive interference may be based on a single underlying psychological system in rats.

Extinction of Pavlovian Conditioning: The Influence of Trial Number and Reinforcement History

Pavlovian conditioning is sensitive to the temporal relationship between conditioned stimulus (CS) and unconditioned stimulus (US). This has motivated models that describe learning as a process that continuously updates associative strength during the trial or specifically encodes the CS-US interval. These models predict that extinction of responding is also continuous, such that response loss is proportional to the cumulative duration of exposure to the CS without the US. We review evidence showing that this prediction is incorrect, and that extinction is trial-based rather than time-based. We also present two experiments that test the importance of trials versus time on the Partial Reinforcement Extinction Effect (PREE), in which responding extinguishes more slowly for a CS that was inconsistently reinforced with the US than for a consistently reinforced one. We show that increasing the number of extinction trials of the partially reinforced CS, relative to the consistently reinforced CS, overcomes the PREE. However, increasing the duration of extinction trials by the same amount does not overcome the PREE. We conclude that animals learn about the likelihood of the US per trial during conditioning, and learn trial-by-trial about the absence of the US during extinction. Moreover, what they learn about the likelihood of the US during conditioning affects how sensitive they are to the absence of the US during extinction.

The Partial Reinforcement Extinction Effect Depends on Learning about Non-Reinforced Trials Rather than Reinforcement Rate

Conditioned responding extinguishes more slowly after partial (inconsistent) reinforcement than after consistent reinforcement. This Partial Reinforcement Extinction Effect (PREE) is usually attributed to learning about nonreinforcement during the partial schedule. An alternative explanation attributes it to any difference in the rate of reinforcement, arguing that animals can detect the change to nonreinforcement more quickly after a denser schedule than a leaner schedule. Experiments 1a and 1b compared extinction of magazine responding to a conditioned stimulus (CS) reinforced with one food pellet per trial and a CS reinforced with two pellets per trial. Despite the difference in reinforcement rate, there was no reliable difference in extinction. Both experiments did demonstrate the conventional PREE comparing a partial CS (50% reinforced) with a consistent CS. Experiments 2 and 3 tested whether the PREE depends specifically on learning about nonreinforced trials during partial reinforcement. Rats were trained with two CS configurations, A and AX. One was partially reinforced, the other consistently reinforced. When AX was partial and A consistent, responding to AX extinguished more slowly than to A. When AX was consistent and A was partial, there was no difference in their extinction. Therefore, pairing X with partial reinforcement allowed rats to show a PREE to AX that did not generalise to A. Pairing A with partial reinforcement meant that rats showed a PREE to A that generalised to AX. Thus, the PREE depends on learning about nonreinforced trials during partial reinforcement and is not due to any difference in per-trial probability of reinforcement

The Partial-Reinforcement Extinction Effect Does Not Result from Reduced Sensitivity to Non-Reinforcement

Five experiments used a magazine approach paradigm with rats to investigate whether learning about non-reinforcement is impaired in the presence of a conditioned stimulus (CS) that had been partially reinforced (PRf). Experiment 1 trained rats with a PRf CS and a continuously reinforced (CRf) CS, then extinguished responding to both CSs presented together as a compound. Probe trials of each CS presented alone revealed that extinction was slower for the PRf CS than the CRf CS, despite being extinguished in compound. In Experiment 2, a CRf light was extinguished in compound with either a CRf CS or a PRf CS that had been matched for overall reinforcement rate. Responding to the light extinguished at the same rate regardless of the reinforcement schedule of the other CS. Experiment 3 replicated this result with a PRf light. Thus, we found no evidence that a PRf CS impairs extinction of another CS presented at the same time. Experiments 4 and 5 extended this approach to study the acquisition of conditioned inhibition by training an inhibitor in compound with either a PRf or CRf excitatory CS. The reinforcement schedule of the excitatory CS had no effect on the acquisition of inhibition. In sum, conditioning with a PRf schedule slows subsequent extinction of that CS but does not affect learning about the non-reinforcement of other stimuli presented at the same time. We conclude that the Partial Reinforcement Extinction Effect is not due to a decrease in sensitivity to non-reinforcement following presentation of a PRf CS.

The Partial Reinforcement Extinction Effect: The Proportion of Trials Reinforced During Conditioning Predicts the Number of Trials to Extinction

Four experiments compared the extinction of responding to a continuously reinforced (CRf) conditioned stimulus (CS) consistently reinforced on every trial, with extinction of responding to a partially reinforced (PRf) CS that had been inconsistently reinforced. To equate the acquisition of responding between the two CSs, the average duration of the CRf CS was extended so that it scheduled the same overall rate of reinforcement per unit time as the PRf CS. Experiment 1 used a within-subjects design to compare the rates of extinction for a 10-s PRf CS reinforced on 33% of trials versus a 30-s CRf CS. Experiment 2 made the same comparison but using a between-subjects design. Experiment 3 compared extinction in a group trained with a 10-s PRf CS reinforced on 20% of trials and a group trained with a 50-s CRf CS. Experiment 4 compared the rates of extinction following two partial reinforcement schedules, a 10-s PRf CS reinforced on 33% of trial versus a 20-s CRf CS reinforced on 66% of trials. In each experiment, responding took longer to extinguish for the CS that scheduled a lower per-trial probability of reinforcement. Modelling of individual extinction curves using Weibull functions indicated that the latency to initiate extinction was directly related to the per-trial probability of reinforcement learned during acquisition. For example, compared to training with a CRf CS, rats reinforced on 33% of trials took approximately three times as many trials to initiate extinction, and rats reinforced on 20% of trials took five times as many trials to initiate extinction. These results provide support for trial-based accounts of extinction (e.g. Capaldi, 1967), whereby rats learn about the expected number of trials per reinforcer, and extinction depends on the number of expected reinforcers that have been omitted rather than on the number of extinction trials per se.

The Modified Law of Effect Explains the Partial Reinforcement Extinction Effect

Thorndike’s Law of Effect provides a framework for understanding the selection of behaviors given specific environmental reward contingencies. Though a highly influential model, especially given its resurgence in popularity to understand habitual behaviors, it fails to predict several well-documented behavioral phenomena and incorrectly views extinction as the unlearning of a previously acquired association. Blaisdell, Stolyarova, & Stahlman (2016) proposed modifications to Thorndike’s original law that address these issues and greatly increases the model’s explanatory power. This modified Law of Effect (MLOE) also provides a testable account of the Partial Reinforcement Extinction Effect (PREE). The PREE is the paradoxical finding of more rapid extinction to a continuously reinforced cue than to a partially reinforced cue, and has challenged many theoretical accounts of learning. Simulations of the MLOE confirm these predictions. Two experimental paradigms, one using pigeons and the other using humans, show support for the Modified Law of Effect’s explanation of the PREE.

Hippocampal-amygdala interactions mediate uncertainty-dependent resistance to extinction following fear conditioning

The partial reinforcement extinction effect (PREE) is a paradoxical learning phenomenon in which omission of reinforcement during acquisition results in more persistent conditioned responding in extinction. Here, we report a significant PREE with an inverted-U, entropy-like distribution against reinforcement probability following tone foot shock fear conditioning in rats, which was associated with increased neural activity in hippocampus and amygdala as indexed by p-ERK and c-fos immunolabelling. In vivo electrophysiological recordings of local field potentials (LFPs) showed that 50% reinforcement was associated with increases in the frequency and power of tone-evoked theta oscillations in both the subiculum region of hippocampus and in basolateral amygdala (BLA) during both acquisition (Day 1) and extinction (Day 2) sessions. Tone-evoked LFPs in 50% reinforced animals also showed increases in coherence and bidirectional Granger Causality between hippocampus and amygdala. The results support a Bayesian interpretation of the PREE, in which the phenomenon is driven by increases in the entropy or uncertainty of stimulus contingencies, and indicate a crucial role for hippocampus in mediating this uncertainty-dependent effect.