Lecture Notes on Timing and Schedules of Reinforcement in Operant Conditioning

Overview

• Topics Covered: Operant conditioning effectiveness, timing, schedules of reinforcement, and brain substrates related to operant conditioning.

Timing in Operant Conditioning

• Importance: Closer in time the behavior and the consequence, better the learning.
• Example: Zero-second delay leads to faster learning versus a four-second or ten-second delay.

Schedules of Reinforcement

General Idea

• The form of the contingency (S-R-O) controls the behavior pattern.
• Types:
  • Fixed Ratio (FR): A set number of responses lead to reinforcement.
    • Example: FR1 (every press gets a pellet), FR2 (every second press gets a pellet).
  • Variable Ratio (VR): Number of responses required for reinforcement varies.
    • Example: VR6 (on average, six presses lead to reinforcement).
  • Fixed Interval (FI): First response after a set time is reinforced.
    • Example: FI 5-min (first press after five minutes gets a pellet).
  • Variable Interval (VI): First response after varying intervals is reinforced.
    • Example: VI 5-min (on average, first press after five minutes is reinforced).

Behavioral Effects of Schedules

• Fixed Ratio & Variable Ratio: Variable ratio leads to a higher and steadier rate of responding compared to fixed ratio, which shows pauses after reinforcement.
• Fixed Interval & Variable Interval: Variable interval produces steady responding, while fixed interval shows a scallop-shaped pattern.

Real-life Examples

• Variable Ratio: Slot machines (gambling).
• Fixed Ratio: Completing assignments (e.g., every ten questions finished).

Brain Substrates in Operant Conditioning

Basal Ganglia

• Components: Caudate, putamen, Globus pallidus.
• Function: Important for S-R learning (Stimulus-Response associations).
• Lesions: Affect S-R learning but not simple R-O learning (Response-Outcome).

Orbitofrontal Cortex (OFC)

• Function: Important for R-O associations (Response-Outcome learning).
• Lesions: Affect response-outcome learning but not S-R associations.

Dopaminergic System

• Structures: Ventral tegmental area (VTA), substantia nigra (SNc).
• Function: Key in motivation and reward (wanting vs. liking).
• Antagonists (e.g., Pimozide): Disrupt operant conditioning.
• Endogenous Opioids: Linked to the 'liking' aspect of rewards.

Behavioral Economics

• Matching Law: Response rates to concurrent variable interval schedules reflect the rate of reinforcement.
• Bliss Point: Individual allocation of resources to maximize subjective value.
• Premack Principle: High-frequency behaviors can reinforce low-frequency behaviors.
• Response Deprivation Hypothesis: Deprivation makes a behavior a potential reinforcer.

Clinical Perspectives

Schizophrenia

• Symptoms: Hallucinations, delusions, flattened affect, social impairment.
• Hippocampal Abnormalities: Impaired relational processing, as seen in tasks like acquired equivalence.
• Antipsychotics: Partially improve associative learning impairments.

Addiction and Operant Conditioning

• Pathological Addiction: Maintained despite negative consequences.
• Positive/Negative Reinforcement: Positive for initial use, negative for avoiding withdrawal.
• Incentive Salience Hypothesis: Addiction is maintained by 'wanting' more than 'liking'.
• Brain Mechanisms: Dopamine release is key for addiction; drugs like Naltrexone can reduce reinforcement.

Treatments

• Pharmacological: Dopamine inhibitors (e.g., Naltrexone) for addiction.
• Behavioral: Extinction, reinforcing alternative behaviors, delayed reinforcement, distancing from triggers.