Chapter 14

Choice

• Two-key procedure
• Concurrent schedules of reinforcement
• Each key associated with separate schedule
• Distribution of time and behaviour
• The measure of choice and preference

Concurrent Ratio Schedules

• Two ratio schedules
• Schedule that gives most rapid reinforcement chosen exclusively
• Rarely used in choice studies

Concurrent Interval Schedules

• Maximize reinforcement
• Must shift between alternatives
• Allows for study of choice behaviour

Interval Schedules

• FI-FI
• Steady-state responding
• Less useful/interesting
• VI-VI
• Not steady-state responding
• Respond to both alternatives
• Sensitive to rate of reinforcemenet
• Most commonly used to study choice

Alternation and the Changeover Response

• Maximize reinforcers from both alternatives
• Frequent shifting becomes reinforcing
• Simple alternation
• Concurrent superstition

Changeover Delay

• COD
• Prevents rapid switching
• Time delay after "changeover" before reinforcement possible

Herrnstein’s (1961) Experiment

• Concurrent VI-VI schedules
• Overall rates of reinforcement held constant
• 40 reinforcers/hour between two alternatives

(Figure 1)

The Matching Law

• The proportion of responses directed toward one alternative should equal the proportion of reinforcers delivered by that alternative.

(Figure 2)

Deviations from Matching

• Undermatching
• Most common
• Response proportions less extreme than reinforcement proportions
• Explanations
• Caused by COD?
• Overmatching
• Response proportions are more extreme than reinforcement proportions
• Rare
• Found when large penalty imposed for switching
• e.g., barrier between keys

Deviations

(Figure 3)

• Bias
• Spend more time on one alternative than predicted
• Side preferences
• Biological predispositions
• Quality and amount

Varying Quality of Reinforcers

• Q1: quality of first reinforcer
• Q2: quality of second reinforcer

(Figure 4)

Varying Amount of Reinforcers

• A1: amount of first reinforcer
• A2: amount of second reinforcer

(Figure 5)

Combining Qualities and Amounts

(Figure 6)

Matching as a Theory of Choice

• Animals match because they are evolved to do so.
• Nice, simple approach, but ultimately wrong.
• Consider a VR-VR schedule
• Exclusively choose one alternative
• Whichever is lower
• Matching law can’t explain this

Melioration Theory

• Invest effort in "best" alternative
• In VI-VI, partition responses to get best reinforcer:response ratio
• Overshooting the goal; feedback loop
• In VR-VR, keep shifting towards lower schedule; gives best reinforcer:response ratio
• Mixture of responding important over long run, but trial-by-trial responding shifts the balance

Optimization Theory

• Optimize reinforcement over long-term
• Minimum work for maximum gain
• Respond to both choices to maximize reinforcement

Momentary Maximization Theory

• Molecular theory
• Select alternative that has highest value at that moment
• Short-term vs. long-term benefits

Delay-reduction Theory

• Immediate or delayed reinforcement
• Basic principles of matching law, and...
• Choice directed towards whichever alternative gives greatest reduction in delay to next reinforcer
• Molar (matching response:reinforcement) and molecular (control by shorter delay) features