Mastering Extinction
Extinction happens all the time. When you withhold your click, you set up an extinction process.
If you are unclear about your criteria or clumsy in your handling skills, you could be setting up your learner for a macro extinction process with all of the painful emotions that go along with it.
Or you could be using a micro extinction strategy to help shape a more complex behavior. In this case you are using extinction to your advantage. Extinction doesn’t have to be something you avoid. It can be something you actively use to create more complex behavior patterns.
In yesterday’s post I described the PORTL games that Dr. Jesús Rosales-Ruiz uses to help his students understand principles of behavior. In his talks he shares some fascinating PORTL experiments to illustrate the difference between resurgence and regression.
Experiment One: Resurgence
The learner was taught a series of behaviors:
Behavior 1: tapping a small block. Once that behavior was confirmed, the block was removed and a toy car was placed on the table.
Behavior 2 was rolling the toy car over the table top. When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it. Pushing a car is an easy guess for what you would do with this kind of object.
When that behavior appeared to be solid, the car was removed and a third object, a key, was placed on the table. Now the behavior was lifting. Fingering a key is a normal response to this kind of object so it was easy to get the learner first to touch the key and then to lift it up off the table. Once the learner was consistently lifting the key, that object was removed and a fourth one was introduced.
Behavior 4 involved the learner putting a wooden ring on her finger. The learner quickly figured this out and began to consistently offer this behavior.
When each of these behaviors seemed solid – tapping the block, pushing the car, lifting the key, putting a ring on her finger – the trainer reviewed, one at a time, what the learner was to do with each of the objects.
The trainer then placed all four objects out on the table, but not in the order in which they had been taught. The trainer observed the learner’s behavior. She did not give any feedback or reinforcement of any kind. The point was to see in what order the learner would interact with each object.
The result: The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.
So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.
The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them, and then you remove reinforcement, you get a resurgence of these previously reinforced behaviors. They reoccur in the order in which they were trained.
Now here’s the fun part. When you instead extinguish the individual behaviors, you get the opposite result. Now you see regression. The individual will go back to the most recently learned behavior. If that doesn’t work, he’ll go a little further back, and then a little further back – thus revealing his training history.
In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.
In regression the order reverses. The most recently taught behavior reappears first.
These differences are illustrated in the second experiment.
Experiment Two: Regression
After a series of behaviors have been learned, this experiment again puts the learner through an extinction process. In the initial set up each time the learner is moved on to a new task, an extinction process is used to eliminate the previous behavior. Here’s the experiment:
The trainer sets out one item on the table. The learner begins to manipulate it, trying to find out what is going to be clickable. The trainer doesn’t click any of this creativity. She waits instead for it to extinguish and then clicks for one simple behavior – touching the object with one finger. That is the “hot” action.
The trainer clicks and reinforces for successful approximations until she has achieved a high degree of consistency in touching the object with one finger.
This was the set up for the experiment. In the next phase she sets ten different objects out in a circle, including the one they had just been working with. The learner begins by touching the familiar object. That gets clicked and reinforced several times, then the trainer stops reinforcing for that object. She is using extinction to eliminate that behavior. The learner begins by experimenting, touching various objects, but she only gets clicked for touching the one that was immediately next to the previously hot object in a counter clockwise direction.
The learner switches over to this object and begins touching it consistently.
So now the handler stops reinforcing for this object and only reinforces for the next object on the circle. The learner again experiments and then discovers that the only object that she gets paid for touching is the third one on the circle.
When this is consistent, the handler again stops reinforcing for touching this object. The learner is catching on to the overall pattern. Now she moves more quickly to the fourth object and discovers that is the “hot” one to touch.
They continue counter clockwise around the circle until every object has been the “hot” one once and touching it has also been extinguished.
At this point the handler stops reinforcing altogether and simply observes the learner’s behavior. The result: the learner quickly switches to moving clockwise around the circle, touching the objects in the reverse order in which she learned them. So she learned them originally counter clockwise: object 1, then object 2, then object 3, then object 4, etc.
Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc. She isn’t getting clicked for any of these touches, but the pattern is very persistent.
So again: in the first experiment where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.
In the second experiment where behaviors were extinguished, the learner went through them in the reverse order.
You won’t find these distinctions in the scientific literature. These two extinction outcomes, resurgence versus regression, are something Jesús and his students have been revealing by playing PORTL games.
Mind Games
Again Play is the key here. PORTL may have a serious purpose behind it, but these are games. All the creativity that comes with play is woven into these experiments. It may turn out that others playing with similar set ups will have different results. That’s a good thing. That simply raises more questions, more puzzles to solve.
Do you have a question about how something works? Great. Design an experiment, test it a few times to work out the kinks in the procedure, and then invite your friends over for a pizza and PORTL party. In the course of an evening you could have enough data to write a paper!
I do like the new twist Jesús has given to this version of the training game. As he has pointed out, we’ve been using lab rats to learn about human behavior. Now we are using humans to model animal behavior. Turnabout is fair play. Much better to frustrate an undergrad than some poor lab rat!
Coming Next: Eureka Moments! What is Insight?
Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.
I hope you will want to share these articles by sending links to this blog to your friends. But please remember this is copyrighted material. All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra Kurland, via theclickercenter.com
Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training. If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:
The Clicker Center Blog 