5GoToSea: Part 12: Creativity Explored

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL
Part 11: Mastering Extinction
Part 12: Creativity Explored

If you are new to this series, I suggest you begin with Part 1.

Part 12: Creativity Explored

Creativity Explored
Do you have a training-related question?  Are you wondering which teaching strategy will work the best?  Are you puzzling over some element in learning theory?  Great.  Design a set up, test it a few times to work out the procedure and then invite your friends over for a pizza and PORTL party.  In the course of an evening you’ll be well on your way to answering your questions, and you might even have collected data to write a scientific paper!

I do like the new name that’s been given to this shaping game, PORTL: Portable Operant Research and Teaching Laboratory.  As Jesús pointed out, we’ve been using lab rats to learn about human behavior.  Now we are doing the reverse.  We’re using humans as a model for animal behavior. Turnabout is fair play.  Much better to frustrate an undergrad than some poor lab rat!

So how does this help us?  Jesús shared several examples where extinction strategies were used to train complex, creative behaviors.

What Is Insight?
The first used a great video from Robert Epstein’s work.  Epstein was B.F. Skinner’s last graduate student.  They were exploring the concept of “insight”.  How do we solve puzzles?  Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into chamber with the box and the banana.  The banana was hung up out of reach.  The pigeon couldn’t reach the banana to peck at it, so an extinction process began.  There was a resurgence of previously trained behaviors. The pigeon began pushing the box, but it was very clear the behavior was deliberate.  The pigeon pushed the box under the banana, hopped up on the box, and pecked the banana. Puzzle solved.

So here’s the question: what is insight? What really is creativity?

Jesús would say this kind of complex puzzle solving was achieved through resurgence. Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, you have to extend the extinction process and hope the subject really does become creative.  But this is the kind of creativity that is truly stressful. It’s much better to analyze the complex end-goal behavior, break it down into all of it’s component tasks, and then train each of the components separately.  The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Inventors and Creators
Jesús’ comment was there is “nothing new under the sun”.  You are not going to do anything that you haven’t already done before.  All the components of what appears to be a novel behavior have been trained in the past. When individuals experiment and find the usual, familiar things aren’t working, some will give up.  Others will keep trying until they come up with a novel combination that works.

We call these people inventors and creators because they are able to find these novel combinations.  That first learning process can be a painful one, but once the new combination is worked out, it’s easy for others to copy the results.

I can absolutely relate to this.  Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off.  But put me in front of a computer that isn’t cooperating, and I shut down fast.  There my experience has produced very different expectations.  I’ve experienced too many situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills.  I don’t have the component skills that make wrestling with a software issue fun.  Extinction has gone too far and been too uncomfortable.  So in one situation I can be very persistent and creative.  In another I’m the one going through the classic cycle of an extinction shut down.

I know first hand both how much fun the creative process can be when the expectation of success is there.  And I also know how painful and unpleasant the extinction process is when that expectation is missing.  So here’s the question: which one are you creating for your horses, your dogs, your children – for yourself?

Coming soon: Part 13: Degrees of Freedom

You can watch the pigeon experiment in this video clip:

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 11: Mastering Extinction

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL
Part 11: Mastering Extinction

If you have not yet read the previous articles, I suggest you begin with Part 1.

Part 11: Mastering Extinction

Using PORTL to Master Extinction
Extinction happens all the time.  When you withhold your click, you set up an extinction process.

If you withhold the click because you are unclear about your criteria or you’re clumsy in your handling skills, you could be setting up your learner for a macro extinction with all of the painful emotions that go along with it.

Or you could be withholding the click with very deliberate intent.  In this case you are using a micro extinction process to help shape a more complex behavior.  You are using extinction to your advantage.  The conclusion: extinction doesn’t have to be something you avoid.  It can be something you actively use to create more complex behavior patterns.

Today’s post shows how the shaping game, PORTL can help us understand how this works.

Jesús watching participants at a conference learning to play PORTL.

Jesús coaching participants at a conference as they learn how to play PORTL.

Shaping with Resurgence
Jesús used videos of PORTL experiments to illustrate what he meant.  He reminded us that there is a difference between resurgence and regression.  The first video example showed an elegant use of resurgence.

The learner was taught Behavior 1: tapping a small block.  Once that behavior was confirmed the block was removed and a toy car was placed on the table.

 

Behavior 2 was rolling the toy car over the table top.  When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it.

When that behavior appeared to be solid, the car was removed and a third object was placed on the table.  Now the behavior was lifting.

Behavior 4 was a different action.  The learner put a wooden ring on her finger.

When each of these behaviors seemed solid, the trainer reviewed one at a time what the learner was to do with each of the objects.

The trainer then placed all four objects out on the table but not in the order in which they had been taught.  There was now no reinforcement given.  The trainer was simply observing the learner’s behavior – not giving any feedback or reinforcement of any kind.  The point was to see in what order the learner would interact with each object.

The result:  The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.

So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.

The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them and then you remove reinforcement, the behaviors occur in the order in which they were trained.  This is resurgence as opposed to regression.

Shaping with Regression
Now here’s the fun part.  When you first extinguish the individual behaviors, you get the opposite result.  Now you see regression.  People will go back to the most recently learned behavior.  If that doesn’t work, they’ll go a little further back, and then a little further back – thus revealing their training history.

Jesús showed a second video, this one was exploring what happens in a shaping session where you reinforce an approximation, and then go through an extinction process so you can switch to a new behavior.  Here’s the set up:

The trainer set out one item on the table.  The learner began to manipulate it, trying to find out what was going to be clickable.  The trainer didn’t click any of this creativity.  She waited for it to extinguish and then clicked for one simple behavior – touching the object with one finger.  That was the “hot” action.

The trainer clicked and reinforced for successful approximations, then she took a break to record her data.  She continued to train in ten click units until she had achieved a high degree of consistency in touching the object with one finger.

This was the set up for the experiment.  In the next phase she set out in a circle nine or ten different objects, including the one they had been working with.  The learner began by touching the familiar object.  That got clicked and reinforced several times, then the trainer stopped reinforcing for that object.  She was using extinction to eliminate that behavior.  The learner began experimenting, touching various objects, but she only got clicked for touching the one that was immediately next to the previously hot object in the counter-clockwise direction.

The learner switched over to this object and began touching it consistently.

So now the handler stopped reinforcing for this object and only reinforced for the next object on the circle.  The learner again experimented and then discovered that the only object that she got paid for touching was the third one on the circle.

When this was consistent, the handler again stopped reinforcing for touching this object.  The learner was catching on to the overall pattern.  Now she moved more quickly to the fourth object and discovered that was the “hot” one to touch.

They continued counter-clockwise around the circle until every object had been the “hot” one once and touching it had also been extinguished.

At this point the handler stopped reinforcing for anything and observed the learner’s behavior. The result: the learner quickly switched to moving clockwise around the circle, touching  the objects in the reverse order in which she had learned them.  So she learned them originally counter-clockwise: object 1, then object 2, then object 3, then object 4, etc.

Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc.  She wasn’t getting clicked for any of these touches, but the pattern was very persistent.

What PORTL Reveals
So again: in the first video where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.

In the second video where behaviors were extinguished, the learner went through them in the reverse order.

You won’t find these distinctions in the scientific literature.  This difference in behavior relating to resurgence and regression is something Jesús and his students have been revealing by playing PORTL.

Coming Soon: Part 12: Creativity Explored

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 10: PORTL

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 10 of a 15 part series.
Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL

If you have not yet read the previous articles, I suggest you begin with Part 1.
Part 10: PORTL

PORTL’s Origins

Jesús and the son of one of the conference attendees playing PORTL during the Five Go To Sea cruise.

Jesús and the son of one of the conference attendees playing PORTL during the Five Go To Sea cruise.

PORTL evolved out of Genabacab, a table game Kay Laurence developed for teaching shaping.  Genabacab has very few instructions and really only one rule: the only person who is allowed to talk is the learner.  The trainer and spectators are not to give any verbal hints or to discuss what is going on until afterwards.

The table game lets you work out shaping plans BEFORE you go to your animal.  Do you want to learn how to attach a cue to a behavior and then change that cue to a new cue?  You can work out the process playing the table game and spare your animals the frustration of your learning curve.

Kay has described workshops at her training center where someone arrives with a “how do I teach this?” type of question. Maybe the handler wants to teach match to sample, or she wants to see if her dog can indicate which object is the biggest one in a set. Instead of going straight out to the dog and confusing it with missteps and false starts, everyone in the group will pull out their Genabacab games.  They will spend hours happily absorbed in developing the best teaching strategies for their dogs.  Their dogs, meanwhile, are spending the day relaxing while their people work away at the puzzle. It’s only once the process is well understood, that the dogs are brought in for training.

Jesús has been using Genabacab to help his students understand the concepts of learning theory.  He wants to bring the game to the scientific community as a research tool, so – with Kay’s blessing – he has renamed it.  It is now PORTL – Portable Operant Research and Teaching Laboratory.  Animal studies are increasingly difficult to do. They are expensive, and there is always the question of ethics.  How fair is it to run studies on lab rats?  PORTL is a much better solution.  You can have a question about how a particular process works, design an experiment using the PORTL game, and in the course of an evening have gathered enough data playing the game with a group of undergrads to write a paper – all without frustrating a single lab rat.  Now that’s progress!

PORTL Games
Jesús’ students meet on a regular basis to play PORTL games.  In his talk he showed some videos that illustrated beautifully how he used it to ask questions about regression and resurgence.  In one video two tasks were taught.  First, the learner was shaped to place one hand over the other – right hand over left, and then to reverse it – left hand over right.  The behavior was put on a fixed ratio of 5.  That means the learner was clicked and reinforced on every fifth swap of hands.

The second task was tapping a block.  Again, the learner was put on a fixed ratio of 5. (The learner was to tap the block five times for each click and treat.)

The trainer then increased the ratio for the tapping to 30.  The learner began  to tap the block, but now there was no click and treat after 5 taps.  The learner kept going to about 13 taps.  At that point she began to experiment.  She reverted back to swapping hands.  Then she tried a few more taps, before going back to hand swaps.  She tapped the block a few more times.  The trainer was still keeping track so each of these taps was counting towards the count of 30 she was looking for.

In the twenties the learner began to be creative.  She tried different ways to move hand over hand.  She’d go back and forth between experimenting with hand swaps and tapping the block.  Finally she reached a count of 30 at which point her handler clicked and reinforced her.  Jesús’ point is now all the extra gunk was also chained in. If the handler were to keep reinforcing the tapping of the block, she would also see the frequency of the hand swapping skyrocket.  That’s not the desired, goal behavior, but it would increase right along with the tapping.

Now you are probably thinking:  “Well that’s just poor training.  No one is going to jump from a fixed ratio of 5 to one of 30.”  My response would be to say that this can happen inadvertently. Suppose a handler has had a behavior on a high rate of reinforcement.  He’s asking the horse to carry himself in a correct bend.  He’s cueing it through gentle requests down the rein.  The horse is responding on a consistent basis, but then he’s distracted or the footing changes so he loses his balance. Whatever the reason, the handler isn’t getting the same consistent response. Instead he’s getting a string of unwanted behavior.  Sometimes the horse almost meets criterion, but not enough to click.  And then he comes through with the right answer.  The handler captures that moment with a click and a treat.  The question is: what is the long term result of that click?  Has the handler just identified a single clickable moment or has he chained in a long string of unwanted behavior?

The horse’s future responses will answer that particular question, but Jesús response in general is: if you want clean behavior, you need to learn to microshape.

Jesús made the further comment that this type of inadvertent chaining happens all too often when people are working with autistic children.  The sad thing here is the previous behavior that is being reinforced is not something harmless like hand swapping, but it’s often self-injurious behavior like head banging.

To sort out the tangle you need to analyze the whole behavior chain rather than focus in on individual behaviors.  These adjunctive behaviors can create a lot of stress. Again Jesús emphasized that’s why it is so important to understand extinction.  You need to understand it so you can master it.

Coming Soon: Part 11: Mastering Extinction

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 9 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction

If you have not yet read the previous articles, I suggest you begin with Part 1.
Part 9: Cues and Extinction

Cue Communication
Cues have a lot to do with reducing the emotional effect of extinction.  Cues can tell an animal whether or not you’re you’re engaged with him in training.  If your cues say “not now”, he knows he can go take a nap.  Kay Laurence has very clear protocols for training.  If someone who is working a dog has a question, the handler first parks the dog.  That is, the handler stands on the lead.  The dog quickly learns this means that he doesn’t need to watch his handler closely.  His handler won’t be asking anything of him as long as her foot is on the lead.  He can take a break from the training conversation.

parking signWith our horses we often forget to put this piece in.  Generally we train by ourselves. The time in the barn is horse time so our focus is on them.  It’s only when someone comes to visit that we discover the “grown-ups” really can’t talk.  Our horse wants to be part of the conversation as well!  If you abruptly ignore your horse, that’s when you can get macro extinctions with all of the associated problems.  The solution is to teach the equine version of “park”.

If you are working in a halter and lead, an easy signal for this is simply to toss the lead over your horse’s neck.  That can very quickly morph into a “park” cue.  Just remember when you walk off, to “unpark” him by taking the lead off his neck.

The larger solution is to become more aware of your body language and the attention your animal is giving to it.  If you see him surfing for answers, interrupt the process. Reset into another teaching opportunity so your animal has a clearer idea of what is wanted and can avoid the frustration of macro extinctions.

Coming soon:  PORTL

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 8: Training with High Rates of Reinforcement

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 8 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement

If you have not yet read the previous articles, I suggest you begin with Part 1.
Part 8: Training With High Rates Of Reinforcement

Raising Criteria
In a successful shaping session with horses it can seem as though they never stop eating.  That doesn’t mean that the criteria are never raised.  Quite the contrary.

In a good shaping session the next criterion you’re going to shift to is already occurring a high percentage of the time BEFORE you make that the new standard.

Nikita grown ups editedSuppose I’m working on having my horse stand politely next to me in the behavior I call: “grown-ups are talking please don’t interrupt”.  My horse is keeping his head consistently positioned so he is looking straight ahead.  I’ve decided that now I also want him to have his ears forward.  That’s a great goal, but if I abruptly stop clicking for good head position because the ears are back, guess what I’ll get – more pinned ears.  Why?  Because I’m frustrating my horse, and that emotion is expressed through pinned ears.

If I frustrate him too much, I’ll also get him swinging his head, nudging my arm, pawing etc. – all the behaviors that I thought I had extinguished as I was building my polite “grown-ups” behavior.

Using Resurgence
What is the solution?  I could begin by separating out ears from other criterion.  Every time I see this horse with his ears forward, click, I’ll reinforce him.  If I’m walking past his stall and he puts his ears forward as I go by, click, he’ll get a treat. Pretty soon I’ll see that my presence is triggering ears forward.  I’ve made it a “hot” behavior.

So now if I withhold the click in grown-ups, I’m likely to get a resurgence of the “hot” behaviors. I’m still using extinction, but I’ve set my horse up for success.  The behavior that is going to pop out is the one I’ve made “hot”, namely ears forward.

As Jesús kept saying: you have to understand extinction in order to master it.

Shaping with What is Already There
In fact, I probably won’t even shift my focus to ears forward until they already occurring at least some of the time.

I’m going to want my horse to stand in grown ups with his ears forward. That’s the goal, but at first, I’ll be happy if he simply takes his nose away from my treat pockets.  As I click him for keeping his head directly between his shoulders, some variability is going to come into the overall behavior.  Sometimes he’ll have his head slightly higher, or lower, his ears forward or back.  Initially I may be so busy monitoring the orientation of his head, I won’t even notice what he is doing with his ears.  But as his head position stabilizes and becomes more consistent, I’ll be able to take in more of the variations.

I’ll become increasingly aware of what he is doing with his ears.  If they are almost always pinned, there’s no point in making that the next criterion.  I’ll be surfing a long extinction wave before ears forward pops out.  In fact for something like ears, the more frustrated he becomes, the less likely they are to go forward.

Instead I’ll wait until his ears are popping forward frequently before I make that the clickable moment.  I’ll be withholding the click for a second or two while I wait for his ears to move.  My horse won’t be perceiving the event as anything negative.  The click will shift seamlessly to the new criterion.  That slight moment of extinction as I withhold the click causes my horse to surf through current “hot” behaviors. I’m using resurgence, but in a way that sets the horse up to have success build on success.

Coming soon: Cues and Extinction

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 7: Emotions

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 7 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions

If you have not yet read the previous articles, I suggest you begin with Part 1.

Part 7: Emotions

Shaping With Micro Versus Macro Extinction
When someone is shaping and they want to raise the criterion, they stop reinforcing for a behavior that was just successful.  The learner goes through a resurgence/regression process.  She begins to offer other behaviors that have worked in the past.

People tend to think of extinction as happening over a long period of time, but Jesús kept emphasizing that it happens over seconds.  Two to three seconds is all you need for a mini extinction. You’ll begin to see the learner offering behavior other than the one that was previously reinforced.

Again this got my attention.  I don’t like the frustration you see when a puzzle appears to be unsolvable.  Shaping shouldn’t be marked by sharp drop offs in reinforcement. I don’t want to see macro extinctions.  If reinforcement is that sticky, it’s time to take a break.  Either put the horse away altogether while you go have a think, or regroup by shifting to another activity.  If you keep waiting, waiting, waiting until your learner finally gets close to the answer, you could lock in some unwanted behavior, and you will almost certainly lock in some unwanted emotions.

Emotions
Jesús pointed out that we use words such as “the animal was being emotional.”  But really what does that mean?   Jesús’ comment was: “we are always emotional.  It isn’t that the extinction process produces emotions.  All processes produce emotions.”

That’s such a good reminder.  We tend to think about emotions when they are the size of a five alarm fire, but really we are always “being emotional”.  There are emotions associated with all behaviors.  Ideally in training we’d like to avoid the five-alarm-fire type. That’s why it is so important to understand these processes.  The sooner you recognize that you are in an extinction process, the sooner you can do something to get out of it.

rat lever press cartoonIn extinction the individual (rat, human, horse, etc.) follows a predictable emotional pattern.

First, you see response bursting.  A rat has been reinforced consistently for pressing a lever.  Abruptly the lever pressing no longer works.  The rat will press the lever with even more energy trying to get it to work.  This has been equated with the classic hitting the button over and over again on the vending machine when your coke doesn’t fall out.

In the next stage you get angry.  Now you’re kicking the coke machine.

Next you see regression.  What behaviors have you seen modeled? What is your past history when things like this fail?

Then there is a pause followed by another period of response bursting.  Gradually the cycles become less until the individual settles into a calmer stage of acceptance.

Some psychologists have equated this pattern with the stages people go through when they are grieving.  When you lose a loved one, a job, a home, you are thrown into an extinction process.  Your loved one is gone.  No change in your behavior is going to bring back the one you’re grieving for.  Your reinforcers are gone and your behavior is ineffective.  Nothing you do will change the reality of your loss.

The stages of grief begin with denial, followed by anger, then depression, bargaining, and finally acceptance and a return to a meaningful life.

It’s interesting to see the comparison people make between these two processes. Understanding does bring with it coping skills.  If you understand the process you are in, you can keep things in perspective and find your way out of emotional tangles faster.  You can also be more understanding towards others (horse or human) if they are caught up in an extinction process.

Coming Soon: Part 8: Training With High Rates Of Reinforcement

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt.6: Accidental Extinction

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 6 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction

If you have not yet read the previous articles, I suggest you begin with Part 1.

Part 6: Accidental Extinction

Accidental Extinction
Natilie grown ups close up Extinction is not a rarity.  Extinction is going on all the time, but we aren’t always aware of it. Suppose you’re working with your horse. Perhaps you’re in the early stages of clicker training and the focus of your lesson is teaching your horse to keep his head in his own space, away from your treat pouch, a lesson I refer to as: “the grown-ups are talking, please don’t interrupt”.  What this implies is that you can stand next to your horse with your pockets full of treats while you carry on a conversation with someone, and your very polite clicker horse will be able to wait politely beside you.

To teach this behavior you’ve been asking your horse to stand quietly with his nose centered between his shoulders.  He’s been doing well.  You’re almost done with the session when your cell phone rings.   You answer it, taking your attention away from your horse.

Your horse doesn’t understand why you’ve abruptly disconnected from him.  You haven’t gone through a teaching process to explain to him that the ring tone of your cell phone is a cue for him to take a break.  Your horse doesn’t know this, and he doesn’t understand why the flow of your session has so abruptly changed.  He offers you a nice bit of grown-ups that meets all the previous criteria, but you aren’t paying any attention.  He doesn’t get clicked.  He tries harder, maybe throwing in some head lowering.  That doesn’t work either so he tries some earlier experiments – some head bobbing, some lip flapping, some gentle nudging, and finally a hard nudge followed by a nip.  That gets your attention, but now you’re thinking what an impatient, muggy horse you have!

Desperation Clicks
Your horse is offering “rude” behavior, bumping, nudging your arm, snuffling around your pockets.  He’s scrolling through the behaviors that he’s tried in the past.  You click something, anything, out of desperation.  What you are reinforcing is not just that single moment, but all the scrolling he’s been doing through his repertoire trying to get you to click. You have just chained in all those other unwanted behaviors.  chain with lockAnd just as a real chain locks things into place, so too, does this chain. You’ve just locked in those behaviors making it very likely that you’ll be seeing them again, especially under similar circumstances.  It’s going to be very hard to break that chain and discard those unwanted segments.  They’ve become an instant part of the whole sequence.

Jesús showed an interesting video from Michelle Pouliot.  Michelle is an excellent trainer.  She’s a guide dog trainer with thirty plus years of experience.  It’s been through her efforts that clicker training has been introduced into the Guide Dogs for the Blind training program.  Michelle also does freestyle with her dogs and has won many national titles, so to say that she’s a good trainer is an understatement.  I find it comforting that even trainers at this level of expertise can have an “oh oops!” happen in training.

Michelle was teaching a labrador to retrieve a dumbbell.  The dog had been successfully delivering the dumbbell to her, but now she wanted to raise the criterion and have the dog place it more firmly in her hands.  When the dog did not get reinforced for the usual behavior, it dropped the dumbbell, did a quick head bob, and then picked the dumbbell up again.  Just as Michelle clicked, the dog sat.  Oh oops!

Michelle lowered her criteria.  The dog handed her the dumbbell, but now he was also sitting as he did so.  Michelle’s hand reaching to take the dumbbell had in one click become a cue to sit.

Mini versus Maxi Extinctions
When the dog started offering behavior to get Michelle to click, that’s extinction.  We don’t tend to think of it in this way.  To develop the behavior we are training we want the offering of behavior.  Shaping depends upon differential reinforcement.  The dog offers a head bob, a paw lift, a sit.  In shaping we pick and choose from among these behaviors.  We think of extinction as something to be avoided. It’s a long drawn out process with lots of painful emotions associated with it.

What Jesús wanted us to understand was that the extinction process can occur in seconds. When you are shaping, you are working with mini extinctions.  When a learner is offering behavior, they are going through a resurgence process.  You don’t have to go hours or even minutes for the extinction process to begin.  It happens in seconds.

My ears perked up at this point in the talk.  I love this concept of mini extinctions.  It fits with microshaping and my own term – shaping on a point of contact.

Shaping on a point of contact involves pressure and release of pressure – but with this distinction.  The pressure remains at a level where it is information only.  It never escalates to become uncomfortable or fear-inducing.  It serves as a clue that helps the learner get to his click and treat faster.

Mini extinctions, micro shaping, and shaping on a point of contact are all learner-friendly because they make use of thin slicing and create high rates of reinforcement.

Microshaping
During the Five Go To Sea conference cruise, Kay Laurence gave a talk on microshaping.  She stressed that it’s not thin slicing alone that defines microshaping. It is high rates of reinforcement.  In microshaping she wants a success rate of 98% or higher. To get that you have to be very skilled at setting up the training environment. The learner is not surfing through a long series of behaviors trying to find the one that is “hot”. The design of the lesson provides very few opportunities for unwanted behavior to creep in. Instead it’s easy for the learner to offer correct responses.

Kay contrasts microshaping with what she refers to as sloppy or dirty shaping.  Here the handler lets the animal offer behavior after behavior looking for the one that will satisfy the criterion.  I’ve always been uncomfortable watching people freeshape in this clumsy fashion. They miss so many opportunities to click because they are looking for too much.  Now Jesús has helped me understand why this type of shaping makes me so uncomfortable.  Mini extinctions are part of puzzle solving.  But they are mini.  Success happens frequently so the frustration level stays low.  You could in fact see it as a positive motivator.  That little bit of: “is it this or is it that?” leads to a feeling of satisfaction each time you make the right choice.

Contrast that with macro Extinction.  Now it’s not “this, that, or maybe this other solution”. Nothing you try seems to work.  The frustration level rises to a level that takes away the fun.

Shaping Confusion

If you were the learner in a shaping game, what would you do with this tea cup?

If you were the learner in a shaping game, what would you do with this tea cup?

When you play shaping games with people and the “trainer” doesn’t have a clear plan, you’ll see this kind of frustration emerge.  Suppose the handler sets a teacup on the table.  Her goal is to get her learner to turn the tea cup upside down.  That’s a hard behavior to get because it’s not something we normally do with tea cups.  The learner gets clicked a couple of times for touching the teacup.  The teacup is clearly the “hot” item, but what is she supposed to do with it?  The learner tries turning the teacup, picking it up, passing it from hand to hand.  Nothing works.

She pretends to drink out of it, she spins it, she scoots it over the table.  Nothing gets clicked. Her frustration rises along with her unwillingness to play the game.  She’s in a macro extinction that can be painful to watch.  She goes back through the history she has with teacups.  What else can she try?  She feels like smashing the teacup over her trainer’s head, but social conventions keep her from offering that behavior!

When animals begin to scroll through behaviors they’ve learned in an attempt to get clicked, you are seeing an extinction process.  You are seeing a resurgence of previously reinforced behaviors.  In the teacup example, when the learner was no longer reinforced for just touching the teacup, when reinforcement for that behavior stopped, she tried things that she had done with tea cups or tea cuplike objects in the past.  Her handler was a new shaper.  She was outcome oriented, so she was looking for big macro responses.  She didn’t yet see the small steps that would take her easily to her goal behavior.  She didn’t yet know how to set aside the larger end goal while she taught her learner the small reaction patterns that would lead seamlessly to the desired result.

Teaching Via Reaction Patterns
In her talk on Microshaping Kay showed us some lovely examples of what it means to train for reaction patterns instead of end goals.

One of the clearest is the way Kay teaches a dog to go to a mat.  The end behavior she’s looking for has the dog going with energy to the mat, so she trains what she wants right from the start.  Instead of shaping general approximations where the dog sniffs at a mat, wanders around it, maybe sits or lies down, leaves and then comes back again, and every now and then puts a paw on it for a click and a treat, she starts out teaching an underlying base behavior.  In this case the base behavior is trotting back towards the handler after the dog has retrieved a tidbit of tossed food.

The dog retrieves it’s treat and then reorients back to the handler who is sitting in a chair.  As the dog trots towards her, the handler clicks and then tosses out another treat.  The dog turns and chases down this goody.  As soon as he has it, he reorients back towards the handler with the eager expectation that she’ll toss another treat.  As he comes back towards her, trotting with enthusiasm, she clicks again and tosses out another treat.

Shaping a dog to trot to a mat.

Shaping a dog to trot to a mat.

When this behavior is solid, she puts a mat down in the path of the dog.  As he comes trotting towards her, she clicks as his feet land on the mat.  It’s a clever way to get the behavior you want from the very first instant.

The dog experiences success after success.  He becomes a confident learner.  Instead of developing the mat behavior through trial and error learning where superstitious patterns may get linked in, he has learned it as a clean behavior.  Later if something stresses the behavior, what he will regress back to is this clean version of mat targeting, and it’s associated eager learning state.

Here’s a video of this shaping process:

Coming soon: Part 7: Emotions

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt. 5: Extinction Reveals The Past

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 5 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past

Part 5: Extinction Reveals The Past

The Extinction Process
In the previous section I  said that extinction produces resurgence and regression.  I went on to talk about extinction without defining it.  In general we understand the meaning of that term, at least how we would use it in everyday language.

Here’s the definition Jesús gave us:

“When reinforcement is no longer forthcoming, when a response becomes less and less frequent, you get operant extinction.”

How does this play out?  What do you see in your animals?

In the controlled environments of a lab experiment, here is what you might see:  A rat is being reinforced consistently for pressing a lever.  When that behavior is well established, the experimenter no longer reinforces lever pressing.  When the behavior fails to pay off, the rat shows a sudden flurry of lever pressing behavior.  When this fails, the rat exhibits more aggressive types of behaviors.  In humans we would equate this to the behavior you see when vending machines fail.  You start out jiggling the knobs and progress towards pounding on the machine.

rat 2The aggressive behavior is followed by a period of the rat giving up.  He ignores the lever.  Then the rat tries again with a flurry of activity, trying to see if the original, reinforced behavior is once again working. The whole cycle repeats itself, but the bursts get smaller and smaller, and the pauses in between become longer.

Throughout all of this process the rat is clearly experiencing emotions we would not want to see in our horses.  When lever pressing fails to work, the rats become aggressive.  In our horses we see displacement aggression.  The horse is frustrated.  A behavior which was reliable is no longer working.  If other horses are nearby, you may see the horse pin his ears and snake his neck out to warn the others away.  Or he may grab at his lead rope, or nip at the handler’s sleeve.

Remember – you are seeing behavior that has been modeled for this horse.  You are seeing his training history.  And perhaps you are also seeing his herd background.  If he’s lived in crowded/confined conditions that promote more horse to horse aggression, it’s possible that’s what you’ll see acted out.

It would be interesting to look at two groups of horses – one containing horses that grew up in stable herds living in large, open spaces.  The other would have horses that were raised in much more confined spaces where competition for resources created more horse to horse aggressive interactions.  What difference, if any, would you see when these horses are exposed to a mild extinction process?  What behaviors would regression reveal?  What does your knowledge of your own horse’s background predict?

For horses with known backgrounds it would be interesting to collect data on their behavior when they are faced with a mild extinction process.  If you know the conditions under which your horse was raised, what type of behaviors would you expect to see during an extinction process?  Would he be the one “banging on the coke machine”, or would he cope well with the change in reinforcement rates?

Coming Soon: Part 6: Accidental Extinction

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 4: Extinction and Shaping

Resurgence and Regression: Understanding Extinction So You Can Master It

Five go to sea bannerFrom a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 4 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping

Part 4: Extinction and Shaping

Extinction
Often clicker trainers say they never use extinction.  I certainly work hard to set up my training so the horses aren’t put into the kind of guessing game that can lead to outbursts of frustration and aggression.  That’s something I very much want to avoid. But that doesn’t mean I don’t use extinction.  That’s what Jesús’ talk made so very clear.

To the people who say they never use extinction, his response is: “What do you mean you never use extinction!  Extinction is at the heart of shaping.  Shaping is differential reinforcement.  It’s the interplay between positive reinforcement and extinction.  So if someone says they aren’t using extinction, probably they don’t understand what they are saying.”

That’s such a wonderfully blunt and typically Jesús comment.  He went on to explain what he meant.  As he said: “If you don’t understand extinction, you won’t be able to master it.”

Regression and Resurgence
Jesús makes a distinction between regression and resurgence.

In regression you revert back to previously extinguished behaviors.

In resurgence you revert back to previously reinforced behavior.

This isn’t just semantics.  Regression and resurgence emerge out of different training strategies and produce different outcomes.

Regression is a term that is used in psychoanalysis and can be defined as: “If the present behavior is not capable of getting reinforcement, one reverts to older forms of response which were once effective.”  In other words, when a behavior that has been generating reinforcement is no longer working, the individual will revert back to behaviors that have worked in the past. The order in which this unfolds is significant.

Under stress you will revert back to an older way of behaving.  If that behavior is not reinforced, you’ll go through another extinction process.  You’ll revert back to even older behaviors.  You’ll keep trying things and trying things, until you either give up entirely, or you are pushed to creativity.  This can be a stressful process which is why some people think of creativity as an unpleasant experience.

Extinction History
Regression emerges because a behavior which normally earns reinforcement is no longer working.  Often we think of extinction as simply a procedure that’s intended to reduce behavior.  You don’t like a dog’s barking, so you never reinforce it in the hope that the behavior will go away.  This simplistic view misses an important key to understanding how to use extinction. The behaviors that emerge in an extinction process are not random. Understanding the order lets you master the process.

That’s one of the many gems from Jesús’ presentation.  Here are some more:

Jesús described extinction as the mirror image of reinforcement.

Extinction tells you what was reinforced in the past.

Reinforcement tells you what behaviors you are building for the future.

I wrote about this in Part 2 of this series: “The Translation to Horses.”  When you are first learning about clicker training, if your handling confuses the horse and puts him into an extinction process, the behaviors he throws at you tells you more about his past than his present.  Don’t blame yourself for the outburst.  Your current training choices didn’t create the behavior you’re now dodging.  Turn your spotlight instead on his past.  That’s where the behavior was learned.

You may be the catalyst, but you are not the cause.  That’s good news.  You don’t have to take his behavior personally. The cause sits not in the present, but in the past.  It’s only natural to become worried by the emotional reaction you’re seeing.  People sometimes inadvertently end up compounding the problem. If their handling skills are clumsy or they don’t yet know how to manage the environment, they can put the horse into even more of an extinction process.

I’ve seen this in beginner handlers.  They don’t yet understand how much a lack of clear criteria can impact a learner.  The horse has offered three or four clickable moments, but the handler has missed them all.

Those missed clicks can put the horse into an extinction process that leads to emotional outbursts.  The handler becomes rattled by this unwanted behavior.  She becomes even more uncertain and inconsistent which leads to more frustration in her horse.  What is he supposed to do?  His growing anxiety leads to displacement behaviors and the emergence of older, unwanted behavior.

That’s where video cameras can be so useful.  Video helps the handler to see the training from the horse’s point of view.  It reveals the good tries the horse is offering and helps the handler understand more clearly what she wants to be reinforcing.  And it aids in learning better handling skills that lead to clean, consistent teaching.

The solution to extinction bursts lies in embracing clicker training, not from running from it.  Through clicker training you’ll be building a repertoire of behaviors that give the horse alternatives to his old patterns.

Regression and resurgence reveal the past.

Reinforcement builds your future.

Coming soon: Part 5: Extinction Reveals the Past

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 3: Unraveling the Regression Mess

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise. toppic1

This is Part 3 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess

Part 3: Unraveling the Regression Mess

Emitted Versus Permitted Behavior
What are the keys to unraveling the regression mess?

The first is to tighten up your training and learn how to set up the environment so the behavior you want is the behavior that is most likely to occur. Jesús made the distinction between emitted and permitted behaviors.

When behavior is emitted, you are waiting to see what the learner offers.  When behavior is permitted, you set up the environment so the behavior you want is the behavior that is most likely to occur.

If you’re waiting, waiting, waiting for the dog to sit or the horse to step on a mat, you may see lots of experimenting before you get something you want to click.  All that experimenting can end up as part of a chain.  And it could also lead to a regression into previously learned, but unwanted behavior.

With the horses we begin with very simple, easily isolated behaviors such as targeting.

With the horses we begin with very simple, easily isolated behaviors such as targeting.

With the horses we begin with very simple, easily isolated behaviors such as targeting and backing.  We set up the environment so the behavior is likely to occur.  You aren’t surfing an extinction wave of behaviors.  Your horse doesn’t have to do a lot of guessing.  The right answer is obvious and easy.

In those first lessons I have people start out with only twenty treats.  That limits how much training you can do.  Before your horse can get too confused or frustrated, you’re stepping away to get another round of treats.

You’re also using that time while you refill your pouch to assess what just occurred.  That first targeting session is just data collecting. You’re finding out if that’s a good starting point, or perhaps you need to find a different lesson.  A horse that is very shut down, or becomes easily stressed when he’s not told exactly what to do, may need you to start with an even simpler step than targeting.  This is a horse that may need to have the clicker carefully charged first by simply feeding one treat after another.  Once he’s showing interest in the food, you’ll add the clicker in.  Now it’s: click then feed, click then feed.  At this point the click is not yet contingent on a specific behavior. You are simply pairing the click with the food.

Once you think your horse is noticing the click and anticipating the food, you’ll begin to turn the click into a functional marker signal.   You’ll begin to pair it with the behavior.  You’ll pick something easy such as targeting, or perhaps a slight moving of his head away from your treat pouch.  It should be something you know you can get so the transition from charging the clicker to using it is a seamless one.

Designing an appropriate lesson plan is just part of the solution.  You also need to have clean handling skills and good timing.  Clicking late, clicking the wrong thing, clicking because you haven’t clicked for a while – all of these things will confuse your learner and lock in more unwanted behavior.  So work on your handling skills. Practice first, preferably in front of a mirror.  Borrow a friend to be your “horse”.  Use your video camera so you can review what you are doing. When your handling is quiet, clean, organized, and second nature, that’s what your training will become – quiet, clean, organized, and second nature.

Broadening the Repertoire
Good handling is part of the solution.  Another is to develop a broad repertoire of behaviors.  The more skills you teach your horse, the more options he’ll have besides aggression. Instead of regressing into aggressive responses, he’ll have other options that work.  This is where trust the process begins to make sense. We’ve all read the stories.  Someone has been struggling with a horse, not seeing much progress, and then the pieces all fall into place.  Instead of snapping at his handler, he’s backing up politely and dropping his head.  Instead of pulling away, he’s offering beautiful lateral flexions.  The older repertoire is still there. Given the right triggers, you might still see him regressing back into “childhood”, just as that professor regressed back when she was trapped in an elevator.  But you’ve given him more tools.  That broader repertoire gives him more options. He’ll regress back to head lowering not aggression.

There were many more gems in Jesús’ talk, but this was a good one.  I’ll stop here for now and let you enjoy it.

Coming Soon: Part 4: Extinction and Shaping

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com