Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 6 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction

If you have not yet read the previous articles, I suggest you begin with Part 1.

Part 6: Accidental Extinction

Accidental Extinction
Extinction is not a rarity.  Extinction is going on all the time, but we aren’t always aware of it. Suppose you’re working with your horse. Perhaps you’re in the early stages of clicker training and the focus of your lesson is teaching your horse to keep his head in his own space, away from your treat pouch, a lesson I refer to as: “the grown-ups are talking, please don’t interrupt”.  What this implies is that you can stand next to your horse with your pockets full of treats while you carry on a conversation with someone, and your very polite clicker horse will be able to wait politely beside you.

To teach this behavior you’ve been asking your horse to stand quietly with his nose centered between his shoulders.  He’s been doing well.  You’re almost done with the session when your cell phone rings.   You answer it, taking your attention away from your horse.

Your horse doesn’t understand why you’ve abruptly disconnected from him.  You haven’t gone through a teaching process to explain to him that the ring tone of your cell phone is a cue for him to take a break.  Your horse doesn’t know this, and he doesn’t understand why the flow of your session has so abruptly changed.  He offers you a nice bit of grown-ups that meets all the previous criteria, but you aren’t paying any attention.  He doesn’t get clicked.  He tries harder, maybe throwing in some head lowering.  That doesn’t work either so he tries some earlier experiments – some head bobbing, some lip flapping, some gentle nudging, and finally a hard nudge followed by a nip.  That gets your attention, but now you’re thinking what an impatient, muggy horse you have!

Desperation Clicks
Your horse is offering “rude” behavior, bumping, nudging your arm, snuffling around your pockets.  He’s scrolling through the behaviors that he’s tried in the past.  You click something, anything, out of desperation.  What you are reinforcing is not just that single moment, but all the scrolling he’s been doing through his repertoire trying to get you to click. You have just chained in all those other unwanted behaviors.  And just as a real chain locks things into place, so too, does this chain. You’ve just locked in those behaviors making it very likely that you’ll be seeing them again, especially under similar circumstances.  It’s going to be very hard to break that chain and discard those unwanted segments.  They’ve become an instant part of the whole sequence.

Jesús showed an interesting video from Michelle Pouliot.  Michelle is an excellent trainer.  She’s a guide dog trainer with thirty plus years of experience.  It’s been through her efforts that clicker training has been introduced into the Guide Dogs for the Blind training program.  Michelle also does freestyle with her dogs and has won many national titles, so to say that she’s a good trainer is an understatement.  I find it comforting that even trainers at this level of expertise can have an “oh oops!” happen in training.

Michelle was teaching a labrador to retrieve a dumbbell.  The dog had been successfully delivering the dumbbell to her, but now she wanted to raise the criterion and have the dog place it more firmly in her hands.  When the dog did not get reinforced for the usual behavior, it dropped the dumbbell, did a quick head bob, and then picked the dumbbell up again.  Just as Michelle clicked, the dog sat.  Oh oops!

Michelle lowered her criteria.  The dog handed her the dumbbell, but now he was also sitting as he did so.  Michelle’s hand reaching to take the dumbbell had in one click become a cue to sit.

Mini versus Maxi Extinctions
When the dog started offering behavior to get Michelle to click, that’s extinction.  We don’t tend to think of it in this way.  To develop the behavior we are training we want the offering of behavior.  Shaping depends upon differential reinforcement.  The dog offers a head bob, a paw lift, a sit.  In shaping we pick and choose from among these behaviors.  We think of extinction as something to be avoided. It’s a long drawn out process with lots of painful emotions associated with it.

What Jesús wanted us to understand was that the extinction process can occur in seconds. When you are shaping, you are working with mini extinctions.  When a learner is offering behavior, they are going through a resurgence process.  You don’t have to go hours or even minutes for the extinction process to begin.  It happens in seconds.

My ears perked up at this point in the talk.  I love this concept of mini extinctions.  It fits with microshaping and my own term – shaping on a point of contact.

Shaping on a point of contact involves pressure and release of pressure – but with this distinction.  The pressure remains at a level where it is information only.  It never escalates to become uncomfortable or fear-inducing.  It serves as a clue that helps the learner get to his click and treat faster.

Mini extinctions, micro shaping, and shaping on a point of contact are all learner-friendly because they make use of thin slicing and create high rates of reinforcement.

Microshaping
During the Five Go To Sea conference cruise, Kay Laurence gave a talk on microshaping.  She stressed that it’s not thin slicing alone that defines microshaping. It is high rates of reinforcement.  In microshaping she wants a success rate of 98% or higher. To get that you have to be very skilled at setting up the training environment. The learner is not surfing through a long series of behaviors trying to find the one that is “hot”. The design of the lesson provides very few opportunities for unwanted behavior to creep in. Instead it’s easy for the learner to offer correct responses.

Kay contrasts microshaping with what she refers to as sloppy or dirty shaping.  Here the handler lets the animal offer behavior after behavior looking for the one that will satisfy the criterion.  I’ve always been uncomfortable watching people freeshape in this clumsy fashion. They miss so many opportunities to click because they are looking for too much.  Now Jesús has helped me understand why this type of shaping makes me so uncomfortable.  Mini extinctions are part of puzzle solving.  But they are mini.  Success happens frequently so the frustration level stays low.  You could in fact see it as a positive motivator.  That little bit of: “is it this or is it that?” leads to a feeling of satisfaction each time you make the right choice.

Contrast that with macro Extinction.  Now it’s not “this, that, or maybe this other solution”. Nothing you try seems to work.  The frustration level rises to a level that takes away the fun.

Shaping Confusion

When you play shaping games with people and the “trainer” doesn’t have a clear plan, you’ll see this kind of frustration emerge.  Suppose the handler sets a teacup on the table.  Her goal is to get her learner to turn the tea cup upside down.  That’s a hard behavior to get because it’s not something we normally do with tea cups.  The learner gets clicked a couple of times for touching the teacup.  The teacup is clearly the “hot” item, but what is she supposed to do with it?  The learner tries turning the teacup, picking it up, passing it from hand to hand.  Nothing works.

She pretends to drink out of it, she spins it, she scoots it over the table.  Nothing gets clicked. Her frustration rises along with her unwillingness to play the game.  She’s in a macro extinction that can be painful to watch.  She goes back through the history she has with teacups.  What else can she try?  She feels like smashing the teacup over her trainer’s head, but social conventions keep her from offering that behavior!

When animals begin to scroll through behaviors they’ve learned in an attempt to get clicked, you are seeing an extinction process.  You are seeing a resurgence of previously reinforced behaviors.  In the teacup example, when the learner was no longer reinforced for just touching the teacup, when reinforcement for that behavior stopped, she tried things that she had done with tea cups or tea cuplike objects in the past.  Her handler was a new shaper.  She was outcome oriented, so she was looking for big macro responses.  She didn’t yet see the small steps that would take her easily to her goal behavior.  She didn’t yet know how to set aside the larger end goal while she taught her learner the small reaction patterns that would lead seamlessly to the desired result.

Teaching Via Reaction Patterns
In her talk on Microshaping Kay showed us some lovely examples of what it means to train for reaction patterns instead of end goals.

One of the clearest is the way Kay teaches a dog to go to a mat.  The end behavior she’s looking for has the dog going with energy to the mat, so she trains what she wants right from the start.  Instead of shaping general approximations where the dog sniffs at a mat, wanders around it, maybe sits or lies down, leaves and then comes back again, and every now and then puts a paw on it for a click and a treat, she starts out teaching an underlying base behavior.  In this case the base behavior is trotting back towards the handler after the dog has retrieved a tidbit of tossed food.

The dog retrieves it’s treat and then reorients back to the handler who is sitting in a chair.  As the dog trots towards her, the handler clicks and then tosses out another treat.  The dog turns and chases down this goody.  As soon as he has it, he reorients back towards the handler with the eager expectation that she’ll toss another treat.  As he comes back towards her, trotting with enthusiasm, she clicks again and tosses out another treat.

When this behavior is solid, she puts a mat down in the path of the dog.  As he comes trotting towards her, she clicks as his feet land on the mat.  It’s a clever way to get the behavior you want from the very first instant.

The dog experiences success after success.  He becomes a confident learner.  Instead of developing the mat behavior through trial and error learning where superstitious patterns may get linked in, he has learned it as a clean behavior.  Later if something stresses the behavior, what he will regress back to is this clean version of mat targeting, and it’s associated eager learning state.

Here’s a video of this shaping process:

Coming soon: Part 7: Emotions

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com