JOY FULL Horses: Understanding Extinction: Part 14

Our Creative Horses
Yesterday I shared with you the story of Robin’s “pose”.  The use of resurgence has helped us develop a much more systematic way of creating unlikely behaviors.  Because we understand the process better, we can be more deliberate in it’s use.  I ended the post by saying: “The end result may look like magic, but there is good science behind it.”

When we open up our training in this way and turn our learners into more active participants, we often find that they are even more creative than we are.  Once again Robin provided me with a great example of this.

When Robin was three I took him to the Equine Affaire to be my demo horse.  I wanted to show people what freeshaping via clicker training looks like.  I didn’t want them just to see the end product of freeshaping.  I wanted them to see me teach Robin a completely novel behavior.  The problem was he already had a pretty extensive repertoire. I was stumped for ideas, but I thought the easiest solution would be to use a prop.  One of my clients had been teaching his horse to flip a hula hoop up over his head.  I thought I could make a start on that with Robin.

Robin had been our first equine retriever.  Picking things up was solidly in repertoire.  I figured if I put the hula hoop on the ground, he would try to pick it up.  I’d be able to reinforce that and build it into Robin holding it longer which might over three days of demos lead to him flipping it over his head.  Such was my level of creativity, that’s all I could think of to work on with a hula hoop.

So during our demo, I brought out the hula hoop and tossed it out on the ground.  I was still explaining freeshaping to the audience so I wasn’t focusing yet on Robin.  While I was talking, he walked over to the hoop and stood with his front feet planted in the middle of it just as he would have stood on a mat.  Before I could respond to him, he reached down, picked up one side of the hoop and began walking himself forward foot by foot with the hoop!  That was his level of creativity!

The Creative Process
Here are the steps the horses have been teaching us:

First, you build a strong history of reinforcement for the component behaviors.

You change the situation somewhat so mild extinction comes into play.

You get a resurgence of these previously reinforced behaviors and new combinations emerge.  That’s creativity.  The most fun for me is seeing what the horses invent. As we have seen, they are often so much more creative than their human partners!

Familiar Landscapes
Kay Laurence might say we were seeing familiar landscapes with fresh eyes.

Dr. Jesús Rosales-Ruiz would say you have to understand the process of extinction before you can master it.  If you understand it, you’ll avoid situations that create macro extinction processes and all the frustration that goes along with them.  Instead you’ll use micro extinctions to build complex behaviors.

I would say that monitoring the level of extinction your learner is experiencing is a keys-to-the-kingdom part of good training.

I’ve just spent a couple of days working with a group of horses I have come to know well. One of them is a retired performance horse. Without going into a lot of details, I would describe him as an emotionally fragile horse. He’s easily worried.  If he thinks he has the right answer, he’s a superstar, but I always have to be careful how far I stretch him into new behaviors.  If he thinks he might get something wrong, he worries.  He’s come out of a training environment in which he had to perform correctly or his rider could get seriously hurt. I suspect he was punished for mistakes which accounts for his worry.

Mastering Micro
His back was looking prematurely aged so I wanted to teach him Robin’s “pilates pose”.  I had already shown him that he could get reinforced for lifting his back up and releasing at the poll.  In this particular session I was holding out for slightly better versions. As I withheld my click, I saw him experimenting.  Was it higher with his poll? Was it more lift of his back? What did I want?

The shifts he was giving me represented micro changes.  They were all within a clickable range.  Clicking him for any of these variations would have been fine, but I was waiting fractionally to see what else would pop out.

I was using micro extinctions to create the next step.  And because I was thinking about this in terms of extinction, I was monitoring closely his emotional responses.  I did not want him to become macro worried.  We were always just a second or two from a click so I could let him experiment without risking the emotional fallout of a larger extinction process.

Micro Masters
Micro is so very much the key.

Macro extinctions are frustrating.  Micro extinctions are part of good teaching.

Macro shaping can be confusing.  Micro shaping is elegant.

Macro negative reinforcement is literally painful. Micro negative reinforcement is clear communication. It is a conversation with cues exchanged in both directions.

When you go micro, your learner is always just a second or two away from a reinforceable moment.  You can cue another behavior, or you can simply click and treat. Either way, you are saying: “Yes! Great idea!” Micro Mastery is what we should be striving for in our training.  When you say someone is a great trainer, you are really saying that individual is a Micro Master.  In training that’s the “black belt” we should be aiming for.

robin-pg-lying-down-micro-masters

With this last section we come to the end of my JOY FULL Horses book – almost.  What remains is one final chapter and that’s what’s coming next.

Coming Next: Doorways

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY FULL Horses: Understanding Extinction: Part 13

Puzzle Solving
In the previous post I gave several examples of how a complex behavior could be taught through the imaginative use of resurgence.  Training is not a random process where you hope your learners will offer you something you can reinforce before the intense emotions associated with extinction shuts them down.

Instead you teach your learners a whole range of behaviors.  You are in effect planting the seeds for what you want to grow.  With this rich repertoire of behaviors to draw on, your learners may come up with some new or unlikely combinations.  We have a procedure for setting up the creative process.  You give your animals the repertoire, the components that form more complex behaviors, and then you set up a puzzle and let extinction be the catalyst for solving it.  Sometimes what pops out of this process is something that wasn’t even on your training radar.

The “Pose”
Robin’s pose is the perfect example of this.  I’ve told the story of how this behavior evolved many times.  I’ll keep it brief here.

Robin first learned a stationary “pose”.  It originally was a by-product of cleaning up his treat taking manners when he was two years old.  He was such an eager learner.  I loved his enthusiasm.  He would grab the treat from my hand and immediately be offering me something even more brilliant than whatever brilliant thing he had just done.  I was enchanted.  The food delivery was a little problematic, but so what.  He was brilliant.

Robin drove home the importance of one of my favorite training mantras.  “If you don’t notice an unwanted behavior when it is just a little thing, don’t worry about it.”  That’s not normally the kind of training advice you expect to get, but here’s where this takes you:  “If you don’t do anything about the behavior when it’s just a little thing, don’t worry. It will get bigger.  Eventually, it will be too big to ignore and then you will do something about it.”

I ignored his all too hasty treat taking for a while, and then I decided I really did need to do something about it.  This was a long time ago. I probably would choose a different method today.  Robin, and all the other horses I’ve worked with have been good teachers.  They have shown me many more options to replace the one I used.  At the time the strategy I chose was to stop the behavior I didn’t want by taking away something Robin wanted.  In other words, I was using negative punishment.

Here was the set up: I put Robin in his stall with a stall guard across the door.  I picked out the biggest carrot from a bag of big carrots and stood across the aisle from him holding it out.  Of course, he reached out for the carrot.  Immediately, I snatched it away and turned my back to him.  I counted to three and offered the carrot again.

Again he reached for it, and again I snatched it away.

I offered it a third time.  This time he hesitated ever so slightly. Click.  I reached into my pocket and handed him a piece of carrot.

A couple of clicks later I could hold the carrot directly under his chin, and he was drawing up away from it.  He wasn’t going to reach for that carrot!  He had learned that drawing back from it generated what he wanted.  If he reached for it, it disappeared, and I handed him nothing.  But if he drew back from it, click, he got exactly what he wanted – a piece of carrot and my laughter.

Robin being Robin made a quick leap from the carrot cueing this behavior to offering it to me without this prompt.  Of course, I clicked him.  He was looking so handsome.  I wasn’t expecting this added bonus, but I was certainly liking it.  As always, when I worked with him, I was enchanted.

He made the leap so fast to offering “the pose” without needing the prompt,  I was blown away by his brilliance.  He started “posing” more often, arching his neck and looking for all the world like a well-trained dressage horse.  I liked the look so much I continued to reinforce it.  It became a default behavior. In the absence of any other active cue from me, if Robin posed, I would click and reinforce him.  In other words I became the cue for the behavior.

It meant that if Robin wanted to interact with me and engage in the clicker game, he had a sure-fire way of doing so.  Even if I was busy doing barn chores, if I saw him posing, I would click, interrupt what I was doing, and reinforce him.

At the time I was keeping my horses at a boarding barn where a good portion of their day was spent in their stalls.  I didn’t want them to become like the proverbial four year old child banging the kitchen pots and pans trying to get mother’s attention.

Robin wasn’t ignored when I was busy with other things.  If he wanted attention, he had a polite way of asking for it.   He never had to become that frustrated “toddler with the kitchen pans” banging on the stall door or raking his teeth up and down the stall walls.

I didn’t create a regression into these unwanted behaviors.  Instead I was able to reinforce a behavior I liked, one that was a useful warm-up for our formal training sessions.  When he asked for attention, Robin was always confident that I would engage with him. He lived secure in a world filled with clicker-training interactions.

Reinforcing him for the pose went on through that winter.  I didn’t have any plans for developing the behavior.  It was simply something I liked. It was Robin who was the creative one.

It must have been late March.  I was lunging him in the arena one evening.  He was giving me a ho hum trot.  There was nothing there I could reinforce.  Robin went once around the circle, twice, three times without reinforcement. Normally I would have been clicking and reinforcing him at a much higher rate, but given the plow-horse trot I was presented with, there was nothing there I wanted to say yes to.

At the time I would not have described it in this way, but I was putting him into an extinction process.  I could see him searching, trying to decide what to do.  On the third time round he had the answer.  He would try his pose.  But in order to pose and still stay in the trot, he had to add energy.

Within one stride he transformed his trot into magazine-cover magnificence.  I captured the moment with a click, and the rest, as the saying goes, is history. That single stride has evolved into horses all over the world moving in magnificent self-carriage.  The “pose” has evolved into a major component of my work.  Robin showed us that we could indeed shape self carriage.  What began as a happy accident has become a deliberate and very systematically-trained behavior.

I don’t introduce the pose to other horses the way I taught Robin.  I needed him to show me what was possible, but today I use other shaping methods.  I originally called it “the pose”, admittedly not a good name.  Since then I’ve expanded the name to the “pilates pose”.  Others have called it crunches, rock backs, “look pretty”, etc.  It really doesn’t matter the name it is given.  What is important is our understanding that horses can become very active partners in maintaining their own good balance and long-term soundness.  It was Robin who first showed us just how possible this is.

Seeing Familiar Landscapes with Fresh Eyes
When I first told this story to Jesús, he commented that the pose came out because of resurgence.  I remember the comment well, though I didn’t understand the significance of it at the time.  Jesús told me later that Robin’s story got him thinking more about resurgence.   It prompted him to use PORTL to look at the different extinction processes. I described the results of those experiments in the earlier sections of this chapter on “Understanding Extinction”.

So here is one further result of Robin’s brilliance.  We now we have a much more systematic way of creating unlikely behaviors.  Because we understand the process better, we can be more deliberate in it’s use. The end result may look like magic, but there is good science behind it.

This is one of my favorite videos.  It was taken a long time ago when Robin was three.  He is lunging around me at liberty.  You’ll see I have two dressage whips in my hands.  I am using them as targets, giving him reference points to balance around.  When this video was made, he had never been in side reins or any other type of mechanical device.  He had not yet had a rider on his back, and yet he is presenting me with this beautiful balance.  I know the lighting is terrible.  The indoor we worked in was poorly lit, but to me it is a shining example of the connections that form when we mix together science, clear communication, and a lot of love.

Coming Next: The Creative Process

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

 

JOY FULL Horses: Understanding Extinction: Part 12

Mastering Micro: Building Unlikely Behaviors with Resurgence
Nothing is either all good or all bad.

We want to use positive reinforcement with our animals because we see it as being both effective and more humane.  But the associations created through positive reinforcement can create addictions to harmful behaviors.  Think about the way advertisers manipulate our behavior to encourage smoking or overeating.

Resurgence and regression can be very negative procedures, but they can also be used to produce what might otherwise be very difficult behaviors to obtain.

If you aren’t sure how you can turn what seems like a negative procedure into a positive teaching strategy, PORTL can once again help to illustrate how this works.

Here’s the set up:

The trainer sets a toy chair on the table for her learner to interact with. The goal is to get the learner to push the chair over the table the way she might push a toy car.

We’ll now observe quietly in the background while the learner begins to interact with the chair.  The trainer could get lucky.  The learner might begin offering the behavior she’s after within the first couple of clicks.  But with this learner there’s no sign of any chair pushing behavior. Why?

History matters.

The learner is going to draw on all of her previous repertoire of things she has done with chairs.  In this case we have a learner who was scolded as a child for pushing her chair over the floor, so she’s not very likely to offer this type of behavior with the toy chair.

A history of punishment has played a role in depressing chair pushing behavior for this learner, but pushing would also have been an unlikely behavior if the trainer had set down a dice. The learner would have tossed the dice or shaken it in her hand because that’s what you do with this kind of object. Pushing a dice over the table like a toy car is not an obvious behavior to try.

Through a series of small approximations, the trainer could try to shaping the behavior she wants.  Her first step would be reinforcing the learner for touching the chair.

The learner in this case is not particularly creative.  She offers simple touches, but nothing else.  Again, the trainer may be dealing with a history of punishment.  Her learner doesn’t have a lot of experience being reinforced for trying things.  In fact, quite the opposite – she may have been punished for stepping “outside the lines”.  She is like so many of our animal learners – hesitant, lacking in confidence, and not showing any outward signs of curiosity.  In her first few attempts she touches the chair, but she doesn’t try any other behaviors.  Getting her to push the chair is going to be hard.

So the trainer takes the chair away and sets out a toy car. Using an object that normally would be pushed makes it very easy to get the desired action.  The learner pushes the car over the table top. Click and treat.

This is repeated several times, and then the trainer takes the car away and sets the chair out.  The learner goes back to touching it.  The chair accidentally falls over – click and treat. The learner latches on to that, expanding her repertoire to two behaviors – touching the chair and knocking it over.

We see this so many times with our animal learners.  One click and suddenly you’ve locked in a behavior you don’t want.  With a creative learner this isn’t a problem.  You can quickly shift the behavior into something you want, but with these “one trick ponies” you have to be so very careful what you click.  In this case the learner persists in knocking the chair over even when she is no longer getting reinforced for the action.

Her trainer makes a quick decision and decides to put everything but pushing the chair like a car on extinction.  Her learner is clearly becoming frustrated.  To avoid having her shut down completely, the trainer takes the chair away and sets the car out again.  The learner immediately starts pushing the car over the table top.  Click and treat.

To help with the generalization the trainer puts a third object out – a small block. The learner pushes the block.  Click and treat.  This is repeated several times, then the trainer takes the block away and sets out the car.  The car is pushed. Click and treat.

The trainer sets the chair out, and the learner pushes the chair.  Job done.

Resurgence and Dog “Yoga”
Using the car in this way is an elegant teaching strategy.  Often when we come up with these clever ways of helping our learner to be successful, we know that it works, but we don’t really have good explanations for why.   Understanding resurgence helps us with the why in this case.  And it helps us to be more deliberate in the use of this kind of teaching strategy.  Here’s another example.

One of Kay Laurence’s students taught her dog to step up with his hind legs onto a chair.  It was elegant training, a beautiful example of setting the learner up for success.  In his talk on extinction, Dr. Jesús Rosales-Ruiz helped us to see that it was also a great example of using resurgence.

Here’s the lesson: First, the dog learned to stand one foot each on four small plastic pods. This alone was impressive training.  The pods were the same ones physiotherapists use to help people improve their balance and proprioception. It took great coordination for the dog to stay balanced on the four pods. But that was only step 1.  Next he learned to keep his front feet on the floor while he maneuvered his hind feet up onto the brick ledge of a fireplace hearth.

Adding in the precision of the pods came next.  Now the dog wasn’t just standing with his front paws on the floor and his hind end up on the ledge.  He was also balancing on all four pods.

This was not done as a cute party trick.  The dog’s owner is a yoga teacher.  Her interest was very much the same as mine – helping her animal learner maintain a healthy spine.  In this orientation she could ask her dog for weight shifts that contribute to a flexible spine.

The last step was setting up a training session next to a chair. The handler withheld the click, putting the dog into an extinction process. With very little experimentation, the dog oriented himself so his hind end was to the chair.  He certainly demonstrated the flexibility of his spine by stepping up onto the chair with his hind legs so he was standing hind end up on the chair and front feet on the floor.

Generalization and Creativity
Jesús commented that if we didn’t know about resurgence we would simply be saying the dog generalized.  That’s not a sufficient explanation.  What we were seeing was a great example of resurgence. PORTL has given us a better understanding of how to encourage this kind of problem solving.  When we want to train for this type of generalization, knowing about the “why” of resurgence helps us to be more deliberate and efficient in our training.

It isn’t positive reinforcement by itself that creates a positive learning experience.  An eagerness for learning comes from being a successful puzzle solver.  That success in turn comes from the kind of efficient, clean training that the clever use of resurgence encourages.

These examples give us a great perspective on creativity.  When we’re training, we aren’t waiting and waiting for our animals to do something we can reinforce.  Instead we can “seed” the behaviors we want them to draw on.  Then we set up the conditions and let them have the pleasure of discovering for themselves new or unlikely combinations.

We have a procedure for setting up the creative process.  You give your learner the repertoire, the components that form more complex behaviors, and then you set a puzzle and let extinction be the catalyst for solving it.

Coming Next: The “Pose”

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY FULL Horses: Understanding Extinction: Part 11

Being Emotional Is Being Alive
In the previous section we looked at degrees of freedom.  We often assume that someone who is at the top of their chosen profession must also be emotionally at the top of the world.  How can this successful actor or professional sports hero be anything but happy?  And yet we hear over and over again how miserably unhappy these people often are.

Degrees of freedom help us understand this paradox.  If you have become “The Expert” because that’s all you can do, you may well feel trapped and isolated.  Emotional  labels become attached to these extreme conditions.  You’ll describe yourself as being depressed, frustrated, anxious, unhappy.

Dr. Jesús Rosales-Ruiz offered us another gem, a reminder, from his discussion of regression and resurgence.  Often when people talk about “emotional behavior” such as aggression, they are forgetting that we are always emotional. It isn’t that now we are happy, and then a switch turns off and we feel nothing.

“Living is being emotional.”

The nature and intensity of the emotions fluctuates.  We experience different degrees depending upon conditions and our reinforcement history.  But thinking in terms of “emotional behavior” is too simplistic. Emotion is part of all behavior. It is not separate from it.

Traveling helps you to understand how much our emotions are a product of the habit patterns that have formed within our familiar environments and how true it is that emotions are always present. Perhaps you are one of the huge number of people who have more to do than you could possibly accomplish in one day. You have a family to care for, a house and barn to maintain, horses to feed and clean up after – not to mention ride.  All that and then there’s also an overfull schedule at work.  You’re always under stress, but it’s become so the norm, you don’t pay much attention to how you’re feeling.  A mildly stressed state is just the normal emotional background “noise”.

And then you treat yourself to the Five Go To Sea cruise where everything is different.  You still have a full day, with more to do and see than any one person could possibly squeeze into a day, but your normal triggers aren’t there.  The phone isn’t ringing.  You aren’t on the internet with the constant influx of work-related emails.  Your co-worker’s voice coming through the office wall isn’t annoying you.  All those triggers are gone and now you get to experience who you are and how you feel without them.  You become acutely aware of just how stressed you’ve been now that you’ve stepped out of your normal habit patterns. You’re still emotional, but now the environment is set up to trigger the kinds of supportive, pleasant emotions you want to experience.

So the next time you find yourself saying that your horse, your dog, your fellow human is being emotional, remind yourself that that’s an ever present condition.  “Living is being emotional.”  The question is, how can you influence conditions so the emotions that support a JOY FULL experience are the ones coming to the fore?

Coming Next: Building Unlikely Behaviors with Resurgence

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY FULL Horses: Understanding Extinction: Part 10

Degrees of Freedom
In the previous section we saw that creativity comes from having a rich repertoire to draw on. This makes puzzle solving much easier.  With my horses I work really hard to create optimistic puzzle solvers.  One way to do this is to expand the repertoire of both the handler and the learner. The broader and more extensive the repertoire, the more options an individual has. If a horse knows only two choices and neither one is working, he’s in trouble.

Dr. Jesús Rosales-Ruiz referred to this as being coerced by your repertoire. Here’s an example to explain what that means: suppose a high school student is a great debater.  In fact, he’s so good, he’s captain of the debating team.  You’d expect someone like that to have really high self-esteem. He’s so successful how could he not?

But look a little closer and you’ll see why.  This individual is great at debating, but he’s no athlete.  He’s left out of a lot of other school events.  He doesn’t play sports.  He doesn’t go to school dances.  He has poor social skills so at lunch he’s off by himself.  Yes, in debating he wins all the prizes, but he has only that one skill.

He’s being coerced into improving his debating skills because that’s all he can do.  He’s the best debater in the school, but that doesn’t keep him from feeling left out and miserable.  With only that one skill he has only one degree of freedom.  Other members of the debating team may not be as good as he is, but they are also involved in other school activities.  Compared to him they have three or four degrees of freedom, and they are much happier.

The captain is far and away the best debater on the team, but he’s been coerced into that position because he has no other choices.  For him, as well as for our horses, the way to improve his emotional well-being is to expand his repertoire so he has more options, more reinforcers available to him.

A great real life example of this is the tennis great Andre Aggassi.  In his autobiography, “Open”, he describes how his father forced him to practice tennis for hours every day.  His class mates spent their free time playing after school sports, hanging out with friends, watching TV, playing video games – in other words developing a broad repertoire of skills.  Aggassi hit tennis balls – tens of thousands of tennis balls.  He hated tennis, but he was forced to play.

His friends went off to the local high school.  He was sent away from home to a tennis academy.  He hated tennis even more, but it was all he knew.  When he turned pro, he was miserable, but how could he quit?  What else could he do?  He had no skills outside of tennis.  All he could do was become better at the game.  He was coerced by his repertoire.  He won Wimbledon and seven other Grand Slams.  He was a 1996 Olympic gold medalist.  He made millions, but he was miserable.

If he had had other choices, who knows what the outcome would have been.  He might still have chosen tennis, and perhaps he would have been an even better player. Whatever the choices, the greater degrees of freedom might well have produced a happier life.

Expanding Repertoires
Kay Laurence uses this concept with her dogs.  If you’re working with an aggressive dog, you want to expand his repertoire. Teach him a dozen new behaviors: sitting, lying down, turning his head to the left, to the right, lifting a paw, walking in a circle, etc.. Now in a threatening situation he has a dozen new ways to respond, instead of just the two or three that he started with.

Coming Next: Being Emotional Is Being Alive

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY FULL Horses: Understanding Extinction: Part 9

Eureka Moments: What is Insight?

Using resurgence – Insight
Yesterday I shared several PORTL games developed by Dr. Jesús Rosales-Ruiz.   The games deliberately used extinction.  What was observed was this: when you have been consistently reinforcing behaviors as you establish them in repertoire, and you then remove all reinforcement for them, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

When you instead extinguish the individual behaviors during the teaching phase, you get a different result.  The student will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

So how does this help us?  How can we use this understanding to shape behavior?  To get the ideas rolling Jesús shared several video examples where resurgence was used to train complex, creative behaviors.

The first video came from Robert Epstein’s work. Epstein was B.F. Skinner’s last graduate student.  Together they were exploring the concept of “insight”.  How do we solve puzzles?  Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

Bird Brains
To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into a chamber with the box and the banana.  The banana was hung up out of reach.  The pigeon couldn’t peck the banana, so an extinction process began. There was a resurgence of previously trained behaviors.  The pigeon was able to push the box under the banana, get up on the box, and peck the banana.

How did the pigeon solve this puzzle so quickly?  What is insight? What really is creativity?  Skinner and Epstein would say the pigeon could solve the problem because it had in its existing repertoire the necessary components.  Pigeons that had no experience pushing the box or jumping up on the box failed to solve the puzzle.

What is Creativity?
Jesús gives us a very process-oriented way thinking about this experiment.  This kind of complex puzzle solving was achieved through resurgence.  Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, the individual will struggle to solve the puzzle.  He will experience a much longer extinction process.  Macro extinction emotions will begin to surface, and you have to hope the subject has the persistence to become truly creative.

This is the kind of creativity that is truly stressful.  It’s much better to analyze the end goal – the complex behavior you want to train – break it down into all of it’s component tasks, and then train each of the components separately.  The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Persistence
Jesús’ comment was there is “nothing new under the sun”. The behaviors you try are all built out of things you’ve done before.  All the components of what appears to be a novel behavior have been trained in the past. So let’s consider what happens when a group of people are presented with a challenging puzzle.  When they begin experimenting and find that the usual, familiar things aren’t working, some will give up quickly.

Others will persist.  They will experiment with novel combinations of what they already know, but again most will quit if they don’t come up with a solution fairly quickly .

A few will keep trying until they stumble across a novel combination that works.  We call these people inventors and creators because they are persistent enough to find these novel combinations.  The discovery process can be a painful one, but once the new combination has been found, it’s easy for everyone else to copy the results.

I can absolutely relate to this.  Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off.  But put me in front of a computer that isn’t cooperating, and I shut down fast. There my experience has produced a different set of expectations. I’ve been in enough situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills.  I don’t have the programing background that makes wrestling with a software issue fun.  Extinction has gone too far and been too uncomfortable.  So in one situation I can be very persistent and creative.  In another I’m the one going through the classic cycle of emotions that macro extinction produces.

I know first hand both how much fun the creative process can be when the expectation of success is there.  And I also know how painful and unpleasant the extinction process is when that expectation is missing.

What I want to create for my learners is a feeling of confidence.  Whether horse or human, I want them to KNOW they can solve whatever training puzzle I throw at them. Build this expectation in early before others have taught them hard lessons about failure, and you get brilliant, enthusiastic, joyful individuals.  They are the optimists of this world.  Whether horse or human, they are fun to be around.  That’s what an understanding of these concepts helps us to create.

Coming Next: Degrees of Freedom

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY Full Horses: Understanding Extinction: Part 8

Mastering Extinction
Extinction happens all the time.  When you withhold your click, you set up an extinction process.

If you are unclear about your criteria or clumsy in your handling skills, you could be setting up your learner for a macro extinction process with all of the painful emotions that go along with it.

Or you could be using a micro extinction strategy to help shape a more complex behavior.  In this case you are using extinction to your advantage.  Extinction doesn’t have to be something you avoid.  It can be something you actively use to create more complex behavior patterns.

In yesterday’s post I described the PORTL games that Dr. Jesús Rosales-Ruiz  uses to help his students understand principles of behavior.  In his talks he shares some fascinating PORTL experiments to illustrate the difference between resurgence and regression.

Experiment One: Resurgence
The learner was taught a series of behaviors:

Behavior 1: tapping a small block. Once that behavior was confirmed, the block was removed and a toy car was placed on the table.

Behavior 2 was rolling the toy car over the table top.  When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it.  Pushing a car is an easy guess for what you would do with this kind of object.

When that behavior appeared to be solid, the car was removed and a third object, a key, was placed on the table.  Now the behavior was lifting.  Fingering a key is a normal response to this kind of object so it was easy to get the learner first to touch the key and then to lift it up off the table.  Once the learner was consistently lifting the key, that object was removed and a fourth one was introduced.

Behavior 4 involved the learner putting a wooden ring on her finger.  The learner quickly figured this out and began to consistently offer this behavior.

When each of these behaviors seemed solid – tapping the block, pushing the car, lifting the key, putting a ring on her finger – the trainer reviewed, one at a time, what the learner was to do with each of the objects.

The trainer then placed all four objects out on the table, but not in the order in which they had been taught.  The trainer observed the learner’s behavior.  She did not give any feedback or reinforcement of any kind.  The point was to see in what order the learner would interact with each object.

The result:  The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.

So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.

The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them, and then you remove reinforcement, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

Now here’s the fun part.  When you instead extinguish the individual behaviors, you get the opposite result.  Now you see regression.  The individual will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back – thus revealing his training history.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

These differences are illustrated in the second experiment.

Experiment Two: Regression
After a series of behaviors have been learned, this experiment again puts the learner through an extinction process.  In the initial set up each time the learner is moved on to a new task, an extinction process is used to eliminate the previous behavior.  Here’s the experiment:

The trainer sets out one item on the table.  The learner begins to manipulate it, trying to find out what is going to be clickable.  The trainer doesn’t click any of this creativity. She waits instead for it to extinguish and then clicks for one simple behavior – touching the object with one finger. That is the “hot” action.

The trainer clicks and reinforces for successful approximations until she has achieved a high degree of consistency in touching the object with one finger.

This was the set up for the experiment.  In the next phase she sets ten different objects out in a circle, including the one they had just been working with.  The learner begins by touching the familiar object.  That gets clicked and reinforced several times, then the trainer stops reinforcing for that object.  She is using extinction to eliminate that behavior.  The learner begins by experimenting, touching various objects, but she only gets clicked for touching the one that was immediately next to the previously hot object in a counter clockwise direction.

The learner switches over to this object and begins touching it consistently.

So now the handler stops reinforcing for this object and only reinforces for the next object on the circle.  The learner again experiments and then discovers that the only object that she gets paid for touching is the third one on the circle.

When this is consistent, the handler again stops reinforcing for touching this object.  The learner is catching on to the overall pattern. Now she moves more quickly to the fourth object and discovers that is the “hot” one to touch.

They continue counter clockwise around the circle until every object has been the “hot” one once and touching it has also been extinguished.

At this point the handler stops reinforcing altogether and simply observes the learner’s behavior.  The result: the learner quickly switches to moving clockwise around the circle, touching the objects in the reverse order in which she learned them.  So she learned them originally counter clockwise: object 1, then object 2, then object 3, then object 4, etc.

Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc.  She isn’t getting clicked for any of these touches, but the pattern is very persistent.

So again: in the first experiment where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.

In the second experiment where behaviors were extinguished, the learner went through them in the reverse order.

You won’t find these distinctions in the scientific literature. These two extinction outcomes, resurgence versus regression, are something Jesús and his students have been revealing by playing PORTL games.

Mind Games
Again Play is the key here.  PORTL may have a serious purpose behind it, but these are games.  All the creativity that comes with play is woven into these experiments.  It may turn out that others playing with similar set ups will have different results.  That’s a good thing.  That simply raises more questions, more puzzles to solve.

Do you have a question about how something works? Great. Design an experiment, test it a few times to work out the kinks in the procedure, and then invite your friends over for a pizza and PORTL party.  In the course of an evening you could have enough data to write a paper!

I do like the new twist Jesús has given to this version of the training game.  As he has pointed out, we’ve been using lab rats to learn about human behavior.  Now we are using humans to model animal behavior. Turnabout is fair play.  Much better to frustrate an undergrad than some poor lab rat!

Coming Next: Eureka Moments!  What is Insight?

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com