Goat Diaries Day 10: Distractions!

Distractions

I’ve been distracted by several projects this week so I am a bit behind getting these Goat Diary reports posted.  That seems very appropriate somehow because today’s post is about distractions!

In one of his Clicker Expo presentations Ken Ramirez talks about the importance of introducing distractions into the environment.  When he was the Director of Training at the Shedd Aquarium, he instructed his trainers to make changes every day to the training environment.  He wanted the dolphins and belugas that were used in the public demos to be so comfortable with change that if a tornado ripped the roof off the Aquarium, they would just think – “Oh look what our trainers have done for us today”.

I have always loved that image.  It creates a high standard of creativity and  consistent good training that is worth aspiring to.  With the goats at this point in their training it was easy to introduce change – essentially everything I did with them was new.  I wasn’t yet thinking about adding distractions as an active strategy.  I was starting with fearful animals so I knew I had a long way to go before they would be comfortable in a changing environment.  In their evening session I was about to discover just how easily something that I didn’t consider a distraction at all could completely derail their eagerness for training.

Lions and Tigers and Bears, Oh My! – And People, Too!  The July Goat Diaries: 7/14/17 7 pm session

In a previous post I shared with you what a happy goat looks like (https://theclickercenterblog.com/2018/01/26/).  I had taken E and P into the arena and watched with delight as they turned the mounting block into a playground.  I wanted to share the fun with Ann.  She can’t see their antics, but she can certainly hear the laughter in their feet as they run across the mounting block.

Ann came in the evening to visit with Fengur.  While she was playing with him, I sat with the goats.  When the arena was free, I set up the camera and brought them in.  Ann stationed herself beside the camera well away from them.  After my big build up about how much fun they had running over the mounting block, they were total fuddy-duddies.  There was no energy, no joy, no laughter, no interest in the mounting block at all – just a cautious inspection from a distance of Ann.  What was she doing out in the middle of the arena?  Having a new person in the arena was clearly a concern.

After a few minutes of non-performance, I decided to put them back.  They followed me into the barn aisle and went eagerly into their stall, knowing that I would be dropping treats on the floor.  It turns out that I neglected to turn on my camera, so none of their non-interest was recorded.

I let the goats settle back into the comfortable familiarity of their stall, then I took them out again individually for another leading session.  The main focus of the session was on treat delivery and their behavior around food.  I was continuing with the work I described in the previous two goat diary posts. (https://theclickercenterblog.com/2018/01/29/  and https://theclickercenterblog.com/2018/02/02/

A Panda Story

This focus on treat delivery and the time it takes to establish good manners was reminding me of Panda’s early training.  Cold winter days are a good time for stories, so I’m going to indulge in a couple, beginning with a favorite Panda story.  Panda is the miniature horse I trained to be Ann’s guide.  I remember when I first started working with Panda, she was as eager as the goats were to get into my pockets.

A week into her training – at about the stage I was now with the goats – I took Panda with me to a clinic I was giving at a barn that was about an hour away.  Ann rode in the front seat with her new guide dog curled between her feet.  Another of my clients was driving.  I was in the backseat with Panda essentially in my lap.  I was definitely a captive audience.  Doing a short session and then putting her back in her stall to process was an impossibility.  I had an hour’s drive with a horse in my lap!  What’s more I had a horse who knew I had treats in my pocket.

For the duration of the drive I clicked and treated anytime Panda’s nose moved even fractionally away from my pocket.  The idea was to keep her on such a high rate of reinforcement that she didn’t have a chance to mug me.  Over and over again, through the food placement I was saying to her – this is where the treats are delivered.  Going to my pocket gains you nothing.  Out here away from me, this is where you will find treats.  You might as well keep your nose here and not waste your energy going to my pockets.

Ann was in the front seat listening to the constant barrage of clicks.  I know they were making her anxious.  She had only recently taken on a new guide dog.  Everything about this dog was a struggle.  He should never have been placed.  The school was hoping that because Ann was such an experienced guide dog user, she would be able to make him work.

“Make the dog work” was truly the philosophy behind this dog’s training.  The result was a dog who showed extreme avoidance behavior.  Ann had one problem animal.  She didn’t want another.  How could she have a guide who needed to be clicked and treated every couple of seconds?  Ann knows how training works.  She knows that we would be building duration, but in that stage where the mugging is still such a strong reaction, the future good manners can seem impossibly far away.

Good manners emerge over time.  They are the result of consistent handling and a growing confidence in the learner.  By the time I handed Panda over to Ann, the guide dog had gone back to the school to be re-trained for a different job.  He went into search and rescue work, a job that suited his temperament much better.  And Panda became Ann’s full time guide much sooner than we had originally planned.

We celebrated the transfer by going out to dinner.  Panda kept her nose to herself and stayed quietly by Ann’s side throughout the evening.  Even when the salad course arrived, all she did was have a curious sniff before ducking her head back under the table to continue her nap. That’s great duration in a behavior that had begun with barely seconds between clicks.

Good manners emerged for Panda, and I was confident that they would also become the norm for the goats.  Time and consistency would create the behavior I wanted.

p46_PandaInRestaurantWithTrainerAlexandraKurlandOwnerAnnEdieNeilSoderstrom 343

Dining out with Panda

(If you want to learn more about Panda and her training, read the Panda Reports on my web site: theclickercenter.com.  Some of her early training is also featured in my DVDs: An Introduction to Clicker Training and Lesson 4: Stimulus Control.

Treats: Whatever Is Logical Do The Opposite

At some point in the distant future, it might be fun to travel with the goats in my car.  But at this point the thought of spending an hour trapped in the backseat of a car with an eager, greedy goat sounded exhausting.  We had a long way to go before they would be as settled about treats as Panda.

You meet your learner where he is not where you want him to be.  When I took P back into the arena, the session was very much focused around food delivery.  The children in the 4-H program may have giggled and let him snatch pretzels from their mouths.  With me P was learning that we played a very different game.

I brought P back out on a lead.  He continued to show good progress. He backed away from my closed hand.  He did a bit of head flinging which means he was feeling frustrated by having to back up.  I’m sure it did conflict with how he thought things should be done.  He wanted to push forward to get to the treats.  That’s what he had always done, but now he had to remember to back up instead.

Whatever is logical, do the opposite.  I could sympathize with his frustration.  From his point of view it made no sense that backing should work.  Going forward was how you get children to spill treats all over the ground.  Why should backing work?!!  We have all been given directions that make no sense.  Why should turning left instead of right get us to our destination?

And how many of us turn right because we’re convinced that should be the correct answer.  Even when we do turn left, it feels wrong.  Surely we’re heading in the wrong direction.  This can’t be right.  We’ll never get there.  Oh look, there’s our destination just ahead. How did that happen!?

It can take a while to relax and trust the directions.  That’s the stage I was in with P.  With a little more reinforcement history behind us, he would relax into the confidence that treats were coming.  There was no need to rush to get them.

The Goat Palace written Dec. 27 – Our Animals Always Tell Us

Meeting your learner where he is, not where you want him to be makes me want to share this story.  It was prompted by the goat’s current training.  If E and P’s treat taking manners were reminding me of Panda, a session I did with Trixie and Thanzi at the end of December made me think of Robin.  There are several of training mantras that apply to this session:

Our animals will always tell us what they need to work on next.

You get what you reinforce. 

My favorite, though, is this one:

If you don’t notice a little resistance, don’t worry about it.  It will get bigger.  And eventually, it will get big enough that you will do something about it.

Before I describe the goat’s training, here is Robin’s story:

Over the winter when Robin was still very new to clicker training, he started to snatch his treat from my hand.  I’d click, he’d grab, and then he’d eagerly be offering me the next clickable behavior.  I ignored the snatching.  He was eager.  It was cold.  He was offering lots of great work.

The snatching increased.  You get what you reinforce.  I didn’t like the snatching, but if it was getting worse, something in our interactions was reinforcing it.

I ignored it.  Robin was eager.  It was cold.  We were having fun – until I wasn’t.  The snatching was becoming more than annoying.  I was starting to count fingers after I gave him a treat.  It was time to do something about the way he took treats.

If you don’t notice a little resistance, don’t worry about it.  It will get bigger.  And eventually, it will get big enough that you will do something about it.

I’ve told the story many times about the way I solved this particular problem.  It’s detailed in both my Riding book and The Step-By-Step Guide in Pictures.  I went back to basics.  I put Robin in his stall with a stall guard across the door.  I stood across the aisle from him and held out the biggest carrot from a bag of big carrots.  Robin stretched his neck out to try and reach it.

I immediately turned my back, removing the carrot from sight and counted to three.  Then I turned back and held the carrot out again.  Robin stretched out his nose.  I turned my back and counted to three.

I again offered the carrot.  This time Robin hesitated ever so slightly.  I clicked, reached into my pocket and handed him a piece of carrot. I was using negative punishment.  I was taking away something Robin wanted – the carrot – to decrease a behavior I didn’t like – the reaching out towards me to get a treat.

(When an activity decreases – it is being punished, either by adding something unpleasant or by taking away something the individual enjoys (positive punishment and negative punishment – it’s just math adding or subtracting).  When an activity increases, it is being reinforced, either by adding something the individual wants or taking away something he doesn’t like. So again there is positive and negative reinforcement.  When the behavior increases it is being strengthened, i.e. reinforced.  When it decreases, it is being punished.  In both – the positive and negative refer to adding or subtracting, not value judgements.)

I offered the carrot again.  Robin hesitated.  Click, I handed him a piece of carrot from my pocket.  Robin is a super fast learner.  He had the dots connected.  If he drew back away from my hand, he got treats.  I could hold the carrot directly under his nose and instead of snatching it off my hand, he arched his neck and drew up away from it.  Click and treat.

I was enchanted.  He looked like a beautiful dressage horse.  Robin being Robin, he quickly made the connection.  If he arched his neck, click, I would give him a treat.  He wasn’t snatching anymore.  Instead he scooped the carrot slice gently off my hand with his enormous soft lips.

He started to offer what I have since called “the pose”.  When I walked by his stall, Robin would draw himself up and arch his neck.  Click.  I’d pause in my barn chores and give him a piece of carrot.  Through the winter I reinforced him a lot for this behavior.  I might have begun with negative punishment as I tried to stop an unwanted behavior – snatching treats off my hand.  Now I was actively reinforcing him for something I wanted – “the pose”.

I should add that this is not the way I teach the pose today.  It popped out when I was working on something else.  Now that I know this behavior is worth going after, I shape it more directly, most often with the aid of targeting.  And in general, when I find myself reaching towards a negative punishment strategy to solve a problem, I go have a cup of tea instead. I think about what I want and look for reinforcement-based teaching strategies instead.

The “pose” is not the best name that I could have come up with for this behavior.  For many people, a pose is a fixed, rigid, stilted posture.  It’s that awful grimace so many of us have when we’re forced to have our picture taken.

Instead, for me, the pose is a very dynamic behavior.  For Robin it has become a default behavior.  I was the cue.  In the absence of any active cue from me, if Robin posed, I would click and reinforce him.  It meant that if he wanted attention from me, he could get me to engage with him using a behavior I actively liked.

Horses are always doing something.  A horse in a stall has a long laundry list of behaviors to choose from.  Some are behaviors that I like, some are behaviors that I can ignore, and some are behaviors that I never want to see.  The laundry list includes taking a nap, eating hay, having a drink, watching the activities in the barn aisle – all perfectly acceptable and easy to ignore.

A horse could also be fighting with his neighbor, kicking the stall door to get attention, cribbing, raking his teeth up and down the wall, pacing, weaving.  These are behaviors I definitely do not want.  But if I fuss at a horse when I see him engaging in them, I could easily be reinforcing them through my attention.  Think of the small child who bangs the kitchen pots and pans while mother is on the phone.   Even negative attention is attention, and that can be better than no attention at all.

Robin doesn’t have to kick the wall to get me to notice him.  All he has to do is pose.  Click and treat.  I love having behaviors which my horses can use to ask for my attention. They know I will always acknowledge their request for connection.

Think of all the ways people interact with one another:  “Good morning.”  “How are you?” “Never better.”  These quick exchanges connect us.  Think how chilling and unpleasant an environment becomes when these social pleasantries are absent.  We need them to tell us things are okay between us.

Robin says good morning by posing.  I respond with a click and a treat.  All is well between us.  Our social bond is strong and getting stronger with each click and treat.

I reinforced Robin for the pose because he looked pretty.  I wasn’t heading for anything in particular beyond that.  This is what makes training so much fun.  Sometimes the next unexpected piece just pops out.

Here’s what happened to the pose.  One evening I had Robin in the arena.  I was asking him to trot around me on a circle.  He was giving me a nothing of a trot.  He looked like an old plow horse.  There was no energy, no pizzazz, nothing I wanted to reinforce.

Robin was expecting me to click.  He went once around the circle.  Nothing.  The way I tell the story was you could all but see the cartoon bubble appearing above his head.  “I’m not being reinforced.”

He went around again.

“What can I do to get reinforced?”

On the third time around he had the answer: “I know! I’ll try the pose!”

The way Dr. Jesus Rosales-Ruiz tells this story is this: by withholding the click I was putting Robin into an extinction process.  He began to regress back through behaviors that had been successful in the past.  The pose had been highly reinforced, so it was the first thing that he tried.

Whichever version of the story you prefer, Robin had to add energy to the trot in order to get into the pose.  Suddenly, his trot looked as though it belonged on a magazine cover.  He was gorgeous!  I clicked and gave him a treat, all the while gushing over how pretty he was.  I sent him back out around me.  It took him a few strides to find his balance, but he once again added the pose to the trot.  It was just one stride that I was clicking – but wow! What a gorgeous stride it was!  The rest is history.  Robin led the way.  He showed us that we could shape the beautiful, suspended balance of a classical dressage horse just through well timed clicks and treats.

Why am I telling this story? Because this morning’s session with Thanzi and Trixie made me think of Robin and the pose.  It reminded me of the expression:

If you don’t notice a little resistance, don’t worry about it.  It will get bigger.  And eventually, it will get big enough that you will do something about it.

In December I had been trying to work them individually.  We had snow Christmas eve and then the temperatures dropped and the wind rose.  Trixie was nervous about being out in the hallway by herself, so I let Thanzi join her.  Suddenly with two goats I had lots of crowding.  Hmm.  You get what you reinforce.  I knew at night when I was tucking them in, I was in a hurry.  It was cold.  It was late.  I just wanted to get done with the final chores and get back inside where it’s warm.  Had I been letting them crowd me and hurry the treat deliver?  Apparently the answer was yes.

I needed to sort out the crowding so in this session I set two mats out face too face.  Trixie hopped on one, Thanzi on the other.  I stood in the middle with both goats crowding into me begging for treats.  I waited.

“Oh right.  Crowding doesn’t get treats.”  They took their noses away from me.  Click. I reached into my pockets.

They were right back, pushing against my hands.  I got the treats out of my pockets and then drew my hands together.  I stood as though in calm meditation, waiting.  First one then the other took her nose away.  I waited until they were both good, then held out the treats to them.

They got their treats, and then they were right back crowding me, pushing against me with their muzzles.  I waited.  They took their noses away.  Click.  Get the treat.  Wait again with hands held together in quiet meditation.  They both drew away from me.  I held out my hands and let them take the treat.

It only took a couple of repetitions. They were both working so hard to stay away from my pockets.  Click, pause, feed.  They were both so good.

I left them in the hallway while I filled their hay feeders.  I was just finishing up when I looked out into the aisle.  They were standing each on her own platform waiting for me.  How can you resist?  I went out and did another round of paying attention to their good manners.

Your animals always tell you what they need to work on.  I don’t know where this will lead me, but I know it is what they need.  If it makes me think of Robin’s pose, I must be on the right track.

Staying Consistent

It’s easy to be focused and consistent through one training session.  It’s much harder to maintain that consistency over time.  When we transferred Panda full time to Ann, it was actually a relief to hand her over.  I missed her constant presence by my side, but maintaining the level of consistency that is needed for a guide was demanding.  When you can see, you don’t need a guide to tell you that you’ve come to a curb. If I started cutting corners in Panda’s training because I didn’t need all the things I had taught her to do, it would undermine her performance as a guide.  Ann would never be able to enjoy the luxury of seeing the curb that’s in front of her.  She would be relying on Panda to point this out to her.  A horse doesn’t know when it doesn’t count so it always has to count.  I followed that mantra throughout Panda’s training.

The same thing applies to the goats.  The same thing applies to the goats.  If sometimes I let them push into me to get treats, I will never get to the consistent good behavior that I want.  But it’s been cold!  It is so easy to get in a hurry and let standards drop.  So their training has been a bit like a yo yo.  I let things slip in my hurry to get chores done and my gloves back on.  They begin to crowd me, but now I am catching it sooner.  The manners pendulum keeps swinging back and forth.  Over time the cumulative effect shows me that the balance is tipping towards good manners.

Just for Fun!

I told you the story of Robin’s pose.  Here’s one of my favorite videos of Robin.  He was only three when this was filmed.  He had not yet been started under saddle.  So he’d never had a rider on his back, and I had never lunged him in side reins or any other type of mechanical device.  This beautiful balance and cadence had been shaped entirely with the clicker.  You’ll see I am holding two dressage whips.  You can call them anything you want, but they are functioning as targets.  They give him points of reference to balance between.  I know the lighting is not good in this video, but this was a long time ago, and this was the best the video camera could do.  Enjoy!

 

Coming Next: The Goat Diaries Day 10: You Can Never Do One Thing

Please Note: if you are new to the Goat Diaries, these are a series of articles that are best read in order.  The first installment was posted on Oct. 2nd.  I suggest you begin there: https://theclickercenterblog.com/2017/10/02/   Two of the goats I write about originally came for a twelve day stay in July.  The July Goat Diaries track their training during this period.  In November these two goats, plus three others returned.  They will be with me through the winter.  The “Goat Palace” reports track their current training.  I wish to thank Sister Mary Elizabeth from the Community of St. Mary in upstate NY for the generous loan of her beautiful cashmere goats.

 

 

 

JOY FULL Horses: Understanding Extinction: Part 12

Mastering Micro: Building Unlikely Behaviors with Resurgence
Nothing is either all good or all bad.

We want to use positive reinforcement with our animals because we see it as being both effective and more humane.  But the associations created through positive reinforcement can create addictions to harmful behaviors.  Think about the way advertisers manipulate our behavior to encourage smoking or overeating.

Resurgence and regression can be very negative procedures, but they can also be used to produce what might otherwise be very difficult behaviors to obtain.

If you aren’t sure how you can turn what seems like a negative procedure into a positive teaching strategy, PORTL can once again help to illustrate how this works.

Here’s the set up:

The trainer sets a toy chair on the table for her learner to interact with. The goal is to get the learner to push the chair over the table the way she might push a toy car.

We’ll now observe quietly in the background while the learner begins to interact with the chair.  The trainer could get lucky.  The learner might begin offering the behavior she’s after within the first couple of clicks.  But with this learner there’s no sign of any chair pushing behavior. Why?

History matters.

The learner is going to draw on all of her previous repertoire of things she has done with chairs.  In this case we have a learner who was scolded as a child for pushing her chair over the floor, so she’s not very likely to offer this type of behavior with the toy chair.

A history of punishment has played a role in depressing chair pushing behavior for this learner, but pushing would also have been an unlikely behavior if the trainer had set down a dice. The learner would have tossed the dice or shaken it in her hand because that’s what you do with this kind of object. Pushing a dice over the table like a toy car is not an obvious behavior to try.

Through a series of small approximations, the trainer could try to shaping the behavior she wants.  Her first step would be reinforcing the learner for touching the chair.

The learner in this case is not particularly creative.  She offers simple touches, but nothing else.  Again, the trainer may be dealing with a history of punishment.  Her learner doesn’t have a lot of experience being reinforced for trying things.  In fact, quite the opposite – she may have been punished for stepping “outside the lines”.  She is like so many of our animal learners – hesitant, lacking in confidence, and not showing any outward signs of curiosity.  In her first few attempts she touches the chair, but she doesn’t try any other behaviors.  Getting her to push the chair is going to be hard.

So the trainer takes the chair away and sets out a toy car. Using an object that normally would be pushed makes it very easy to get the desired action.  The learner pushes the car over the table top. Click and treat.

This is repeated several times, and then the trainer takes the car away and sets the chair out.  The learner goes back to touching it.  The chair accidentally falls over – click and treat. The learner latches on to that, expanding her repertoire to two behaviors – touching the chair and knocking it over.

We see this so many times with our animal learners.  One click and suddenly you’ve locked in a behavior you don’t want.  With a creative learner this isn’t a problem.  You can quickly shift the behavior into something you want, but with these “one trick ponies” you have to be so very careful what you click.  In this case the learner persists in knocking the chair over even when she is no longer getting reinforced for the action.

Her trainer makes a quick decision and decides to put everything but pushing the chair like a car on extinction.  Her learner is clearly becoming frustrated.  To avoid having her shut down completely, the trainer takes the chair away and sets the car out again.  The learner immediately starts pushing the car over the table top.  Click and treat.

To help with the generalization the trainer puts a third object out – a small block. The learner pushes the block.  Click and treat.  This is repeated several times, then the trainer takes the block away and sets out the car.  The car is pushed. Click and treat.

The trainer sets the chair out, and the learner pushes the chair.  Job done.

Resurgence and Dog “Yoga”
Using the car in this way is an elegant teaching strategy.  Often when we come up with these clever ways of helping our learner to be successful, we know that it works, but we don’t really have good explanations for why.   Understanding resurgence helps us with the why in this case.  And it helps us to be more deliberate in the use of this kind of teaching strategy.  Here’s another example.

One of Kay Laurence’s students taught her dog to step up with his hind legs onto a chair.  It was elegant training, a beautiful example of setting the learner up for success.  In his talk on extinction, Dr. Jesús Rosales-Ruiz helped us to see that it was also a great example of using resurgence.

Here’s the lesson: First, the dog learned to stand one foot each on four small plastic pods. This alone was impressive training.  The pods were the same ones physiotherapists use to help people improve their balance and proprioception. It took great coordination for the dog to stay balanced on the four pods. But that was only step 1.  Next he learned to keep his front feet on the floor while he maneuvered his hind feet up onto the brick ledge of a fireplace hearth.

Adding in the precision of the pods came next.  Now the dog wasn’t just standing with his front paws on the floor and his hind end up on the ledge.  He was also balancing on all four pods.

This was not done as a cute party trick.  The dog’s owner is a yoga teacher.  Her interest was very much the same as mine – helping her animal learner maintain a healthy spine.  In this orientation she could ask her dog for weight shifts that contribute to a flexible spine.

The last step was setting up a training session next to a chair. The handler withheld the click, putting the dog into an extinction process. With very little experimentation, the dog oriented himself so his hind end was to the chair.  He certainly demonstrated the flexibility of his spine by stepping up onto the chair with his hind legs so he was standing hind end up on the chair and front feet on the floor.

Generalization and Creativity
Jesús commented that if we didn’t know about resurgence we would simply be saying the dog generalized.  That’s not a sufficient explanation.  What we were seeing was a great example of resurgence. PORTL has given us a better understanding of how to encourage this kind of problem solving.  When we want to train for this type of generalization, knowing about the “why” of resurgence helps us to be more deliberate and efficient in our training.

It isn’t positive reinforcement by itself that creates a positive learning experience.  An eagerness for learning comes from being a successful puzzle solver.  That success in turn comes from the kind of efficient, clean training that the clever use of resurgence encourages.

These examples give us a great perspective on creativity.  When we’re training, we aren’t waiting and waiting for our animals to do something we can reinforce.  Instead we can “seed” the behaviors we want them to draw on.  Then we set up the conditions and let them have the pleasure of discovering for themselves new or unlikely combinations.

We have a procedure for setting up the creative process.  You give your learner the repertoire, the components that form more complex behaviors, and then you set a puzzle and let extinction be the catalyst for solving it.

Coming Next: The “Pose”

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY FULL Horses: Understanding Extinction: Part 9

Eureka Moments: What is Insight?

Using resurgence – Insight
Yesterday I shared several PORTL games developed by Dr. Jesús Rosales-Ruiz.   The games deliberately used extinction.  What was observed was this: when you have been consistently reinforcing behaviors as you establish them in repertoire, and you then remove all reinforcement for them, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

When you instead extinguish the individual behaviors during the teaching phase, you get a different result.  The student will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

So how does this help us?  How can we use this understanding to shape behavior?  To get the ideas rolling Jesús shared several video examples where resurgence was used to train complex, creative behaviors.

The first video came from Robert Epstein’s work. Epstein was B.F. Skinner’s last graduate student.  Together they were exploring the concept of “insight”.  How do we solve puzzles?  Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

Bird Brains
To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into a chamber with the box and the banana.  The banana was hung up out of reach.  The pigeon couldn’t peck the banana, so an extinction process began. There was a resurgence of previously trained behaviors.  The pigeon was able to push the box under the banana, get up on the box, and peck the banana.

How did the pigeon solve this puzzle so quickly?  What is insight? What really is creativity?  Skinner and Epstein would say the pigeon could solve the problem because it had in its existing repertoire the necessary components.  Pigeons that had no experience pushing the box or jumping up on the box failed to solve the puzzle.

What is Creativity?
Jesús gives us a very process-oriented way thinking about this experiment.  This kind of complex puzzle solving was achieved through resurgence.  Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, the individual will struggle to solve the puzzle.  He will experience a much longer extinction process.  Macro extinction emotions will begin to surface, and you have to hope the subject has the persistence to become truly creative.

This is the kind of creativity that is truly stressful.  It’s much better to analyze the end goal – the complex behavior you want to train – break it down into all of it’s component tasks, and then train each of the components separately.  The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Persistence
Jesús’ comment was there is “nothing new under the sun”. The behaviors you try are all built out of things you’ve done before.  All the components of what appears to be a novel behavior have been trained in the past. So let’s consider what happens when a group of people are presented with a challenging puzzle.  When they begin experimenting and find that the usual, familiar things aren’t working, some will give up quickly.

Others will persist.  They will experiment with novel combinations of what they already know, but again most will quit if they don’t come up with a solution fairly quickly .

A few will keep trying until they stumble across a novel combination that works.  We call these people inventors and creators because they are persistent enough to find these novel combinations.  The discovery process can be a painful one, but once the new combination has been found, it’s easy for everyone else to copy the results.

I can absolutely relate to this.  Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off.  But put me in front of a computer that isn’t cooperating, and I shut down fast. There my experience has produced a different set of expectations. I’ve been in enough situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills.  I don’t have the programing background that makes wrestling with a software issue fun.  Extinction has gone too far and been too uncomfortable.  So in one situation I can be very persistent and creative.  In another I’m the one going through the classic cycle of emotions that macro extinction produces.

I know first hand both how much fun the creative process can be when the expectation of success is there.  And I also know how painful and unpleasant the extinction process is when that expectation is missing.

What I want to create for my learners is a feeling of confidence.  Whether horse or human, I want them to KNOW they can solve whatever training puzzle I throw at them. Build this expectation in early before others have taught them hard lessons about failure, and you get brilliant, enthusiastic, joyful individuals.  They are the optimists of this world.  Whether horse or human, they are fun to be around.  That’s what an understanding of these concepts helps us to create.

Coming Next: Degrees of Freedom

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY Full Horses: Understanding Extinction: Part 8

Mastering Extinction
Extinction happens all the time.  When you withhold your click, you set up an extinction process.

If you are unclear about your criteria or clumsy in your handling skills, you could be setting up your learner for a macro extinction process with all of the painful emotions that go along with it.

Or you could be using a micro extinction strategy to help shape a more complex behavior.  In this case you are using extinction to your advantage.  Extinction doesn’t have to be something you avoid.  It can be something you actively use to create more complex behavior patterns.

In yesterday’s post I described the PORTL games that Dr. Jesús Rosales-Ruiz  uses to help his students understand principles of behavior.  In his talks he shares some fascinating PORTL experiments to illustrate the difference between resurgence and regression.

Experiment One: Resurgence
The learner was taught a series of behaviors:

Behavior 1: tapping a small block. Once that behavior was confirmed, the block was removed and a toy car was placed on the table.

Behavior 2 was rolling the toy car over the table top.  When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it.  Pushing a car is an easy guess for what you would do with this kind of object.

When that behavior appeared to be solid, the car was removed and a third object, a key, was placed on the table.  Now the behavior was lifting.  Fingering a key is a normal response to this kind of object so it was easy to get the learner first to touch the key and then to lift it up off the table.  Once the learner was consistently lifting the key, that object was removed and a fourth one was introduced.

Behavior 4 involved the learner putting a wooden ring on her finger.  The learner quickly figured this out and began to consistently offer this behavior.

When each of these behaviors seemed solid – tapping the block, pushing the car, lifting the key, putting a ring on her finger – the trainer reviewed, one at a time, what the learner was to do with each of the objects.

The trainer then placed all four objects out on the table, but not in the order in which they had been taught.  The trainer observed the learner’s behavior.  She did not give any feedback or reinforcement of any kind.  The point was to see in what order the learner would interact with each object.

The result:  The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.

So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.

The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them, and then you remove reinforcement, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

Now here’s the fun part.  When you instead extinguish the individual behaviors, you get the opposite result.  Now you see regression.  The individual will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back – thus revealing his training history.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

These differences are illustrated in the second experiment.

Experiment Two: Regression
After a series of behaviors have been learned, this experiment again puts the learner through an extinction process.  In the initial set up each time the learner is moved on to a new task, an extinction process is used to eliminate the previous behavior.  Here’s the experiment:

The trainer sets out one item on the table.  The learner begins to manipulate it, trying to find out what is going to be clickable.  The trainer doesn’t click any of this creativity. She waits instead for it to extinguish and then clicks for one simple behavior – touching the object with one finger. That is the “hot” action.

The trainer clicks and reinforces for successful approximations until she has achieved a high degree of consistency in touching the object with one finger.

This was the set up for the experiment.  In the next phase she sets ten different objects out in a circle, including the one they had just been working with.  The learner begins by touching the familiar object.  That gets clicked and reinforced several times, then the trainer stops reinforcing for that object.  She is using extinction to eliminate that behavior.  The learner begins by experimenting, touching various objects, but she only gets clicked for touching the one that was immediately next to the previously hot object in a counter clockwise direction.

The learner switches over to this object and begins touching it consistently.

So now the handler stops reinforcing for this object and only reinforces for the next object on the circle.  The learner again experiments and then discovers that the only object that she gets paid for touching is the third one on the circle.

When this is consistent, the handler again stops reinforcing for touching this object.  The learner is catching on to the overall pattern. Now she moves more quickly to the fourth object and discovers that is the “hot” one to touch.

They continue counter clockwise around the circle until every object has been the “hot” one once and touching it has also been extinguished.

At this point the handler stops reinforcing altogether and simply observes the learner’s behavior.  The result: the learner quickly switches to moving clockwise around the circle, touching the objects in the reverse order in which she learned them.  So she learned them originally counter clockwise: object 1, then object 2, then object 3, then object 4, etc.

Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc.  She isn’t getting clicked for any of these touches, but the pattern is very persistent.

So again: in the first experiment where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.

In the second experiment where behaviors were extinguished, the learner went through them in the reverse order.

You won’t find these distinctions in the scientific literature. These two extinction outcomes, resurgence versus regression, are something Jesús and his students have been revealing by playing PORTL games.

Mind Games
Again Play is the key here.  PORTL may have a serious purpose behind it, but these are games.  All the creativity that comes with play is woven into these experiments.  It may turn out that others playing with similar set ups will have different results.  That’s a good thing.  That simply raises more questions, more puzzles to solve.

Do you have a question about how something works? Great. Design an experiment, test it a few times to work out the kinks in the procedure, and then invite your friends over for a pizza and PORTL party.  In the course of an evening you could have enough data to write a paper!

I do like the new twist Jesús has given to this version of the training game.  As he has pointed out, we’ve been using lab rats to learn about human behavior.  Now we are using humans to model animal behavior. Turnabout is fair play.  Much better to frustrate an undergrad than some poor lab rat!

Coming Next: Eureka Moments!  What is Insight?

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY FULL HORSES: Understanding Extinction – Part 5

Using “Hot” Behaviors

The Measure of Success
When horses are engaged in a successful shaping session, it can seem as though they never stop eating.  If you aren’t familiar with clicker training it can look as though the handler is constantly clicking and treating.  Don’t they ever stop feeding?  How is this going to work?  How do you raise criteria if you’re always feeding?

In a good shaping session the next criterion you’re going to shift to is already occurring a high percentage of the time BEFORE you make it the new standard.  Suppose I’m working on grown-ups, and I’ve decided that I want my horse to have his ears forward.  That’s a great goal, but if I abruptly stop clicking for good head position because the ears are back, guess what I’ll get – more pinned ears.  Why? Because I’m frustrating my horse and that emotion is expressed through pinned ears.

I’ll also get him swinging his head, nudging my arm, pawing etc., all the behaviors that I thought I had extinguished as I was building my grown-ups.

Using “Hot” Behaviors
What is the solution?  I could begin by separating out ears from other criterion. During casual exchanges when we aren’t in a formal training session, every time I see my horse with his ears forward, click, I’ll reinforce him.  If I’m walking past his stall and he puts his ears forward, click, he’ll get a treat. Pretty soon I’ll see that my presence is triggering ears forward.  I’ve made it a “hot” behavior.

So, now if I withhold the click in grown-ups, I’m likely to get a resurgence of “hot” behaviors.  I’m still using extinction, but I’ve set my horse up for success. The behavior that is going to pop out is the one I’ve recently made “hot” – in this case ears forward.

Click For What You Already Have
I won’t even shift my focus to ears forward until they are already occurring at a high frequency.  My goal is to have him standing beside me with his ears forward, but initially I’m happy if he simply takes his nose away from my arm.

As I click him for keeping his head directly between his shoulders, some variability is going to come into the overall behavior.  Sometimes he’ll have his head slightly higher, or lower, his ears forward or back.  I may be so busy monitoring the orientation of his head, I won’t even notice what he is doing with his ears.

As his head stabilizes and his overall orientation becomes more consistent, I’ll be able to take in more of these subtle variations. The movement of his ears pricking forward will catch my attention.  I’ll become increasingly aware of what he is doing with his ears.  If they are almost always pinned, there’s no point in making ears forward the next criterion.  I’ll be surfing a long extinction wave before ears forward pops out. In fact for something like ears, the more frustrated he becomes, the less likely they are to go forward.

So I’ll “prime the pump” instead.  I’ll make ears forward a hot behavior.  Now when he’s in grown-ups, if I make ears forward the next criterion, I’ll be withholding my click for only a second or two.  My horse won’t be perceiving the event as unpleasant or frustrating. The click will shift seamlessly to the new criterion.  That slight moment of extinction causes my horse to surf through current “hot” behaviors. I’m using resurgence, but in a way that sets the horse up to have success build on success.

Coming Next: Cues and Extinction

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

JOY Full Horses: Understanding Extinction: Part 2

Yesterday’s post ended with a quote from Dr. Jesús Rosales-Ruiz: “If you don’t understand extinction, you won’t be able to master it.”  Today’s post will begin to unravel what that means.

Regression and Resurgence
I’ve talked about regression in previous posts.  Now we need to add in resurgence and the distinction between them.

In regression you revert back to previously extinguished behaviors.

In resurgence you revert back to previously reinforced behavior.

This isn’t just semantics.  According to Jesús regression and resurgence emerge out of different training strategies and produce different outcomes.

Regression can be defined as: “If the present behavior is not capable of getting reinforcement, one reverts to older forms of response which were once effective, but which have previously been extinguished.” The order in which this unfolds is significant.

Under stress you will revert to an older way of behaving.  If that behavior is not reinforced, you’ll go through another extinction process.  You’ll revert back to even older behaviors. You’ll keep trying things and trying things, until you either give up entirely, or you are pushed to creativity.  This can be a stressful process which is why some people equate creativity with an unpleasant experience.  If you were to suggest to them that they take a creative writing class, they would be running for the hills!  In their experience there’s nothing fun about being creative.  How very sad!

Extinction History
Regression emerges because a behavior which normally earns reinforcement is no longer working.  Often we think of extinction as simply a procedure that’s intended to reduce behavior.  You don’t like a dog’s barking so you never reinforce it in the hope that the behavior will go away.  This simplistic view misses an important key to understanding how to use extinction. A dog that isn’t barking is still doing something.  What is the “something” that takes the place of the barking?  The behaviors that emerge in an extinction process are not random. Understanding the order lets you master the process.

Jesús described extinction as the mirror image of reinforcement.

Extinction tells you what was reinforced in the past.

Reinforcement tells you what behaviors you are building for the future.

I wrote about this is previous posts.  (https://theclickercenterblog.com/2016/11/17/)  When you are first learning about clicker training, if your handling confuses the horse and puts him into an extinction process, you are revealing his past.   You may see his frustration expressed as pawing, pinned ears, even occasionally biting.  Don’t blame yourself for the outburst. You didn’t create the behavior you’re now dodging.  Turn your spotlight instead on his past.  That’s where the behavior was learned.

The Catalyst, Not the Cause
You may be the catalyst, but you are not the cause.  That’s good news. You don’t have to take his behavior personally. The cause sits not in the present, but in the past. It’s only natural to become worried by the emotional reaction you’re seeing. People sometimes inadvertently end up compounding the problem. If their handling skills are clumsy or they don’t yet know how to manage the environment, they can put the horse into even more of an extinction process.

I’ve seen this in beginner handlers.  They don’t yet understand how much a lack of clear criteria can impact a learner.  Everything starts out so wonderfully.  The horse gets clicked and reinforced for touching a target.   What fun!  But then the handler gets distracted.  She misses three or four clickable moments.  Those missed clicks can put the horse into an extinction process that leads to emotional outbursts. That’s where video cameras can be so useful. Video helps the handler see the training from the horse’s point of view.  It reveals the good tries he’s offering and helps the handler understand more clearly what needs to be reinforced.

The solution to the extinction puzzle lies in embracing clicker training.  Through clicker training you’ll be building a repertoire of behaviors that gives the horse alternatives to his old patterns. Instead of reverting back to behaviors you don’t want, now the extinction process will be popping out behaviors you’ve planted, behaviors you like and that you can reinforce.  Suddenly, you aren’t in an extinction process anymore.  You’re back on track with high rates of reinforcement.

This will take bit more unraveling of the extinction puzzle to understand.

Coming Next: Extinction Reveals the Past

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

 

JOY FULL Horses: Understanding Extinction: Part 1

Today’s post begins the final chapter of JOY Full Horses, but don’t worry about the book ending too soon.  I still have a month’s worth of posts to go.  The first installment went up January 3, 2016.  I thought maybe I’d be done around April. Ha!  I hadn’t realized how much work was going to be involved in putting up each post.  My goal is to finish by January 3, 2017.  We’ll see if I make it.  In the meantime, if you have friends who would enjoy reading the JOY Full Horses blogs, do share the links.   The more the merrier in this Holiday season!

Enjoy this next installment.

Extinction and Shaping
In the last couple of posts I’ve been sharing with you a presentation given by Dr. Jesús Rosales-Ruiz on extinction.  Many clicker trainers will say they never use extinction.  I know I work hard to set up my training so horses aren’t put into the kind of guessing game that leads to outbursts of frustration and aggression.  That’s something I very much want to avoid. But that doesn’t mean I don’t use extinction. That’s what Dr. Jesús Rosales-Ruiz makes so clear when he talks about resurgence and regression.

To the people who say they never use extinction, his response is: “What do you mean you never use extinction!  Extinction is at the heart of shaping.  Shaping is differential reinforcement. It’s the interplay between positive reinforcement and extinction.  So if someone says they aren’t using extinction, probably they don’t understand what they are saying.”

That’s such a wonderfully blunt and typical Jesús comment. Thankfully he doesn’t leave people just with that.  He goes on to explain what he means.  At the core of his presentations is this statement: “If you don’t understand extinction, you won’t be able to master it.”

So what does it means to master extinction, and how do you put it to use in a positive way?  That’s what I’ll be exploring in the coming posts.

Coming Soon: Regression and Resurgence

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com