Extinction

by

Goat Diaries Day 10: Distractions!

Distractions

I’ve been distracted by several projects this week so I am a bit behind getting these Goat Diary reports posted.  That seems very appropriate somehow because today’s post is about distractions!

In one of his Clicker Expo presentations Ken Ramirez talks about the importance of introducing distractions into the environment.  When he was the Director of Training at the Shedd Aquarium, he instructed his trainers to make changes every day to the training environment.  He wanted the dolphins and belugas that were used in the public demos to be so comfortable with change that if a tornado ripped the roof off the Aquarium, they would just think – “Oh look what our trainers have done for us today”.

I have always loved that image.  It creates a high standard of creativity and  consistent good training that is worth aspiring to.  With the goats at this point in their training it was easy to introduce change – essentially everything I did with them was new.  I wasn’t yet thinking about adding distractions as an active strategy.  I was starting with fearful animals so I knew I had a long way to go before they would be comfortable in a changing environment.  In their evening session I was about to discover just how easily something that I didn’t consider a distraction at all could completely derail their eagerness for training.

Lions and Tigers and Bears, Oh My! – And People, Too!  The July Goat Diaries: 7/14/17 7 pm session

In a previous post I shared with you what a happy goat looks like (https://theclickercenterblog.com/2018/01/26/).  I had taken E and P into the arena and watched with delight as they turned the mounting block into a playground.  I wanted to share the fun with Ann.  She can’t see their antics, but she can certainly hear the laughter in their feet as they run across the mounting block.

Ann came in the evening to visit with Fengur.  While she was playing with him, I sat with the goats.  When the arena was free, I set up the camera and brought them in.  Ann stationed herself beside the camera well away from them.  After my big build up about how much fun they had running over the mounting block, they were total fuddy-duddies.  There was no energy, no joy, no laughter, no interest in the mounting block at all – just a cautious inspection from a distance of Ann.  What was she doing out in the middle of the arena?  Having a new person in the arena was clearly a concern.

After a few minutes of non-performance, I decided to put them back.  They followed me into the barn aisle and went eagerly into their stall, knowing that I would be dropping treats on the floor.  It turns out that I neglected to turn on my camera, so none of their non-interest was recorded.

I let the goats settle back into the comfortable familiarity of their stall, then I took them out again individually for another leading session.  The main focus of the session was on treat delivery and their behavior around food.  I was continuing with the work I described in the previous two goat diary posts. (https://theclickercenterblog.com/2018/01/29/  and https://theclickercenterblog.com/2018/02/02/

A Panda Story

This focus on treat delivery and the time it takes to establish good manners was reminding me of Panda’s early training.  Cold winter days are a good time for stories, so I’m going to indulge in a couple, beginning with a favorite Panda story.  Panda is the miniature horse I trained to be Ann’s guide.  I remember when I first started working with Panda, she was as eager as the goats were to get into my pockets.

A week into her training – at about the stage I was now with the goats – I took Panda with me to a clinic I was giving at a barn that was about an hour away.  Ann rode in the front seat with her new guide dog curled between her feet.  Another of my clients was driving.  I was in the backseat with Panda essentially in my lap.  I was definitely a captive audience.  Doing a short session and then putting her back in her stall to process was an impossibility.  I had an hour’s drive with a horse in my lap!  What’s more I had a horse who knew I had treats in my pocket.

For the duration of the drive I clicked and treated anytime Panda’s nose moved even fractionally away from my pocket.  The idea was to keep her on such a high rate of reinforcement that she didn’t have a chance to mug me.  Over and over again, through the food placement I was saying to her – this is where the treats are delivered.  Going to my pocket gains you nothing.  Out here away from me, this is where you will find treats.  You might as well keep your nose here and not waste your energy going to my pockets.

Ann was in the front seat listening to the constant barrage of clicks.  I know they were making her anxious.  She had only recently taken on a new guide dog.  Everything about this dog was a struggle.  He should never have been placed.  The school was hoping that because Ann was such an experienced guide dog user, she would be able to make him work.

“Make the dog work” was truly the philosophy behind this dog’s training.  The result was a dog who showed extreme avoidance behavior.  Ann had one problem animal.  She didn’t want another.  How could she have a guide who needed to be clicked and treated every couple of seconds?  Ann knows how training works.  She knows that we would be building duration, but in that stage where the mugging is still such a strong reaction, the future good manners can seem impossibly far away.

Good manners emerge over time.  They are the result of consistent handling and a growing confidence in the learner.  By the time I handed Panda over to Ann, the guide dog had gone back to the school to be re-trained for a different job.  He went into search and rescue work, a job that suited his temperament much better.  And Panda became Ann’s full time guide much sooner than we had originally planned.

We celebrated the transfer by going out to dinner.  Panda kept her nose to herself and stayed quietly by Ann’s side throughout the evening.  Even when the salad course arrived, all she did was have a curious sniff before ducking her head back under the table to continue her nap. That’s great duration in a behavior that had begun with barely seconds between clicks.

Good manners emerged for Panda, and I was confident that they would also become the norm for the goats.  Time and consistency would create the behavior I wanted.

p46_PandaInRestaurantWithTrainerAlexandraKurlandOwnerAnnEdieNeilSoderstrom 343

Dining out with Panda

(If you want to learn more about Panda and her training, read the Panda Reports on my web site: theclickercenter.com.  Some of her early training is also featured in my DVDs: An Introduction to Clicker Training and Lesson 4: Stimulus Control.

Treats: Whatever Is Logical Do The Opposite

At some point in the distant future, it might be fun to travel with the goats in my car.  But at this point the thought of spending an hour trapped in the backseat of a car with an eager, greedy goat sounded exhausting.  We had a long way to go before they would be as settled about treats as Panda.

You meet your learner where he is not where you want him to be.  When I took P back into the arena, the session was very much focused around food delivery.  The children in the 4-H program may have giggled and let him snatch pretzels from their mouths.  With me P was learning that we played a very different game.

I brought P back out on a lead.  He continued to show good progress. He backed away from my closed hand.  He did a bit of head flinging which means he was feeling frustrated by having to back up.  I’m sure it did conflict with how he thought things should be done.  He wanted to push forward to get to the treats.  That’s what he had always done, but now he had to remember to back up instead.

Whatever is logical, do the opposite.  I could sympathize with his frustration.  From his point of view it made no sense that backing should work.  Going forward was how you get children to spill treats all over the ground.  Why should backing work?!!  We have all been given directions that make no sense.  Why should turning left instead of right get us to our destination?

And how many of us turn right because we’re convinced that should be the correct answer.  Even when we do turn left, it feels wrong.  Surely we’re heading in the wrong direction.  This can’t be right.  We’ll never get there.  Oh look, there’s our destination just ahead. How did that happen!?

It can take a while to relax and trust the directions.  That’s the stage I was in with P.  With a little more reinforcement history behind us, he would relax into the confidence that treats were coming.  There was no need to rush to get them.

The Goat Palace written Dec. 27 – Our Animals Always Tell Us

Meeting your learner where he is, not where you want him to be makes me want to share this story.  It was prompted by the goat’s current training.  If E and P’s treat taking manners were reminding me of Panda, a session I did with Trixie and Thanzi at the end of December made me think of Robin.  There are several of training mantras that apply to this session:

Our animals will always tell us what they need to work on next.

You get what you reinforce. 

My favorite, though, is this one:

If you don’t notice a little resistance, don’t worry about it.  It will get bigger.  And eventually, it will get big enough that you will do something about it.

Before I describe the goat’s training, here is Robin’s story:

Over the winter when Robin was still very new to clicker training, he started to snatch his treat from my hand.  I’d click, he’d grab, and then he’d eagerly be offering me the next clickable behavior.  I ignored the snatching.  He was eager.  It was cold.  He was offering lots of great work.

The snatching increased.  You get what you reinforce.  I didn’t like the snatching, but if it was getting worse, something in our interactions was reinforcing it.

I ignored it.  Robin was eager.  It was cold.  We were having fun – until I wasn’t.  The snatching was becoming more than annoying.  I was starting to count fingers after I gave him a treat.  It was time to do something about the way he took treats.

If you don’t notice a little resistance, don’t worry about it.  It will get bigger.  And eventually, it will get big enough that you will do something about it.

I’ve told the story many times about the way I solved this particular problem.  It’s detailed in both my Riding book and The Step-By-Step Guide in Pictures.  I went back to basics.  I put Robin in his stall with a stall guard across the door.  I stood across the aisle from him and held out the biggest carrot from a bag of big carrots.  Robin stretched his neck out to try and reach it.

I immediately turned my back, removing the carrot from sight and counted to three.  Then I turned back and held the carrot out again.  Robin stretched out his nose.  I turned my back and counted to three.

I again offered the carrot.  This time Robin hesitated ever so slightly.  I clicked, reached into my pocket and handed him a piece of carrot. I was using negative punishment.  I was taking away something Robin wanted – the carrot – to decrease a behavior I didn’t like – the reaching out towards me to get a treat.

(When an activity decreases – it is being punished, either by adding something unpleasant or by taking away something the individual enjoys (positive punishment and negative punishment – it’s just math adding or subtracting).  When an activity increases, it is being reinforced, either by adding something the individual wants or taking away something he doesn’t like. So again there is positive and negative reinforcement.  When the behavior increases it is being strengthened, i.e. reinforced.  When it decreases, it is being punished.  In both – the positive and negative refer to adding or subtracting, not value judgements.)

I offered the carrot again.  Robin hesitated.  Click, I handed him a piece of carrot from my pocket.  Robin is a super fast learner.  He had the dots connected.  If he drew back away from my hand, he got treats.  I could hold the carrot directly under his nose and instead of snatching it off my hand, he arched his neck and drew up away from it.  Click and treat.

I was enchanted.  He looked like a beautiful dressage horse.  Robin being Robin, he quickly made the connection.  If he arched his neck, click, I would give him a treat.  He wasn’t snatching anymore.  Instead he scooped the carrot slice gently off my hand with his enormous soft lips.

He started to offer what I have since called “the pose”.  When I walked by his stall, Robin would draw himself up and arch his neck.  Click.  I’d pause in my barn chores and give him a piece of carrot.  Through the winter I reinforced him a lot for this behavior.  I might have begun with negative punishment as I tried to stop an unwanted behavior – snatching treats off my hand.  Now I was actively reinforcing him for something I wanted – “the pose”.

I should add that this is not the way I teach the pose today.  It popped out when I was working on something else.  Now that I know this behavior is worth going after, I shape it more directly, most often with the aid of targeting.  And in general, when I find myself reaching towards a negative punishment strategy to solve a problem, I go have a cup of tea instead. I think about what I want and look for reinforcement-based teaching strategies instead.

The “pose” is not the best name that I could have come up with for this behavior.  For many people, a pose is a fixed, rigid, stilted posture.  It’s that awful grimace so many of us have when we’re forced to have our picture taken.

Instead, for me, the pose is a very dynamic behavior.  For Robin it has become a default behavior.  I was the cue.  In the absence of any active cue from me, if Robin posed, I would click and reinforce him.  It meant that if he wanted attention from me, he could get me to engage with him using a behavior I actively liked.

Horses are always doing something.  A horse in a stall has a long laundry list of behaviors to choose from.  Some are behaviors that I like, some are behaviors that I can ignore, and some are behaviors that I never want to see.  The laundry list includes taking a nap, eating hay, having a drink, watching the activities in the barn aisle – all perfectly acceptable and easy to ignore.

A horse could also be fighting with his neighbor, kicking the stall door to get attention, cribbing, raking his teeth up and down the wall, pacing, weaving.  These are behaviors I definitely do not want.  But if I fuss at a horse when I see him engaging in them, I could easily be reinforcing them through my attention.  Think of the small child who bangs the kitchen pots and pans while mother is on the phone.   Even negative attention is attention, and that can be better than no attention at all.

Robin doesn’t have to kick the wall to get me to notice him.  All he has to do is pose.  Click and treat.  I love having behaviors which my horses can use to ask for my attention. They know I will always acknowledge their request for connection.

Think of all the ways people interact with one another:  “Good morning.”  “How are you?” “Never better.”  These quick exchanges connect us.  Think how chilling and unpleasant an environment becomes when these social pleasantries are absent.  We need them to tell us things are okay between us.

Robin says good morning by posing.  I respond with a click and a treat.  All is well between us.  Our social bond is strong and getting stronger with each click and treat.

I reinforced Robin for the pose because he looked pretty.  I wasn’t heading for anything in particular beyond that.  This is what makes training so much fun.  Sometimes the next unexpected piece just pops out.

Here’s what happened to the pose.  One evening I had Robin in the arena.  I was asking him to trot around me on a circle.  He was giving me a nothing of a trot.  He looked like an old plow horse.  There was no energy, no pizzazz, nothing I wanted to reinforce.

Robin was expecting me to click.  He went once around the circle.  Nothing.  The way I tell the story was you could all but see the cartoon bubble appearing above his head.  “I’m not being reinforced.”

He went around again.

“What can I do to get reinforced?”

On the third time around he had the answer: “I know! I’ll try the pose!”

The way Dr. Jesus Rosales-Ruiz tells this story is this: by withholding the click I was putting Robin into an extinction process.  He began to regress back through behaviors that had been successful in the past.  The pose had been highly reinforced, so it was the first thing that he tried.

Whichever version of the story you prefer, Robin had to add energy to the trot in order to get into the pose.  Suddenly, his trot looked as though it belonged on a magazine cover.  He was gorgeous!  I clicked and gave him a treat, all the while gushing over how pretty he was.  I sent him back out around me.  It took him a few strides to find his balance, but he once again added the pose to the trot.  It was just one stride that I was clicking – but wow! What a gorgeous stride it was!  The rest is history.  Robin led the way.  He showed us that we could shape the beautiful, suspended balance of a classical dressage horse just through well timed clicks and treats.

Why am I telling this story? Because this morning’s session with Thanzi and Trixie made me think of Robin and the pose.  It reminded me of the expression:

If you don’t notice a little resistance, don’t worry about it.  It will get bigger.  And eventually, it will get big enough that you will do something about it.

In December I had been trying to work them individually.  We had snow Christmas eve and then the temperatures dropped and the wind rose.  Trixie was nervous about being out in the hallway by herself, so I let Thanzi join her.  Suddenly with two goats I had lots of crowding.  Hmm.  You get what you reinforce.  I knew at night when I was tucking them in, I was in a hurry.  It was cold.  It was late.  I just wanted to get done with the final chores and get back inside where it’s warm.  Had I been letting them crowd me and hurry the treat deliver?  Apparently the answer was yes.

I needed to sort out the crowding so in this session I set two mats out face too face.  Trixie hopped on one, Thanzi on the other.  I stood in the middle with both goats crowding into me begging for treats.  I waited.

“Oh right.  Crowding doesn’t get treats.”  They took their noses away from me.  Click. I reached into my pockets.

They were right back, pushing against my hands.  I got the treats out of my pockets and then drew my hands together.  I stood as though in calm meditation, waiting.  First one then the other took her nose away.  I waited until they were both good, then held out the treats to them.

They got their treats, and then they were right back crowding me, pushing against me with their muzzles.  I waited.  They took their noses away.  Click.  Get the treat.  Wait again with hands held together in quiet meditation.  They both drew away from me.  I held out my hands and let them take the treat.

It only took a couple of repetitions. They were both working so hard to stay away from my pockets.  Click, pause, feed.  They were both so good.

I left them in the hallway while I filled their hay feeders.  I was just finishing up when I looked out into the aisle.  They were standing each on her own platform waiting for me.  How can you resist?  I went out and did another round of paying attention to their good manners.

Your animals always tell you what they need to work on.  I don’t know where this will lead me, but I know it is what they need.  If it makes me think of Robin’s pose, I must be on the right track.

Staying Consistent

It’s easy to be focused and consistent through one training session.  It’s much harder to maintain that consistency over time.  When we transferred Panda full time to Ann, it was actually a relief to hand her over.  I missed her constant presence by my side, but maintaining the level of consistency that is needed for a guide was demanding.  When you can see, you don’t need a guide to tell you that you’ve come to a curb. If I started cutting corners in Panda’s training because I didn’t need all the things I had taught her to do, it would undermine her performance as a guide.  Ann would never be able to enjoy the luxury of seeing the curb that’s in front of her.  She would be relying on Panda to point this out to her.  A horse doesn’t know when it doesn’t count so it always has to count.  I followed that mantra throughout Panda’s training.

The same thing applies to the goats.  The same thing applies to the goats.  If sometimes I let them push into me to get treats, I will never get to the consistent good behavior that I want.  But it’s been cold!  It is so easy to get in a hurry and let standards drop.  So their training has been a bit like a yo yo.  I let things slip in my hurry to get chores done and my gloves back on.  They begin to crowd me, but now I am catching it sooner.  The manners pendulum keeps swinging back and forth.  Over time the cumulative effect shows me that the balance is tipping towards good manners.

Just for Fun!

I told you the story of Robin’s pose.  Here’s one of my favorite videos of Robin.  He was only three when this was filmed.  He had not yet been started under saddle.  So he’d never had a rider on his back, and I had never lunged him in side reins or any other type of mechanical device.  This beautiful balance and cadence had been shaped entirely with the clicker.  You’ll see I am holding two dressage whips.  You can call them anything you want, but they are functioning as targets.  They give him points of reference to balance between.  I know the lighting is not good in this video, but this was a long time ago, and this was the best the video camera could do.  Enjoy!

 

Coming Next: The Goat Diaries Day 10: You Can Never Do One Thing

Please Note: if you are new to the Goat Diaries, these are a series of articles that are best read in order.  The first installment was posted on Oct. 2nd.  I suggest you begin there: https://theclickercenterblog.com/2017/10/02/   Two of the goats I write about originally came for a twelve day stay in July.  The July Goat Diaries track their training during this period.  In November these two goats, plus three others returned.  They will be with me through the winter.  The “Goat Palace” reports track their current training.  I wish to thank Sister Mary Elizabeth from the Community of St. Mary in upstate NY for the generous loan of her beautiful cashmere goats.

 

 

 

by

Goat Diaries – Clicker Training Day 2: These Goats Are Smart!

The goat palace is almost finished.  We were hoping to get it done yesterday afternoon, but we didn’t quite make it.  The three yearlings are feeling very squashed in the stall by the oldest female, Thanzi.  She is making it very clear that they are TO STAY IN YOUR CORNER.  I am glad we decided in our construction to use the entire space the lean-to provided and didn’t just settle for making a small goat pen.  They will have plenty of room to spread out.

So for this morning it is back to July and the Goat Diaries.  I had gotten as far as mid-morning of E and P’s second day of clicker training.

Training Rhythms

Good training begins to have a rhythm to it, especially in these early stages where you are asking for simple behaviors, and you’re keeping the rates of reinforcement high. It’s get the behavior – click and feed, get the behavior – click and feed, – get the behavior, click and feed. It becomes a training loop. We’re looking for clean loops.

When a loop is clean you get to move on, and not only do you get to move on you should move on. That’s the mantra of loopy training. Often people change criteria too fast which ends up confusing the learners. Or they stay too long at one step so they build a glass ceiling into their training.  To the learner backing up means three steps and only three steps. If the handler asks for four, there’s frustration. The learner knows the behavior. It’s three steps and three steps only!

The mantra of loopy training helps you to know when to move on. It also helps you to know when you should pause for a moment to let your learner show you what he has learned. Canine trainer, Kay Laurence refers to these pauses as puzzle moments.

In these early sessions with these goats I was beginning to establish some training loops. P in particular was such a fast learner, it was time to give him some puzzle moments to see what dots he was connecting.  If you aren’t sure what a puzzle moment looks like, P is about to show you.

Session 3: 11 am
I started with P out in the pen. He was ready, eager to touch a target, but my attention was elsewhere.  I was busy setting up the camera. I was very aware that I might be missing a window of opportunity. We began with a little targeting. He oriented to it, I clicked, fed, and then clicked and fed again while he was still out of my space. The jumping up on me to try to get the food that he had been doing in the previous session was almost completely gone.  My active use of food delivery was paying off.

Click for targeting. Feed where the perfect goat would be. The perfect goat would have all four feet on the ground. He would be looking straight ahead, and he would be outside my personal space.

After I clicked, I fed P so he had to take a step or two back to get the food. My concern here was the food delivery caused him to curl his neck so his head was in the orientation it would be for butting with his horns. I didn’t want to trigger that behavior. But head butting is a forward moving behavior. Here he was moving back, so I hoped that his feet would keep his head from thinking he should be charging me.

Get them while they’re standing still.

I fed P so he had to back up a couple of steps to get to the treat in my hand. Before he could come forward again, click, I was giving him a treat – this time where he was standing. I wanted him to get the idea. Standing still, away from me, is a good thing. Click treat, click treat. I was tightening the training loop down to the tiny fraction of a second in which he was standing still looking straight ahead.

The neighbors were mowing the hill up above the barn. P kept turning his head to the side to check them out. His feet were still, but I didn’t want to make such a full head turn part of the behavior. I had to wait, hoping his feet would be still when he finally looked back in my direction. Click then treat.

When I clicked, I used my food delivery to move him back a couple of steps. I wanted to be able to click again while he was still standing back out of my space. I also wanted his head to be straight. If I clicked too many times when his head was turned, I was concerned that I would build that into the base behavior. So I had to wait to click until his feet were still AND he had his head straight. Asking for two criteria at once was pushing my luck. The first couple of times he was too quick for me. He straightened his head, but just as I began to click, he was shifting forward.

I moved him back again with the food delivery. He took his treat from my hand.  Before I could click again, he had come forward into my space.

I work hard to avoid putting my learners into a macro extinction process.  Here’s what that means: This behavior has been consistently working to get me to hand you treats. Only now suddenly, it’s not. You’re not going to be reinforced for this very successful behavior.

We all know how frustrating this can be. You put your money in the vending machine and nothing comes out. Time to shake the vending machine!

My training rhythm was broken and P didn’t yet have enough experience in the game to know what to do. His repertoire of behaviors was still too limited to offer me something I could reinforce. Instead he was trying to go directly to my pockets. I suspect by this point the small children he had grown up with would have dropped pretzels and peanuts all over the floor and everyone would be happy. The children would be giggling, and P would be gobbling up the goodies. Only this wasn’t how I played the game. How annoying!

P gave a little chuff of a sneeze. I had llamas years ago, so I recognized this sound as a sign of frustration. He tried both my pockets. Nothing. He gave a head toss which I dodged, and then I got lucky. He dropped his head away from me enough so that I could reinforce him. The food delivery moved him out of my space, and we were back on track building good behavior.

Goats day 2 what frustration looks like 4 photos.png

 

Goats Day 2 back up to get clicked 3 photos.png

Training is not without moments of frustration. I was beginning to recognize what this looked like in a goat. A little tail wiggle, a snort, a head butting gesture – these all told me that P was struggling a bit to make sense of what was happening. Why wasn’t I just giving him treats! That’s what the children would have done. And if they didn’t give him treats, he’d just jump up on them, and that was sure to make them scatter their peanuts and pretzels on the ground!

But here this was different. He was clearly frustrated. Doing what had always worked in the past, namely crowding into me didn’t work. Looking away, taking a step back, produced treats!  It made no sense to him, so while it produced treats it also produced a puzzled goat.  And a puzzled goat can very quickly become a frustrated goat.  Noted.

I was monitoring carefully. Always I am asking myself is this working? Is this the best strategy? How much frustration is too much? What should I change? Should I stop?

Puzzle solving!

There is a time to be clicking, and a time to just wait it out and let your learner work out the puzzle. Through the food delivery, I had shown P the answer. Back away and you get treats. Would he put the pieces of the puzzle together? I waited. The skill here is to be quiet, to remain as non-reactive as you can be and let him figure out the answer. A puzzle you solve for yourself, is an answer you will own.

He could sniff at my pockets. I remained non-reactive. How frustrating! I was not playing the game fair. The children would have been flailing their arms about and pushing him away. Which meant they would also have been dropping treats. Push on the vending machine, and it scatters goodies over the ground, except not now.

His feet took him back a couple of steps. Click – treat. The next time the backing was even more definite.

He caught on fast and began to back away from me. When he came forward into my space, now I could wait. It was a puzzle moment. What would he do? I had shown him the answer through the food delivery. Would he find it now on his own?

The answer was yes! He backed up, not just a little, but multiple steps. And he backed with energy. Very neat!

Goats day 2 Quick study 5 photos.png

P was definitely a quick study. He was beginning to understand that he could get the food by doing other things besides jumping up or bumping my pockets. It was a really fun session watching him catch on so fast. Though I got the impression that he was still very confused. Backing was clearly working, but it didn’t make sense to him. How could backing up get treats to appear? He was a very puzzled goat.

I sympathized. We’ve all been given sets of instructions that make no sense. Whatever is logical – do the opposite. How maddening is that! Especially when it works!

I would find out in the next session if P could reconcile himself to this new inside-out world order.

(Note: we had moved on in the treats. I was now using a mix of peanuts, peanut hulls, sunflower seeds and hay stretcher pellets as treats.)

Training time for this session: 6 minutes.

Video: Video: Goat Diaries Day 2: A Quick Study: Note you will need a password to watch this video: GoatDiariesDay 2 E Learns
“A puzzle solved is a behavior owned.” P showed me he was making the connections – fast!

Video: GOAT DIARIES/Day 2/Problem Solving: Note you will need a password to watch this video: GoatDiariesDay 2 E Learns

 

Coming next: Day 2 Continued – Two Different Learners

by

Summer Pleasures – Watermelon Parties and The Two Sides of Freedom

Watermelon Parties

watermelon

Summer means watermelon parties for the horses.  They are always a surprise.  As I walk through the barn, bowl in hand, I’ll announce: “It’s party time!”

Watermelon parties are held outside. That was quick learning on my part. It’s astounding the amount of happy drool even a few pieces of watermelon can produce.

Robin and Fengur follow me outside.  While I pass out chunks of watermelon, they stand waiting, one on either side of me.  There’s no pushing, no trying to jump the queue, no grumbling at the other horse. We have a happy time together. The horses get to enjoy one of their favorite treats, and I get to enjoy their obvious pleasure.

Summer also means sharing an afternoon nap with Robin. I’ve just come in from mowing the lower pasture. It’s time for a cool down. I’m sitting in a chair in the barn aisle, cold drink by my side, computer on my lap, and Robin dozing beside me. Fengur has wandered off to the hay box to snack. He’ll join us in a little while.

Robin asleep lip drooping

The view from my chair – Robin’s lower lip droops while he naps beside me.

Why am I writing about these simple summer pleasures? My horses live in a world of yes. I’ve been thinking a lot lately about what this means. Living in a world of yes gives me the freedom to enjoy these simple pleasures. But the freedom isn’t one-sided. Living in a world of yes gives my horses just as much freedom.

We often think of training in terms of what we need from our animals. When I walk down the barn aisle, I need you, horse, to move out of my space. When the door bell rings, I need you, dog, to go sit on your mat. I’ll teach these things using clicks and treats, but the behaviors are for my benefit more than my animal companions. The freedom to ask is all on my side.

That’s not how things are in my barn. It’s set up to maximize choice for the horses. Doors are left open so they are free to go where they want. Right now what Robin wants is to nap in the barn aisle. I couldn’t give Robin this luxury of choice if I hadn’t also given him behaviors that let us share space amiably.

When I walk down the barn aisle, Robin will often pose. It’s a simple gesture, a slight arch of the neck is all that’s needed. If he thinks I’m not paying attention, he’ll give a low rumble of a nicker. I’ll click, and give him a treat. Often I’ll get a hug in return.  That’s good reinforcement for me.

The pose is a guaranteed way to get attention from me. If Robin wants to interact, he knows how to cue me. And I am under excellent stimulus control! That’s how cues should work. They create a give and take, a back and forth dialog. They erase hierarchy and create instead the three C’s of clicker training. Those three C’s lead in turn to the freedom my horses and I enjoy sharing the barn together.

Before I can tell you what the three C’s are, we have to go back a few steps to commands.  It’s not just in horse training that commands rule. They control most of our interactions from early childhood on.  Commands have a “do it or else” threat backing them up. Parents tell children what to do.  In school it is obey your teachers or face the penalties. In our communities it’s stop at red lights or get a ticket. Pay your taxes or go to jail. We all know the underlying threat is there. Stay within the rules and stay safe. Stray too far over the line and you risk punishment.

This is how we govern ourselves, so it is little wonder that it is also how we interact with our animals. With both horses and dogs – commands have been the norm. We tell our dogs to “sit”. When it is a true command, it is expected that the dog will obey – or else! The command is hierarchical which means it is also unidirectional. A sergeant gives a command to a private. The private does what he’s told.  He doesn’t turn things around give a command back to the sergeant.

We give commands to our horses, to our dogs – never the reverse. We expect our commands to be obeyed. We say “sit”, and the dog sits. I tell. You obey. Because they are hierarchical, commands exclude dialog. The conversation is all one-sided. Commands put us in a frame that keeps us from seeing deep into the intelligence and personality of the individual we’re directing.

Cues are different. Cues are taught with positive reinforcement. At first, this sounds like a huge difference, but for many handlers it represents a change in procedure, but not yet of mind set. The handler may be using treats as reinforcement, but the cues are still taught with an element of coercion.  How can this be? It’s not until you scratch below the surface, that you’ll begin to understand the ever widening gulf that the use of cues versus commands creates.

dog touching a targetTo help you see the coercive element, let’s look at how twenty plus years ago we were originally instructed how to teach cues.  You used your shaping skills to get a behavior to happen. It might be something as simple as touching a target. Cues evolve out of the shaping process. The appearance of the target quickly becomes the cue to orient to it.  But this cue is often not fully recognized by a novice handler.  We’re such a verbal species, this handler wants her animal to wait until she says “touch”.  As she understand it, that’s the cue.  So what does she do? She begins by saying “touch” and clicking and reinforcing her learner for orienting to the target.

This part is easy. Whether she had said anything or not, her learner was going to touch the target. She’s ready to make a discrimination. Now she presents the target, but she says nothing. What does her learner do? He orients to the target, just as he’s been doing in all the previous trials. He expects to hear the click and be given a treat, but nothing happens. His person just changed the rules which has plunged him into a frustrating puzzle.

He’s in an extinction process. He’s no longer being reinforced for a behavior that has worked for him in the past. He’ll go through the normal trajectory of an extinction process. That means he’ll try harder. He’ll try behaviors that worked in the past, and he’ll become frustrated, anxious, even angry, before he’ll give up for a moment. In that moment of giving up, his person will say “touch” and present the target again.

She wants him to learn the distinction. In the presence of the cue perform the behavior – click and treat. In the absence do nothing.

The problem with this approach is she never taught her learner what “do nothing” looks like. She stepped from the world of commands into what she thinks of as a kinder world of cues, but she didn’t entirely shed the mantle of “do it or else”. With cues the threat of punishment may not be there, but extinction is still an unpleasant and frustrating experience. Why isn’t this key on my computer which was just working now locked up and frozen?!! Until you can find your way out of the puzzle, you can feel very trapped and helpless. A good trainer doesn’t leave her learner there very long. She’s looking for any hesitation that let’s her explain to her learner the on-off nature of cues.

There’s another way to teach this that doesn’t put the learner into this extinction bind.  This other way recognizes that cues create a dialog, a back and forth conversation.  I want my learner to wait for a specific signal before moving towards the target.  Let’s begin by creating a base behavior, a starting point.  For my horses this is the behavior I refer to as: “the grown-ups are talking please don’t interrupt”.  I will reinforce my horse for standing beside me with his head looking forward.  He’ll earn lots of clicks and treats for this behavior.  And he’ll begin to associate a very specific stance that I’m in with this behavior.  When I am standing with my hands folded in front of me, it’s a good bet to try looking straight ahead – click and treat.

Ruth Scilla grown ups.png

“Grown-ups”

In separate sessions he’ll also be reinforced for orienting to a target.  When both behaviors are well established, I’ll combine them.  Now I’ll look for grown-ups.  I’ll fold my hands in front of me, knowing I’ll get the response I’m looking for.  Only now, instead of clicking and reinforcing him, I’ll hold out the target to touch.  Click the quick response and treat.

The message is so much more interesting than the one created by using an extinction procedure to introduce cues.  Cues have just become reinforcers which means they have become part of a conversation.  If you want to interact with the target, here’s an easy way to get me to produce it – just shift into grown-ups.  That will cue me to lift the target up.  A conversation has begun.  We’re at the very elementary stage of “See spot run”.  I’m teaching my horses the behaviors they can use to communicate with me, and I am showing them how the process works.  You can be heard.  You WILL be heard.  Let’s talk!

The conversation that emerges over time comes from looking more deeply at what cues really are. We can think of them as a softer form of commands, but that doesn’t oblige us to step out of our hierarchical mindset. It is still I give a signal. You – my animal companion – respond. Click and treat. Diagram this out. The arrows all point in one direction.

Signal from human leads to response from animal

Peel another layer of understanding about how cues work and you come to this:

It isn’t just that cues are taught with positive reinforcement. Cues can be given by anyone or anything. A curtain going up cues an actor to begin speaking his lines. We would never say the curtain commanded the actor.

If cues can be given by anyone or anything, that means they are not hierarchical. We cue our animals, and they cue us. Cues create a back and forth exchange. They lead to conversation – to a real listening to our animals. We adjust our behavior based on their response. Cues lead to the three C’s of clicker training which I can now say are: communication, choice, and connection. And in my barn that in turn creates opportunities for more freedom. It means doors can be left open. It means I can have watermelon parties and sit with my horses while we both enjoy the afternoon breeze through the barn aisle.

Let’s parse this some more.

The mindset that commands create is very much centered around stopping behavior. Other training options won’t make sense. They won’t work.

Cue-based training makes it easier for you to see your horse’s behavior as communication, as a bid for attention. That makes it easier for you to look for solutions that satisfy his needs.

Let’s see how these differences play out in a typical boarding barn scenario. Your horse is hungry. His initial whicker has been ignored. In frustration he’s escalated into banging on his stall door. His human caretakers see this as “demanding” hay. In a command-based frame demanding hay equal rebellious behavior which can’t be tolerated. The behavior must be stopped.

Within this frame the only training options you can think of are those centered around stopping the unwanted behavior. Other options don’t make sense and won’t work. The command-based frame narrows your field of view. It’s as though you have a tight beam focused on the problem behavior. Everything within that beam is crystal clear, but everything outside the beam might as well not exist. You can’t even begin to think about other solutions. You are targeted on the unwanted behavior.  Banging on the stall door must be addressed and addressed directly.

Now let’s look at the contrast that a cue-based frame creates. Your horse is hungry. His initial whicker to you is noticed and responded to. You appreciate his alerting you to the lack of hay. You have read how important gut fill is in preventing ulcers. You attend to your horse’s needs. Within this frame many options become available including hanging a slow feeder in his stall so he doesn’t have to become anxious about his hay.

Which training options make sense will depend upon which frame you are in. If you are a teacher and you want your instructions to be effective, you need to help your students open a frame that matches what you are trying to teach.

In her presentations Dr. Susan Friedman uses a graphic showing a hierarchy of behavior change procedures beginning with the most positive, least intrusive procedures.

Dr. Susan Friedman's Hierarchy of interventions

You begin by looking at health and nutritional considerations and then move to antecedent arrangements. Hanging a hay net for our hungry horse would fit in here. Her graphic pictures a car moving along a highway. As you begin to approach more invasive procedures, there are speed bumps blocking the way. They are there to slow you down, to make you think about other approaches before you bring in the heavy guns of positive punishment. The hierarchy doesn’t exclude positive punishment as a possible solution, but it does say you would use this only when everything else has first been tried.

This hierarchy makes sense when you are looking at behavior from a cue-based perspective. From a command-based frame, the car enters not at the bottom of the roadway, but at the top.

My Changes To Procedural Changes slide

The first intervention is positive punishment. The barriers are still there, but now they act to keep you from seeing other options. It is only when punishment fails, that you are dragged, kicking and screaming, to consider other ways of changing behavior.  I’ve heard these stories so many times from people who are attending their first clicker training clinic. They’ve been brought there by “that horse” – the one who challenges everything they thought they knew about training. Nothing else worked, but then they tried, as a last resort, a bit of clicker training and everything changed! So here they are, ready to learn more.

They don’t yet know what an exciting world they are entering. Everything they have thought about training is about to be turned truly upside down and inside out. That’s all right. They have the fun of watermelon parties ahead of them.

Live in a World of Yes.png

If you want to learn more about living in a world of yes and the freedom that creates for both you and your animal companions, come join us in Milwaukee for the Training Thoughtfully conference.  https://www.trainingthoughtfullymilwaukee.com/

by

JOY FULL Horses: Understanding Extinction: Part 12

Mastering Micro: Building Unlikely Behaviors with Resurgence
Nothing is either all good or all bad.

We want to use positive reinforcement with our animals because we see it as being both effective and more humane.  But the associations created through positive reinforcement can create addictions to harmful behaviors.  Think about the way advertisers manipulate our behavior to encourage smoking or overeating.

Resurgence and regression can be very negative procedures, but they can also be used to produce what might otherwise be very difficult behaviors to obtain.

If you aren’t sure how you can turn what seems like a negative procedure into a positive teaching strategy, PORTL can once again help to illustrate how this works.

Here’s the set up:

The trainer sets a toy chair on the table for her learner to interact with. The goal is to get the learner to push the chair over the table the way she might push a toy car.

We’ll now observe quietly in the background while the learner begins to interact with the chair.  The trainer could get lucky.  The learner might begin offering the behavior she’s after within the first couple of clicks.  But with this learner there’s no sign of any chair pushing behavior. Why?

History matters.

The learner is going to draw on all of her previous repertoire of things she has done with chairs.  In this case we have a learner who was scolded as a child for pushing her chair over the floor, so she’s not very likely to offer this type of behavior with the toy chair.

A history of punishment has played a role in depressing chair pushing behavior for this learner, but pushing would also have been an unlikely behavior if the trainer had set down a dice. The learner would have tossed the dice or shaken it in her hand because that’s what you do with this kind of object. Pushing a dice over the table like a toy car is not an obvious behavior to try.

Through a series of small approximations, the trainer could try to shaping the behavior she wants.  Her first step would be reinforcing the learner for touching the chair.

The learner in this case is not particularly creative.  She offers simple touches, but nothing else.  Again, the trainer may be dealing with a history of punishment.  Her learner doesn’t have a lot of experience being reinforced for trying things.  In fact, quite the opposite – she may have been punished for stepping “outside the lines”.  She is like so many of our animal learners – hesitant, lacking in confidence, and not showing any outward signs of curiosity.  In her first few attempts she touches the chair, but she doesn’t try any other behaviors.  Getting her to push the chair is going to be hard.

So the trainer takes the chair away and sets out a toy car. Using an object that normally would be pushed makes it very easy to get the desired action.  The learner pushes the car over the table top. Click and treat.

This is repeated several times, and then the trainer takes the car away and sets the chair out.  The learner goes back to touching it.  The chair accidentally falls over – click and treat. The learner latches on to that, expanding her repertoire to two behaviors – touching the chair and knocking it over.

We see this so many times with our animal learners.  One click and suddenly you’ve locked in a behavior you don’t want.  With a creative learner this isn’t a problem.  You can quickly shift the behavior into something you want, but with these “one trick ponies” you have to be so very careful what you click.  In this case the learner persists in knocking the chair over even when she is no longer getting reinforced for the action.

Her trainer makes a quick decision and decides to put everything but pushing the chair like a car on extinction.  Her learner is clearly becoming frustrated.  To avoid having her shut down completely, the trainer takes the chair away and sets the car out again.  The learner immediately starts pushing the car over the table top.  Click and treat.

To help with the generalization the trainer puts a third object out – a small block. The learner pushes the block.  Click and treat.  This is repeated several times, then the trainer takes the block away and sets out the car.  The car is pushed. Click and treat.

The trainer sets the chair out, and the learner pushes the chair.  Job done.

Resurgence and Dog “Yoga”
Using the car in this way is an elegant teaching strategy.  Often when we come up with these clever ways of helping our learner to be successful, we know that it works, but we don’t really have good explanations for why.   Understanding resurgence helps us with the why in this case.  And it helps us to be more deliberate in the use of this kind of teaching strategy.  Here’s another example.

One of Kay Laurence’s students taught her dog to step up with his hind legs onto a chair.  It was elegant training, a beautiful example of setting the learner up for success.  In his talk on extinction, Dr. Jesús Rosales-Ruiz helped us to see that it was also a great example of using resurgence.

Here’s the lesson: First, the dog learned to stand one foot each on four small plastic pods. This alone was impressive training.  The pods were the same ones physiotherapists use to help people improve their balance and proprioception. It took great coordination for the dog to stay balanced on the four pods. But that was only step 1.  Next he learned to keep his front feet on the floor while he maneuvered his hind feet up onto the brick ledge of a fireplace hearth.

Adding in the precision of the pods came next.  Now the dog wasn’t just standing with his front paws on the floor and his hind end up on the ledge.  He was also balancing on all four pods.

This was not done as a cute party trick.  The dog’s owner is a yoga teacher.  Her interest was very much the same as mine – helping her animal learner maintain a healthy spine.  In this orientation she could ask her dog for weight shifts that contribute to a flexible spine.

The last step was setting up a training session next to a chair. The handler withheld the click, putting the dog into an extinction process. With very little experimentation, the dog oriented himself so his hind end was to the chair.  He certainly demonstrated the flexibility of his spine by stepping up onto the chair with his hind legs so he was standing hind end up on the chair and front feet on the floor.

Generalization and Creativity
Jesús commented that if we didn’t know about resurgence we would simply be saying the dog generalized.  That’s not a sufficient explanation.  What we were seeing was a great example of resurgence. PORTL has given us a better understanding of how to encourage this kind of problem solving.  When we want to train for this type of generalization, knowing about the “why” of resurgence helps us to be more deliberate and efficient in our training.

It isn’t positive reinforcement by itself that creates a positive learning experience.  An eagerness for learning comes from being a successful puzzle solver.  That success in turn comes from the kind of efficient, clean training that the clever use of resurgence encourages.

These examples give us a great perspective on creativity.  When we’re training, we aren’t waiting and waiting for our animals to do something we can reinforce.  Instead we can “seed” the behaviors we want them to draw on.  Then we set up the conditions and let them have the pleasure of discovering for themselves new or unlikely combinations.

We have a procedure for setting up the creative process.  You give your learner the repertoire, the components that form more complex behaviors, and then you set a puzzle and let extinction be the catalyst for solving it.

Coming Next: The “Pose”

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY FULL Horses: Understanding Extinction: Part 9

Eureka Moments: What is Insight?

Using resurgence – Insight
Yesterday I shared several PORTL games developed by Dr. Jesús Rosales-Ruiz.   The games deliberately used extinction.  What was observed was this: when you have been consistently reinforcing behaviors as you establish them in repertoire, and you then remove all reinforcement for them, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

When you instead extinguish the individual behaviors during the teaching phase, you get a different result.  The student will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

So how does this help us?  How can we use this understanding to shape behavior?  To get the ideas rolling Jesús shared several video examples where resurgence was used to train complex, creative behaviors.

The first video came from Robert Epstein’s work. Epstein was B.F. Skinner’s last graduate student.  Together they were exploring the concept of “insight”.  How do we solve puzzles?  Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

Bird Brains
To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into a chamber with the box and the banana.  The banana was hung up out of reach.  The pigeon couldn’t peck the banana, so an extinction process began. There was a resurgence of previously trained behaviors.  The pigeon was able to push the box under the banana, get up on the box, and peck the banana.

How did the pigeon solve this puzzle so quickly?  What is insight? What really is creativity?  Skinner and Epstein would say the pigeon could solve the problem because it had in its existing repertoire the necessary components.  Pigeons that had no experience pushing the box or jumping up on the box failed to solve the puzzle.

What is Creativity?
Jesús gives us a very process-oriented way thinking about this experiment.  This kind of complex puzzle solving was achieved through resurgence.  Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, the individual will struggle to solve the puzzle.  He will experience a much longer extinction process.  Macro extinction emotions will begin to surface, and you have to hope the subject has the persistence to become truly creative.

This is the kind of creativity that is truly stressful.  It’s much better to analyze the end goal – the complex behavior you want to train – break it down into all of it’s component tasks, and then train each of the components separately.  The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Persistence
Jesús’ comment was there is “nothing new under the sun”. The behaviors you try are all built out of things you’ve done before.  All the components of what appears to be a novel behavior have been trained in the past. So let’s consider what happens when a group of people are presented with a challenging puzzle.  When they begin experimenting and find that the usual, familiar things aren’t working, some will give up quickly.

Others will persist.  They will experiment with novel combinations of what they already know, but again most will quit if they don’t come up with a solution fairly quickly .

A few will keep trying until they stumble across a novel combination that works.  We call these people inventors and creators because they are persistent enough to find these novel combinations.  The discovery process can be a painful one, but once the new combination has been found, it’s easy for everyone else to copy the results.

I can absolutely relate to this.  Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off.  But put me in front of a computer that isn’t cooperating, and I shut down fast. There my experience has produced a different set of expectations. I’ve been in enough situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills.  I don’t have the programing background that makes wrestling with a software issue fun.  Extinction has gone too far and been too uncomfortable.  So in one situation I can be very persistent and creative.  In another I’m the one going through the classic cycle of emotions that macro extinction produces.

I know first hand both how much fun the creative process can be when the expectation of success is there.  And I also know how painful and unpleasant the extinction process is when that expectation is missing.

What I want to create for my learners is a feeling of confidence.  Whether horse or human, I want them to KNOW they can solve whatever training puzzle I throw at them. Build this expectation in early before others have taught them hard lessons about failure, and you get brilliant, enthusiastic, joyful individuals.  They are the optimists of this world.  Whether horse or human, they are fun to be around.  That’s what an understanding of these concepts helps us to create.

Coming Next: Degrees of Freedom

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY Full Horses: Understanding Extinction: Part 8

Mastering Extinction
Extinction happens all the time.  When you withhold your click, you set up an extinction process.

If you are unclear about your criteria or clumsy in your handling skills, you could be setting up your learner for a macro extinction process with all of the painful emotions that go along with it.

Or you could be using a micro extinction strategy to help shape a more complex behavior.  In this case you are using extinction to your advantage.  Extinction doesn’t have to be something you avoid.  It can be something you actively use to create more complex behavior patterns.

In yesterday’s post I described the PORTL games that Dr. Jesús Rosales-Ruiz  uses to help his students understand principles of behavior.  In his talks he shares some fascinating PORTL experiments to illustrate the difference between resurgence and regression.

Experiment One: Resurgence
The learner was taught a series of behaviors:

Behavior 1: tapping a small block. Once that behavior was confirmed, the block was removed and a toy car was placed on the table.

Behavior 2 was rolling the toy car over the table top.  When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it.  Pushing a car is an easy guess for what you would do with this kind of object.

When that behavior appeared to be solid, the car was removed and a third object, a key, was placed on the table.  Now the behavior was lifting.  Fingering a key is a normal response to this kind of object so it was easy to get the learner first to touch the key and then to lift it up off the table.  Once the learner was consistently lifting the key, that object was removed and a fourth one was introduced.

Behavior 4 involved the learner putting a wooden ring on her finger.  The learner quickly figured this out and began to consistently offer this behavior.

When each of these behaviors seemed solid – tapping the block, pushing the car, lifting the key, putting a ring on her finger – the trainer reviewed, one at a time, what the learner was to do with each of the objects.

The trainer then placed all four objects out on the table, but not in the order in which they had been taught.  The trainer observed the learner’s behavior.  She did not give any feedback or reinforcement of any kind.  The point was to see in what order the learner would interact with each object.

The result:  The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.

So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.

The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them, and then you remove reinforcement, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

Now here’s the fun part.  When you instead extinguish the individual behaviors, you get the opposite result.  Now you see regression.  The individual will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back – thus revealing his training history.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

These differences are illustrated in the second experiment.

Experiment Two: Regression
After a series of behaviors have been learned, this experiment again puts the learner through an extinction process.  In the initial set up each time the learner is moved on to a new task, an extinction process is used to eliminate the previous behavior.  Here’s the experiment:

The trainer sets out one item on the table.  The learner begins to manipulate it, trying to find out what is going to be clickable.  The trainer doesn’t click any of this creativity. She waits instead for it to extinguish and then clicks for one simple behavior – touching the object with one finger. That is the “hot” action.

The trainer clicks and reinforces for successful approximations until she has achieved a high degree of consistency in touching the object with one finger.

This was the set up for the experiment.  In the next phase she sets ten different objects out in a circle, including the one they had just been working with.  The learner begins by touching the familiar object.  That gets clicked and reinforced several times, then the trainer stops reinforcing for that object.  She is using extinction to eliminate that behavior.  The learner begins by experimenting, touching various objects, but she only gets clicked for touching the one that was immediately next to the previously hot object in a counter clockwise direction.

The learner switches over to this object and begins touching it consistently.

So now the handler stops reinforcing for this object and only reinforces for the next object on the circle.  The learner again experiments and then discovers that the only object that she gets paid for touching is the third one on the circle.

When this is consistent, the handler again stops reinforcing for touching this object.  The learner is catching on to the overall pattern. Now she moves more quickly to the fourth object and discovers that is the “hot” one to touch.

They continue counter clockwise around the circle until every object has been the “hot” one once and touching it has also been extinguished.

At this point the handler stops reinforcing altogether and simply observes the learner’s behavior.  The result: the learner quickly switches to moving clockwise around the circle, touching the objects in the reverse order in which she learned them.  So she learned them originally counter clockwise: object 1, then object 2, then object 3, then object 4, etc.

Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc.  She isn’t getting clicked for any of these touches, but the pattern is very persistent.

So again: in the first experiment where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.

In the second experiment where behaviors were extinguished, the learner went through them in the reverse order.

You won’t find these distinctions in the scientific literature. These two extinction outcomes, resurgence versus regression, are something Jesús and his students have been revealing by playing PORTL games.

Mind Games
Again Play is the key here.  PORTL may have a serious purpose behind it, but these are games.  All the creativity that comes with play is woven into these experiments.  It may turn out that others playing with similar set ups will have different results.  That’s a good thing.  That simply raises more questions, more puzzles to solve.

Do you have a question about how something works? Great. Design an experiment, test it a few times to work out the kinks in the procedure, and then invite your friends over for a pizza and PORTL party.  In the course of an evening you could have enough data to write a paper!

I do like the new twist Jesús has given to this version of the training game.  As he has pointed out, we’ve been using lab rats to learn about human behavior.  Now we are using humans to model animal behavior. Turnabout is fair play.  Much better to frustrate an undergrad than some poor lab rat!

Coming Next: Eureka Moments!  What is Insight?

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY FULL Horses: Understanding Extinction Part 7

The Training Game
I’ve mentioned training games several times.  The original clicker training game was a close cousin to the children’s game “Hot and Cold”.  The learner was sent out of ear shot while the rest of the group chose a goal behavior.  When the learner returned, the only instructions she was given were to offer behavior.  If she did something that her designated trainer liked, she would be clicked. She was then to go to her handler for a treat.

I’ve seen situations where the learner got the behavior seamlessly.  One easy click after another led the learner directly to the goal behavior.  I’ve seen other situations where the same behavior tripped people up completely.

When we train our animals, we want the first scenario – seamless, successful training.  That’s what we want for our equine learners.  But in the training game, we often learn the most when we experience clumsy shaping.  It can be frustrating to struggle through a session that lacks a clear training plan, but you do gain a great appreciation for what NOT to do.

Genabacab
Kay Laurence developed a different style of training game.  In this one trainer and learner are seated opposite one another at a table.  Instead of acting out the behavior like a game of charades, the learner manipulates objects which the trainer has set out on the table.

alex-genabacab-with-caption

Kay always has great fun collecting objects for the table game.  She has small plastic fruits and cakes, toy cars, small cones, plastic insects of various varieties.  It’s a colourful mixture that she hands over to her trainers.  When I play the table game at clinics, I raid the host’s kitchen junk drawer.  My toys aren’t as much fun as Kay’s, but they serve the purpose just as well.

Kay calls her game Genabacab.  It has very few instructions and really only one rule: the only person who is allowed to talk is the learner. The trainer and spectators are not to give any verbal hints or to discuss what is going on until afterwards.

The table game lets you work out shaping plans BEFORE you go to your animal.  Do you want to learn how to attach a cue to a behavior and then change that cue to a new cue? You can work out the process playing the table game and spare your animals the frustration of your learning curve.

Kay has described workshops at her training center where someone arrives with a “how do I teach this?” type of question.  Maybe the handler wants to teach match to sample, or she wants to see if her dog can indicate which object is bigger or smaller.  Instead of going straight out to the dog and confusing it with missteps and false starts, everyone in the group will pull out their Genabacab games. Kay says people will often spend half the day happily absorbed in developing the best teaching strategies for their dogs.  The dogs spend the day relaxing while their people work away at the puzzle.  It’s only once the process is well understood, that the dogs are brought in for training.

PORTL
Dr. Jesús Rosales-Ruiz and his students at the University of North Texas have been using Genabacab to understand basic principles of behavior.  He wants to bring the game to the scientific community as a research tool, so he gave his version a new name:  PORTL – Portable Operant Research and Teaching Laboratory.   Kay still has her Genabacab for teaching her canine handlers and Jesús has PORTL for teaching behavior analysis.  On the surface they are similar games, but they serve different functions.

Animal studies are increasingly difficult to do because of ethical concerns and expense.  PORTL offers an alternative for research.  You can have a question about how a particular process works, design an experiment using the PORTL game, and in hour’s time have gathered enough data to write a paper – all without frustrating a single lab rat. Now that’s progress!

His students meet on a regular basis to play PORTL games. When they turned their attention to the extinction process, they made some interesting discoveries.

In one game, the learner was shaped to place one hand over the other – right hand over left, and then to reverse it – left hand over right.  The behavior was put on a fixed ratio of 5, meaning the learner was clicked and reinforced on every fifth swap of hands.

The second task was tapping a block.  Again the learner was put on a fixed ratio of 5. (The learner was to tap the block five times for each click and treat.)

The trainer then increased the ratio for the tapping to 30. The learner began to tap the block, but now there was no click and treat after 5 taps.  The learner kept going to about 13 taps.  At that point she began to experiment.  She reverted back to swapping hands.  Then she tried a few more taps, before going back to hand swaps.  She tapped the block a few more times.  The trainer was still keeping track so each of these taps was counting towards the count of 30 she was looking for.

In the twenties the learner began to be creative.  She tried different ways to move hand over hand.  She’d go back and forth between experimenting with hand swaps and tapping the block.  Finally she reached a count of 30 at which point her handler clicked and reinforced her.  All the extra gunk was also chained in.  Now as the handler kept reinforcing the tapping of the block, the frequency of the hand swapping also skyrocketed.  That behavior was no longer being intentionally reinforced, but it increased right along with the tapping.

Now you may be thinking:  “Well that’s just poor training.  No one is going to jump from a fixed ratio of 5 to one of 30.” My response would be to say that this can happen inadvertently.

Suppose a handler has had a behavior on a high rate of reinforcement. The horse is responding on a consistent basis, but then he’s distracted. He’s no longer offering the same consistent response.  Instead the handler is seeing a string of unwanted behaviors.  Sometimes the horse almost meets criterion, but not enough to click. And then he comes through with the right answer.  The handler captures that moment with a click and a treat.  The question is: what is the long term result of that click? Has the handler just identified a single clickable moment or has she chained in a long string of “junk” behavior?

The horse’s future responses will answer that particular question, but Jesús’ response in general is: if you want clean behavior, you need to train in clean loops.  Kay and I would add that you need to microshape.  You need to learn to set up your training so the behavior you want is the behavior you get.

Here’s a link to a great youtube video of  a PORTL game presented by Mary Hunter.   Many of you will know Mary from her StaleCheerios.com blogs. Mary is president of The Art and Science of Animal Training, the organization that puts on the annual conference of that same name in Dallas TX.  She and Jesús will be presenting a program on PORTL at this year’s clicker Expos.

Coming Next: Mastering Extinction

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY FULL Horses: Understanding Extinction: Part 6

Cues and Extinction
In Part 2 of the JOY FULL horses posts I wrote at length about cues.  We went through the list of ten things you should know about cues.  That list took us from the basics of cues to some very elegant training concepts.  Cues also play a role in this discussion of extinction.  They have a lot to do with reducing the emotional effect of extinction.

Cues can tell an animal whether or not you’re engaged with him in training.  If your cues say “not now”, he knows he can go take a nap. Kay Laurence has very clear protocols for her training classes. If someone with a dog has a question for her, the handler is first to park the dog.  Parking means the handler anchors the dog to one spot by standing on the leash.  With her hands off the leash, she can now switch her attention away from her dog to Kay.  The dog quickly learns that a parked leash means he doesn’t need to watch his handler closely.  He can take a break from the training conversation.

Teaching “Chill”
With our horses we often forget to put this piece in.  We are usually training by ourselves.  The time in the barn is our time to relax and be with our horses.  It’s only when someone comes to visit that we discover the grown-ups really can’t talk.  Your horse wants to be part of the conversation, as well!  If you abruptly ignore him, that’s when you can get macro extinctions with all of the associated problems. The solution is to teach an equine version of “park”.

The bigger lesson is to become more aware of your body language and the attention your animal is giving to it.  If you see him surfing for answers, intercept the process.  Reset the conversation.  Turn it into a teaching opportunity that gives your learner a clearer idea of what is wanted so you can both avoid the frustration of macro extinctions.

Coming Next: Training Games

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY FULL HORSES: Understanding Extinction – Part 5

Using “Hot” Behaviors

The Measure of Success
When horses are engaged in a successful shaping session, it can seem as though they never stop eating.  If you aren’t familiar with clicker training it can look as though the handler is constantly clicking and treating.  Don’t they ever stop feeding?  How is this going to work?  How do you raise criteria if you’re always feeding?

In a good shaping session the next criterion you’re going to shift to is already occurring a high percentage of the time BEFORE you make it the new standard.  Suppose I’m working on grown-ups, and I’ve decided that I want my horse to have his ears forward.  That’s a great goal, but if I abruptly stop clicking for good head position because the ears are back, guess what I’ll get – more pinned ears.  Why? Because I’m frustrating my horse and that emotion is expressed through pinned ears.

I’ll also get him swinging his head, nudging my arm, pawing etc., all the behaviors that I thought I had extinguished as I was building my grown-ups.

Using “Hot” Behaviors
What is the solution?  I could begin by separating out ears from other criterion. During casual exchanges when we aren’t in a formal training session, every time I see my horse with his ears forward, click, I’ll reinforce him.  If I’m walking past his stall and he puts his ears forward, click, he’ll get a treat. Pretty soon I’ll see that my presence is triggering ears forward.  I’ve made it a “hot” behavior.

So, now if I withhold the click in grown-ups, I’m likely to get a resurgence of “hot” behaviors.  I’m still using extinction, but I’ve set my horse up for success. The behavior that is going to pop out is the one I’ve recently made “hot” – in this case ears forward.

Click For What You Already Have
I won’t even shift my focus to ears forward until they are already occurring at a high frequency.  My goal is to have him standing beside me with his ears forward, but initially I’m happy if he simply takes his nose away from my arm.

As I click him for keeping his head directly between his shoulders, some variability is going to come into the overall behavior.  Sometimes he’ll have his head slightly higher, or lower, his ears forward or back.  I may be so busy monitoring the orientation of his head, I won’t even notice what he is doing with his ears.

As his head stabilizes and his overall orientation becomes more consistent, I’ll be able to take in more of these subtle variations. The movement of his ears pricking forward will catch my attention.  I’ll become increasingly aware of what he is doing with his ears.  If they are almost always pinned, there’s no point in making ears forward the next criterion.  I’ll be surfing a long extinction wave before ears forward pops out. In fact for something like ears, the more frustrated he becomes, the less likely they are to go forward.

So I’ll “prime the pump” instead.  I’ll make ears forward a hot behavior.  Now when he’s in grown-ups, if I make ears forward the next criterion, I’ll be withholding my click for only a second or two.  My horse won’t be perceiving the event as unpleasant or frustrating. The click will shift seamlessly to the new criterion.  That slight moment of extinction causes my horse to surf through current “hot” behaviors. I’m using resurgence, but in a way that sets the horse up to have success build on success.

Coming Next: Cues and Extinction

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY Full Horses: Understanding Extinction: Part 4

Extinction: Big, Small and Accidental

Accidental Extinction
Extinction is not a rarity.  Extinction is going on all the time, but we aren’t always aware of it.  Suppose you’re working with your horse. Perhaps you’re in the early stages of clicker training and the focus of your lesson is “grown-ups are talking”.  You’re walking a few steps, stopping and asking your horse to stand quietly with his nose centered between his shoulders.  He’s been doing well.  You’re almost done with the session when your cell phone rings.   You answer it, taking your attention away from your horse.

Your horse doesn’t realize that you’ve disconnected from him.  You haven’t gone through a teaching process to tell him that the ring tone of your cell phone is a cue for him to take a nap.  While you are on the phone, you will not be engaging with him.  Your horse doesn’t know this, and he doesn’t understand why the flow of your session has so abruptly changed.

He offers you a nice bit of “grown-ups” that meets all the previous criteria, but you aren’t paying any attention.  He doesn’t get clicked.  He tries harder, maybe throwing in some head lowering.  That doesn’t work either so he tries some earlier experiments – some head bobbing, some lip flapping, some gentle nudging, and finally a hard nudge.  That gets your attention, but now you’re thinking what an impatient, muggy horse you have!

Your horse is offering “rude” behavior, bumping, nudging your arm, snuffling around your pockets.  He’s scrolling through the behaviors that he’s tried in the past.  You click something, anything out of desperation.

What you are reinforcing is not just that single moment, but all the scrolling through his repertoire he’s been doing trying to get you to click. You have just locked into your future training all those other unwanted behaviors.  It’s going to be very hard to convince your horse to put into moth balls those unwanted segments. They’ve become an instant part of the whole sequence.  If the current behavior isn’t working, scroll through all your past mugging behavior.  That will get your person’s attention back where it belongs – on you!  That’s what he has just learned through that one desperation click.

Case in point: Jesús showed a video of an experienced trainer teaching a dog to retrieve a dumbbell.  The dog had been successfully delivering the dumbbell to the handler, but now she wanted to raise the criterion and have the dog place it more firmly in her hands. When the dog did not get reinforced for the usual behavior, he dropped the dumbbell, did a quick head bob, and then picked the dumbbell up again. Just as the handler clicked, the dog sat. Oh oops!

She lowered her criteria.  The dog handed her the dumbbell, but now he was also sitting as he did so.  Her hand reaching to take the dumbbell had in one click become a cue to sit.

Mini versus Maxi Extinctions
When the dog started offering behavior to get his handler to click, that’s the extinction process at work.  We don’t tend to think of it in this way.  To develop the behavior we are training we actively want the offering of behavior.  Shaping depends upon differential reinforcement.  The dog offers a head bob, a paw lift, a sit.  We pick and choose among these behaviors.  We think of extinction as something separate, something to be avoided. It’s a long drawn out process with lots of painful emotions associated with it.

Jesús wants us to understand that the process can occur in seconds. When you are shaping, you are working with mini extinctions.  When learners are offering behavior, they are going through a resurgence process.  You don’t have to go hours or even minutes for the extinction process to begin.  It happens in seconds.

My ears perked up the first time I heard Jesús talk about extinction in this way.  I love this concept of mini extinctions.  It fits with microshaping and shaping on a point of contact.  All three are learner-friendly because they make use of thin slicing and create high rates of reinforcement.

We looked at Microshaping in previous sections (https://theclickercenterblog.com/2016/11/10/).  Kay Laurence stresses that it’s not thin slicing alone that defines microshaping.  It is high rates of reinforcement.  In microshaping Kay wants a success rate of 98% or higher. To get that you have to be very skilled at setting up the training environment.  The learner is not surfing through a long series of behaviors trying to find the one that is “hot”.

Instead the learner is set up to keep giving correct responses. There are very few opportunities for unwanted behaviors to creep in.

Kay contrasts microshaping with what she refers to as sloppy or dirty shaping. Here the handler lets the animal offer behavior after behavior looking for the one that will satisfy the criterion.  I’ve always been uncomfortable watching people freeshape in this clumsy fashion.  They miss so many opportunities to click because they are looking for too much.

Now Jesús has helped me understand why this type of shaping makes me so uncomfortable.  Mini extinctions are part of puzzle solving.  But they are mini. Success happens frequently so the frustration level stays low.  You could in fact see it as a positive motivator.  That little bit of: “is it this or is it that?” leads to a feeling of satisfaction each time you make the right choice.

Contrast that with macro extinction.  Now it’s not “this or that, or this other solution either.”  In fact nothing you try seems to work.  The frustration level rises to a level that takes away the fun. You can see this when you play shaping games with people who are new to training.  It’s supposed to be a fun experience, but when the one doing the clicking doesn’t have a clear plan, it’s anything but.

Training Game Mishaps
Suppose you’re the learner in one of these games.  The person who is acting as your trainer sets a teacup on the table. You get clicked a couple of times for touching the teacup.  Okay, so far so good.  The teacup is clearly a “hot” item, but what are you supposed to do with it?

You try turning the teacup, picking it up, turning it upside down.  Nothing works.  You pretend to drink out of it, you spin it, you hold it delicately with your little finger out, you scoot it over the table. Nothing gets clicked.  Your frustration rises in direct contrast to your willingness to play the game.  You’re in a macro extinction that can be painful to watch.  You go back through the history you have with teacups.  What else can you try?  Nothing is working.  You want to give up or better yet throw the tea cup at your trainer!

This offering of behavior is part of the extinction process.  You are experiencing a resurgence of previously reinforced behaviors.  In the teacup example, when you were no longer reinforced for just touching the teacup, when reinforcement for that behavior stopped, you tried things that you had done with tea cups or tea cuplike objects in the past. But in this case your trainer is a new shaper.  She is outcome oriented, so she is looking for big macro responses. She doesn’t yet know how to set her learner up to give her the small reaction patterns that would lead seamlessly to an end goal.  The result is an unhappy and very frustrated learner.  Both the learner and the trainer go away feeling unsuccessful, and they both vow never to play the training game again!

Micro Extinctions
When someone is shaping and they want to raise the criterion, they stop reinforcing for a behavior that was just successful. The learner goes through a resurgence/regression process.  She begins to offer other behaviors that have worked in the past.  People tend to think of extinction as happening over a long period of time, but Jesús kept emphasizing that it happens over seconds.  Two to three seconds is all you need for a mini extinction. You’ll begin to see the learner offering behavior other than the one that was just being reinforced.

Again, this got my attention.  I don’t like the frustration you see when a puzzle appears to be unsolvable.  Shaping shouldn’t be marked by sharp drop offs in reinforcement.  I don’t want to see macro extinctions.  If reinforcement is that sticky, it’s time to change your lesson plan.  Either put the horse away altogether while you go have a think, or regroup by shifting to another activity.  If you keep waiting, waiting, waiting until your learner finally gets close to the answer, you could lock in some unwanted behavior, and you will almost certainly lock in some unwanted emotions.

What are some good teaching strategies that help you avoid the frustration of macro extinctions, and that lead you instead to the elegant use of micro extinctions?  That’s what we’ll be exploring in the next section.

Coming Next: Using “Hot” Behaviors

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com