An Introduction to Clicker Training: Day 1

This is Part 1 of a 4 part article.

The last time I posted I had just returned from the Five Go To Sea conference cruise.  It’s now November and I have just finished the 2015 clinic season.  Thank you to everyone who was able to join me this year.   Sharing clicker training is such a joy.  I am looking forward to seeing you again in 2016.

I’ll have my full  2016 clinic schedule posted on my web site soon.  In 2016 I’ll be back at the Cavalia retirement farm for another series of clinics.  And I’ll be returning to several of my long time clinic locations for more Clicker Intensives.

My conference schedule is already posted.  2016 is going to be a great year for learning!

The last clinic of the 2015 season was at Cindy Martin’s farm in Arkansas. One of the participants brought a horse who was completely new to clicker training. Usually when I have a start-up horse we are in narrow barn aisles with poor lighting and limited site lines.  Cindy’s barn is a perfect film studio. It had an extra wide aisle, tall ceilings and great lighting.  So we were we able to video this horse’s progress over three days.

What follows is a detailed account of the start-up process.  Whether you are brand new to clicker training or an experienced trainer, I think you will find this of interest.  This report takes you very systematically through the process that I recommend for introducing a horse to clicker training.  I have included video as well as the discussions that followed each training session.

Enjoy!

 

Step One: Introductions

Nick grown ups look away

My clinics always begin with a Friday night gathering.  When you talk to experienced clicker trainers, they always emphasize the importance of building a relationship with the animal you are training.  Relationship can be one of those fuzzy, feel-good words.  You can have all different kinds of relationships, some good, some quite toxic.  When science-based trainers define relationship, they are referring to a history of reinforcement.

In clicker training we put the emphasis on creating a history of positive reinforcement.  One of the metaphors that’s often used is that of building up a bank account by making many deposits via positive reinforcement.  As the bank account grows, if something happens and you do have to use a correction, your “bank account” can withstand a small withdrawal.

So training begins by building up that history of reinforcement with the animals you’re working with.  I’ve always felt that it was important to treat the people I work with with the same consideration that I give to their horses.  While I’m clicking and treating the horse, I don’t want to be barking commands at the person and criticizing every little mishap.  I also don’t want to spend three days working with a group of people who are essentially strangers.

I know I’ll be seeing many of the participants again at future events, so I like to get to know them as individuals, not a sea of anonymous faces.  So we spend the Friday night gathering in introductions.  I want to know what brought people to a clicker clinic.  What is their horse background? What is it they are hoping to get from the weekend?  I tell them that this is their first shaping exercise of the weekend.   Their introductions, what they are specifically looking for, help shape the clinic experience they are going to have.

Wendy, the owner of our first-time clicker horse, told us his story.  She was given Nick after friends of hers had given up on him.  They had hoped he would be a reining horse champion.  They had bred their mare to a top stallion.  At eighteen months he was sent off to a top trainer who thought the world of him.  He considered him his best futurity prospect of that year.  By the time Nick was two, the trainer was less excited by him, and by three he was saying Nick would not make it as a reiner.  It’s a familiar story.  His disappointed owners took him out of training, and then found that they no longer wanted him, so they gave him to Wendy.

Wendy’s training concerns centered around a desire to feel more connected to Nick.  He was very aloof.  He wasn’t hard to catch out in the pasture, but he never came directly up to her.  He would always stop ten to twenty feet away from her.  Out on trails she described him as safe to ride, but easily distracted.  It was hard to get him to focus on the rider.

People often come to clicker training as a last ditch effort to “fix” a problem horse.  Nick didn’t need “repairing”.  He was not a “broken” horse.  As I explained to the group Saturday morning, we weren’t trying to fix anything.  We were simply introducing a nice horse to clicker training.

What to Feed

We began with a discussion of the treats, both what to use and how to handle them in these early training sessions.  I rejected some flavored commercial horse treats that Wendy had brought in favor of some plain timothy alfalfa pellets Cindy had for an insulin-resistant older horse.

In these early start-up lessons you are asking for simple behaviors.  All the horse has to do is touch a target or move his nose away from your treat pocket.  You want to keep the rate of reinforcement high so the horse stays engaged in the game.   That means you are going through a lot of treats fast.  When I don’t recognize the commercial horse treat, I don’t know what I am feeding.  Is this something that is designed to be truly that – a treat, something you feed in quantities of one or two at a time, or is it something we can safely use in larger quantities during a training session?

If I’m not sure of ingredients, and especially of the sugar content, I prefer to use something like the timothy alfalfa pellets.  They are bland enough that a horse isn’t going to have a “sugar high” during the training, but still enough of a treat that he’ll want to figure out how to get me to give him more.

Protective Contact and the Importance of Choice

In addition to a discussion of what to feed, I also talked about protective contact.  What this means is the handler is separated from the horse by a barrier.  There are a number of reasons for using protective contact to introduce a horse to clicker training, even with a horse you know well.

When a horse is loose in a small paddock or a stall, he is free to interact with you – or not.  He has choice.  That’s key to clicker training.  If you go in with a horse, even if he is at liberty, his previous learning may interfere with his ability to figure out this new clicker game.  He’ll be so busy responding to previously learned cues, he won’t even be aware that there’s a puzzle to solve.  In fact, the more well-trained a horse is, the more important this reason for using protective contact becomes.

And if a horse isn’t so well trained, all the safety reasons for using protective contact come into play.  Without the barrier, if a horse crowds you trying to get to the treats in your pocket, you’ll need to do something to push him away.  If you’re having to correct him for this unwanted behavior, you’re creating a bind for yourself.  On the one hand you want your horse to feel safe experimenting and offering behavior.  And on the other hand you’re still saying no, don’t do that.  You’re essentially poisoning your first clicker encounter.

The barrier removes the safety concerns and gives your horse choice.  Choice is very important.  Current research is confirming that choice is reinforcing.  When we put a horse into protective contact, we are giving him the choice to interact with us or not.  We are also keeping things safe.  I don’t know the horses I am introducing to the clicker.  I don’t know which one is going to get super excited about the food and push into my space.  I don’t know which horse is going to show a huge regression into unwanted behaviors when the constraints of punishment are removed. (See my blog post:  Resurgence and Regression: Five Go To Sea Conference Presentation.  This is a twelve part article.  Part 1 was posted on May 21, 2015.)  Until the horse shows me that it is safe to go in with him with my pockets filled with treats, I stay with protective contact.

This is one of those soap box issues for me.  I know that there are many people in the horse community who will NEVER try clicker training.  Feed horses!  Give horses choice!  Horrors!

That’s fine.  Clicker training doesn’t have to be everyone’s cup of tea.  But, clicker training doesn’t just introduce the use of marker signals coupled with reinforcers.  It also brings into the horse world this idea of training with protective contact.  If we can introduce the concept of protective contact into mainstream horse training, we will be doing a very good thing indeed.  People go in with horses much too soon, and as a result, they end up reaching for punishment solutions first.

I remember watching a clinic years ago where a very well known clinician made everyone climb in and out of the round pen – even though there was a perfectly good gate.  The reason was this:  if a horse every charged you, you’d know how to climb the fence so you’d be able to get out of that pen fast!

There’s something wrong with this reasoning.  I don’t want to be in the pen with a horse until he tells me he’s comfortable having me there.  If he feels threatened, I don’t want him thinking he needs to attack me to remove the danger.

I want the horse to show me that he understands enough of “my language” to be able to figure out the puzzles I’m presenting.  Horses are punished for so many reasons, including not responding fast enough to commands.

When you’re afraid, it’s hard to think straight and follow instructions.  We know that from our own experience.  So imagine what it must be like for a horse.  He’s struggling to figure out what is wanted.  If he hesitates, he’ll be punished. If he reacts fast, but guesses wrong, he’ll be punished.

When a horse isn’t not sure what is wanted, is it any wonder he feels threatened and frustrates easily?  Aggression comes from a place of fear.  If I am working with a horse that is quick to lash out to protect himself, I want a barrier between us.  That way I won’t be adding fuel to the emotional fire by correcting him to keep myself safe.  I can just step back out of the way while he goes through the “learning how to learn” process.

Throughout the horse industry, if we treated horses more like zoo animals and used more protective contact, we would see an overall shift towards kinder training.  “Aggression comes from a place of fear” doesn’t just apply to horses.  Think about that the next time you are watching someone cracking a whip or swinging a lead rope at a horse.

Enough of the soap box.  Nick was stabled overnight in a large 14 by 16 box stall with a door that opened out onto a small outside covered run.  A stall guard was already set up in the stall door so creating protective contact was easy.

Keep your First Sessions Short: The Twenty Treat Strategy

The first lesson I teach is generally targeting.  For this first lesson I count out twenty treats.  I want to limit how long the session can last by limiting the number of treats I start out with.  When I run out of treats, I am obliged to take a break from active training.  These breaks do a number of good things.

First, they show the horse that this interesting game begins, ends, and then comes back again.  For many horses during this first introduction into clicker training is a true “Helen Keller” moment.  (If you don’t understand this reference, watch the old movie “The Miracle Worker”.)

When the horse figures out that he can control your behavior,  you often see what are referred to as “light bulb moments”.  For some horses this a huge sea change.  The horse needs to understand that this amazing experience is something that will continue on past the first introduction.  That’s part of what helps him to settle into the experience.

By giving breaks your horse is learning that the game stops, but it comes back again.  For many horses clicker training is a world changer.  Their human makes sense!  They can actually train their human!  Their world opens up.  You’re offering the target, and clicking and treating.

And then you close the door and walk away – and for the horse it’s back to business as usual.  All that clear communication vanishes.  For some horses this can be really hard on them.  Get back here and train me!  It’s bang, bang on the stall door.  If you’re in a boarding barn, that’s going to make you very unpopular with the owner!  By giving breaks, the horse learns that the game has pauses and then starts up again.

So I like to do a short session and then go away for a few minutes.  Then I come back to do another round.  The game ends, and then it begins again.  That’s hugely reassuring to the horse.

The other thing that the breaks create is an opportunity for the handler to think about the training.  Clicker training is fun.  You see your horse touching a target with more confidence.  He’s making progress.  It’s like the old potato chip commercial – “bet you can’t eat just one.”  You’ll be thinking: “I’ll just have him target one more time.  How about one more time after that?” 

If you fill your pockets at the start, you could end up training and training.  Caught up in the momentum of the moment, you might not notice some unwanted behavior that’s crept into the game.  Yes, your horse is eagerly touching the target, but he’s also pinning his ears flat to tell his neighbor in the adjoining stall to stay away!  That’s not the kind of behavior you want him to be weaving into his first clicker training experience.

If you take a short break to think about each session, you’re more likely to pick up on these unwanted behaviors, and you can make adjustments to your training environment as needed.  You want to catch any “speed bumps” in your training early on so you can make the necessary changes that ensure that the behavior you want is the behavior you get.

The breaks also give you time to think about what to do with the next round of twenty treats.  And they let you appreciate the steady, good progress you see your horse making.

Choosing Your Target

I generally introduce horses to the clicker via basic targeting.  Most horses are curious.  If you are holding something in your hand that doesn’t look too much like a goblin, they will come up to see what you’ve got.  Click!  They have just earned a treat.

Lots of things make good targets.  They need to be easy to handle, horse safe, and nothing that looks too scary.  Small plastic cones work great, as do empty water bottles and the lids off of supplement containers. In other words, you don’t have to spend a lot of money on targets.  Look around your barn.  You are bound to find something that will work.  From the collection Cindy had in her barn, I chose a great target stick.  It was made from an old riding crop.  She had stuck  a bit of foam from a pool noodle over one end and wrapped it in duct tape.  Perfect.

Planning Your Exit Strategy

Before I could introduce Nick to targets, I first had to think about my exit strategy.  If I’m going to take breaks, I need a way to end a session that’s okay for the horse.  I’ll be taking the treats, the attention, the game away with me.  I don’t want the horse left thinking that whatever he just did it made me leave.  He promises never to do that again!  If I’m clicking and treating for desired behavior, that’s the opposite of what I want, so I need to think out in advance how I’m going to end a training session.

The horses I work with regularly are all familiar with treats being tossed on the ground.  When I need to break away from a conversation we’re having, I toss a few treats on the ground or into a feed bucket for them.  While they are searching around for the goodies, I can slip away.  However, I can’t assume that other horses will know where to look when I toss a treat into a feed bucket.  So  Step One for Nick begins with this very easy lesson.  I simply opened his stall door, dropped a feed tub onto the floor and began dropping treats one or two at a time into the bucket.  Is it any wonder horses love clicker training!

This first video clip shows Nick learning that the sound of treats being tossed into his bucket means goodies are to be found there.  You will see that Wendy’s description of Nick was very accurate.  While he is willing to come to the front of the stall to take treats, he is very cautious.  This was a good first step both to help build his confidence, and to begin to let me learn more about him.

Video: An Introduction to Clicker Training: Session 1: Creating an Exit Strategy

 

Data Collecting: The First Targeting Session
Next came Nick’s first targeting session.  I view this first round of targeting very much as data collecting.  I am not thinking of it as the “teaching” part of the training.  This is exploratory training.  Nick’s response to the target will tell me where I can begin.  If he’s reluctant to even come up close to the target, I know that I’m going to have to break this lesson down into much smaller steps, beginning with more time spent just clicking him for acknowledging my presence.

On the other hand if he eagerly touches the target and then grabs for the treat and remains fixated on my pockets, I know I have a very different starting point for the training.  It’s all data collecting.  Until I ask Nick a few questions, I won’t know where to begin.  As you watch this next video, you can join us in the data collecting.  It shows Nick’s first targeting session.

Video: An Introduction to Clicker Training: Day 1 – The First Targeting Session

As you can see these sessions are short.  Putting only twenty treats in my pocket forces me to stop after just a couple of rounds of targeting.  While I am getting more treats, I can think about what I learned about Nick.  What did I like about this round?  What unwanted behaviors, if any, do I need to be aware of?  What do I want to do next?

Data Assessing: What Did We Learn?

The two main questions are: is there anything about this horse’s behavior that would suggest that it would be unsafe to go in the stall with him with my pockets filled with treats?  If the answer comes back yes, or even, I’m not sure, I stay with the protective contact.

The other question I ask is what should I do with the next round of twenty treats?  Should I continue with the targeting, or does his behavior suggest that I should shift to one of the other foundation lessons?  I have six to choose from: targeting, backing, head lowering, the grown-ups are talking – please don’t interrupt (meaning take your nose away from my treat pockets), ears forward (which I refer to as Happy Faces) and standing on a mat.  Taken together these six behaviors work beautifully to introduce horse and handler to clicker training.  They are great behaviors to choose because the end result of these lessons is a horse who has beautiful ground manners and lots of emotional self-control.

Often when I ask a group: “what did we just learn about this horse?”, they are stumped for words.  They don’t know what they are supposed to say.  We are so used to criticizing what we see.  That’s how we’ve been trained in school, at work, even in our own families.  “What’s wrong with this picture” is what we know, and yet, that doesn’t quite seem to be what I’m asking for.

There may indeed be things the horse presents that are a concern, but I really am looking at this through a different lens, one that is often very unfamiliar to first-time clicker trainers.  They haven’t spent a lot of time practicing finding things they like, so this is a great opportunity to develop that skill.  Let’s listen in to this first assessment process:

Video: Introduction to Clicker Training- Group Discussion After the 1st Target Session

Especially if you are new to clicker training, I recommend that you videotape your first clicker sessions.  They really can be fascinating.  The change in your horse’s understanding can happen so fast.  You can use the pauses between training sessions to review your video.  Often you’ll see little things in your handling that you’ll want to change.  If you are reaching into your treat pocket ahead of the click, or you’re feeding too far forward so your horse gets pulled onto his forehand, you’ll catch these handling glitches and be able to change them for the better in the next round.

Details That Make A Difference

To help with this type of data collecting I’ll point out a few handling details that you can watch for in this next video clip:

Note how I handle the target.  I position it so it is easier for Nick to touch it than it is for him to get to me.  If he starts exploring past the end of the target stick, I very quietly ease myself out of his reach while still keeping the target stick easily available.

After I click, the target stick drops down into a neutral position.  I give him his treat, and then immediately bring the target back up so it is available to him to touch.  The movement of the target stick as I bring it back up helps to attract his attention.  Taking it down after I click means that each round is a discreet trial.

Note also how deliberate I am in how I present the treat.  After I click, and NOT before, I begin to reach into my pocket to get the treat.  I am prompt in beginning the reinforcement delivery but that doesn’t mean I move fast.  Note the rhythm of the overall lesson. I maintain a steady pace which allows me to set the tone of the lesson.

Some common things to watch for in your own handling would be:

* Are you reaching into your treat pouch ahead of the click?  The movement of your hand will overshadow the click.  It will become your marker signal which isn’t what we want.

A good marker signal has the following characteristics:  It is quick.  It is unique, meaning it stands out from other stimuli in the background. It is non-emotional.  The click of the clicker meets all these criteria.  It gives you so much precision when you begin to work on details in performance.  And it means your horse does not need to be watching you to respond to a marker signal.  That’s a huge advantage for riding.  So take care now to make the click of the clicker a clean marker signal. (Note, very quickly you will shift from the actual clicker to a tongue click.  You’ll see me do this with Nick in the clips that follow.)

* Are you feeding out away from your body where the perfect horse would be?  If you feed in close, you’ll be reinforcing your horse for coming into your space.  I want to feed Nick so his head is away from me.  More than that, I want to take full advantage of the fact that I’m using food in my training, so I’m going to feed him to encourage good balance.  In this first lesson that means his head will be even between his shoulders and at a good height so he isn’t pulled onto his forehand to get the treat.

* Are you remembering to take the target down in-between trials.

* Are you bringing the target up promptly so it is approximately in the same orientation that it was in the previous trials?  Often people move the target around from one spot to another in this first round.  First the target is high, then it is low  That means that every time the horse touches the target it’s a very different behavior.  Even if the horse is managing to touch the target, he may be struggling to understand the underlying concept behind the click and treat.

* Are you maintaining a steady pace throughout the cycle?  Sometimes people rush to get the food out.  They are forgetting that the click buys them time.  It marks the moment you like.  You can think of it like a place holder.  You want your horse to understand that the behavior that was occurring at the exact moment that you clicked is worth trying again.  As long as you begin to reach into your pocket promptly, you don’t have to be fast in the delivery.  This isn’t “click and shove” training.  Your horse can see that you are getting a treat.  He knows it’s coming.  The whole process is part of the reinforcement.  It’s like Christmas.  It isn’t just ripping open a present that’s reinforcing.  It’s all of the anticipation leading up to the event that makes the presents so exciting.  With a beginner horse you can’t mess around and take too much time getting your treat to him.  That can be frustrating, but nor do you need to feel as though you have to rush.

Look for these details as you watch Nick’s second targeting session.

Training Sessions #2: More Targeting

Video An Introduction to Clicker Training: Day 1 – 2nd Target session

In this next round of targeting Nick is interested.  He’s waiting for me at the front of his stall, but he’s still unsure what this is all about.  Nick is slow to touch the target.  Waiting for him to touch the target can feel a bit like watching paint dry.   I have to decide: do I wait for an actual touch, or do I click any orientation towards the target?  With Nick I decided to wait for him to touch the target.  It wouldn’t have been wrong to click for approximations. That’s one of the great advantages of clicker training.  There is no one only one right moment to click.

Someone else might make different decisions.  Nick’s behavior will tell us if we are on the right track.  That’s one of the reasons to begin with these short sessions.  They give you time to consider the choices you’ve made so you can make adjustments.

In the discussion that followed this section, I again reminded people to count out their twenty treats so these early sessions remain short.  As long as I am still working with protective contact, I continue to count out twenty treats.  Once I decide that it’s time to go into the stall with the horse,  I then add more treats to my pockets.

As a group we agreed that Nick was remaining very calm and polite, but he didn’t yet get what this game was all about.  As one of the participants said: “That’s what is so exciting about watching this. That moment when a horse understands that he just made you click and hand him a treat, it changes the whole world for that horse.”

Often times in more traditional forms of training horses are corrected for offering behavior that hasn’t been asked for.  When you hold a target up, these horses wait to be told what to do.  Taking the initiative is not something that feels safe to them.  You’ll see them remain hesitant about touching the target.  Both Nick’s background and his behavior matches this scenario.

You don’t want to rush through these simple targeting lessons.  Again it is about choice.  I want Nick to discover that he has control of this lesson.  That’s what will build his confidence.  He needs me to give him whatever time it takes to make this discovery.  That’s what will turn him into a bright-eyed clicker star.

This is another good reason to use protective contact.  The stall guard limits my ability to step in a get “things done”.  This lesson isn’t about getting a horse to touch a target.  If I were goal oriented, that’s all I would see.  How fast did I get this horse to touch the target?

That’s just a behavior.  It’s a tool I’m using to teach Nick much more fundamental lessons.  I want him to understand the core values of clicker training.  I want him to understand that he does have choice, that his feelings matter.  I want him to understand that what is evolving is a conversation, and that his voice counts.

In the third round of targeting, you’ll see more changes.  Again we decided to continue with the targeting.

Video: Introduction to Clicker Training: Day 1 – 3rd Targeting Session


As I finished this round, I decided that I would do just one more round of training that morning – in part because I was looking at how much of the original bowl of treats I had gone through.

Measuring out the treats into a bowl was a good reminder to everyone that you need to keep track of how much you are feeding.  In these initial training sessions you are working with very simple behaviors.  You can go through a lot of treats fast. It’s one touch of a target, click, treat.  One quick moment of grown-ups, click treat.

Later you’ll be asking the horse for so much more.  You’ll be building complex sequences of behavior and asking for much longer duration.  You’ll be getting much more behavior for every single click.  When you reach the stage of filling your pockets at the start of a training session,  this early discipline of counting out your twenty treats will help you keep track of how much you are feeding.

Video:  Introduction to Clicker Training: Day 1 – 4th session

This fourth round of targeting shows the evolution of the food delivery.  Remember in the first round I fed Nick approximately where the target had been.  Now I can move into his space to feed him.  Initially it’s easy to think in terms of single behaviors.  You are having a horse touch a target.  But really what you have is a cycle of behaviors.  I refer to this as loopy training.

As a shorthand way of describing clicker training, we will often say that behavior leads to a click which leads to a treat.

You can write this as:

Behavior leads to  Click leads to Treat
or the short hand of:
Behavior => Click => Treat

It’s an easy way to think of clicker training. If you like it, click and reinforce it.  Reinforce is the key.  When you write out that single phrase, it’s easy to think of this event as an isolated unit, but reinforce means to strengthen.  If a behavior has been reinforced, you should see it occurring more and more frequently. So really what we have is a loop of behavior which we can write out as:

Behavior => Click => Treat =>
More of the Behavior => Click => Treat =>
More of the Behavior => Click => Treat => etc.

The food delivery is a dynamic part of this process.  I could simply click and continue to feed pretty much as I did in the first round of training, but that would be giving up so so much of the value that I get from my treats.  People who don’t use food in their training are at such a huge disadvantage. I know that’s not how they think about food, They see it as a nuisance, a distraction, a bribe.  All these negative labels can keep someone from seeing the enormous advantages using food as a reinforcer gives me.  It isn’t simply that getting something he wants want makes the horse more eager to do it again.  I get so many more good things from this part of the cycle:

* I can help a horse find good balance through my food placement.  You can see that in this fourth round of the training.  I place the food so Nick not only straightens out his head, but I am also placing it at a height that encourages him to lift up in his shoulders and come off his forehand.

* I can also set up the next phase of the training.  At some point I will want to ask Nick to back up.  By getting the backing first from the food delivery, I can map out the behavior before asking for it directly.  I can see how easily – or not – Nick backs.  If his feet get stuck and he struggles to back up to get to his treat, I know I will need to be careful how I introduce backing when I ask for it directly.  I may be dealing with a horse with arthritic hocks so backing is painful.  Or backing may have been taught so punitively that the horse resists and resents any attempt to make him back – even if it is to get to a treat.  Or the horse may simply never have been asked to back, and he can’t figure out how to move his feet.  Whatever previous experience Nick has with being asked to back, introducing it via food delivery gives him a huge jump-start on solving the clicker backing puzzle.

What this boils down to is I use food delivery dynamically.  These simple targeting lessons introduce this concept early on in the process.  The horse becomes accustomed to moving his feet to get treat, plus he becomes familiar with the feel of good balance.  The horse is learning to follow the handler’s body language creating the foundation for both leading and liberty work.

Nick’s owner was thrilled.  These four short training sessions showed her the structure she had been looking for.  She appreciated how systematic and organized the process was.  She saw how valuable the breaks were.  As she said, when you are new to clicker training, you need to step back to take stock of where you are.  Is this behavior going the direction you want, or is it going off the rails somehow?  Video really helps you keep track.  Most of us have cell phones or cameras that can take short video clips.  If you don’t have a tripod, you can always prop your phone up on a hay bale or a fence rail.

During the breaks you can review your video.  You’ll see your hand creeping towards your treat pouch ahead of the click.  No wonder your horse was more interested in mugging you than touching the target!  Details like this really do matter.  It’s so much better to catch them early on so your handling isn’t creating confusion and unwanted behavior.

The total training time for Nick that morning was twelve and a half minutes spread out over a forty-nine minute session.  If you were working on your own, you might spend a few minutes introducing your horse to a target, take a break to review your video, do a couple of barn chores, then return for another round of targeting.  Breaking up the individual targeting sessions by putting barn chores in between spreads the sessions out nicely.  After you’ve reviewed your video, it also gives you time to think about what you want to do next.

When you are brand new to clicker training, it’s perfectly understandable that you may not be completely sold on this style of training.  All you’re doing with these first few sessions is testing the waters.  Is this something you’re going to enjoy and find useful?

The structure I’ve shown in these video clips gives you a safe way to become familiar with the overall process.  If you need to take a pause from clicker training, you’ll be able to do so.  Context cues matter.  If you don’t have a target, your horse won’t be expecting the game, so you can handle him business as usual when you put away your treat pouch.

Having said that, when I introduce a horse to clicker training, I feel as though I am making a commitment to that horse.  I don’t want to show the horse that we can communicate clearly and then snatch that experience away.  Once I open up the communication channels, I want to keep them open, active, and ever-enlarging.

For Nick I may have decided that this fourth session would be the last for the morning, but it was definitely not the last of the weekend.  We continued on in the afternoon with another round of targeting.

Just as the clinic participants had to wait to see how Nick processed his morning lessons, you will have to wait to see the afternoon sessions.  I’ll post those in Part 2 of this series.

This is Part 1 of a 4 Part series on introducing a horse to the clicker.

Alexandra Kurland
theclickercenter.com
theclickercentercourse.com

My thanks to Cindy Martin for organizing and hosting the November clinic, and to all the clinic participants, especially Wendy Stephens and her beautiful Nick.

Please note: This article gives you wonderful details to get you started with the clicker, but it is not intended as complete instruction.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

 

 

 

An Invitation

The series on Resurgence has generated a lot of interest, and I know from my in-box, also a lot of questions.  So I’ve invited Dr. Jesús Rosales-Ruiz and Kay Laurence to join me for an on-line question and answer session.  I hope you will join us, as well.

AK JRR Kay on cruiseWhen: June 21, 2015

What Time?  1 pm Eastern Time Zone (Check on line for time zone converters if you aren’t sure of the time difference.)

Cost: $35.00

Space is limited.  If you would like to attend, do sign up early.  The meeting is limited to a maximum of 25 people.

How do you sign up?  Email me at: kurlanda@crisny.org.  I will send you a paypal invoice for the meeting.

What do you need to participate?  For optimum audio quality, a headset is recommended.

What should I bring?  Lots of questions!

Alexandra Kurland
theclickercenter.com

5GoToSea: Part 15: Micro Masters

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL
Part 11: Mastering Extinction
Part 12: Creativity Explored
Part 13: Degrees of Freedom
Part 14: The Positive Side of Resurgence
Part 15: Micro Masters

If you have not read the previous installments of this series, I suggest you begin with Part 1. Part 1 was published on May 21, 2015.

Part 15: Micro Masters

The “Pose”
Jesús closed his presentation with two horse examples.  The first was Robin’s “pose”. I’ve told the story of the “pose” many times.  I’ll keep it brief here.  Robin first learned a stationary “pose”.  It originally was a by-product of cleaning up his treat taking manners when he was two years old.  During the process he started “posing”, arching his neck and looking like a very pretty dressage horse.  I liked the look so I continued to reinforce it.  It became a default behavior.  In the absence of any other active cue from me, if Robin posed, I would click and reinforce him.  I became the cue for the behavior.

Offering “the pose” meant that if Robin wanted to interact with me and engage in the clicker game, he had a sure fire way of doing so.  Even if I was busy doing barn chores, if I saw him posing, I would click and reinforce him.  I never wanted him to feel like the proverbial toddler who is banging the kitchen pots and pans to get his mother’s attention. If Robin wanted attention from me, he had a behavior which he could use to satisfy his need for social interaction.

Because Robin wasn’t ignored, he didn’t go through an extinction process.  I didn’t see a regression into the unwanted behaviors that macro extinctions can cause. Instead I was able to reinforce a behavior I liked, one that was a useful warm up for our formal training sessions.  For his part Robin was confident that I would engage with him when he asked for attention.

Reinforcing him for the stationary pose went on through the winter.  I didn’t have any plans for developing the behavior.  It was simply something I liked.  It was Robin who was the creative one!

It must have been late March.  I was lunging him in the arena one evening.  He was giving me a ho hum trot.  There was nothing there I could reinforce.  Robin went once around the circle, twice, three times without reinforcement.  Normally I would have been clicking and reinforcing him at a much higher rate, but given the plow horse trot I was presented with, there was nothing there I wanted to say yes to.

At the time I would not have described it in these terms, but I was putting him into an extinction process.  I could see him searching, trying to decide what to do.  On the third time round he had the answer.  He would try his pose.  But in order to pose and still stay in the trot, he had to add energy.  Within one stride he transformed into magazine-cover magnificence.  I captured the moment with a click and the rest is history.  The “pose” has evolved into a major component of my work.  Robin showed us that we could indeed shape self carriage.  What began as a happy accident for Robin has become a deliberate and very systematically trained behavior in other horses.

Our Creative Horses
When I first told this story to Jesús, he commented that the pose came out because of resurgence.  At the time, I didn’t understand the significance of what he was saying, but I remembered what he said.  And Jesús remembered the story.  It got him thinking about the procedure and how we might use it to make deliberate use of resurgence.  The result: we now have a systematic way of creating unlikely behaviors. The end result can look like magic, but there is good science behind it.  Here are the steps:

First, you build a strong history of reinforcement for the component behaviors.

Next, you change the situation somewhat so extinction comes into play.

This generates a resurgence of previously reinforced behaviors.  The result: new combinations emerge.  That’s creativity.  The most fun for me is seeing what the horses invent.  They are often so much more creative than their human partners!

Seeing Familiar Landscapes with Fresh Eyes
Kay Laurence might say we are seeing familiar landscapes with fresh eyes.

Jesús would say you have to understand the process of extinction so you can master it. If you understand it, you won’t be frustrating your animals.  Instead, you’ll know how to use extinction to generate complex behaviors.

I would say that monitoring the level of extinction your learner is experiencing is a keys-to-the-kingdom part of good training.  I recently spent a couple of days working with a group of horses I have come to know well.  One of them is a retired performance horse.  Without going into a lot of details, I would describe him as an emotionally fragile horse.  He’s easily worried. If he thinks he has the right answer, he’s a superstar, but I always have to be careful how far I stretch him into new behaviors.  If he thinks he might get something wrong, he worries.  He’s come out of a training environment in which he had to perform correctly or his rider could get seriously hurt. I suspect he was corrected for mistakes which accounts for his worry.

Mastering Micro
This past weekend I was working among other things on this horse’s pose.  He’s very much got the idea that he gets reinforced for lifting up through his topline and releasing at the poll.  I was holding out for slightly better versions.  As I withheld my click, I saw him experimenting. Was it higher with his poll?  Was it more lift of his back? What did I want?

The shifts he was giving me represented micro changes.  The variations were all within a clickable range.  Clicking him for any of these variations would not have been wrong, but I was waiting fractionally to see what else would pop out.  I was using micro extinctions to create the next step.  And because I was thinking about this in terms of extinction, I was monitoring closely how this related to his emotional level. I did not want him to become macro worried.

We were always just a second or two away from a click so I could let him experiment within a micro extinction without risking the emotional fallout of a larger extinction process.

Micro is so very much the key.

Macro extinctions are painful.  Micro extinctions are part of good shaping.

Macro shaping can be frustrating.  Micro shaping is elegant.

Macro negative reinforcement is literally painful. Micro is again good shaping.

When you go micro, your learner is always just a second or two away from a reinforceable moment.  You can cue another behavior.  You can click and treat. Either way, you are saying: “Yes! Great idea!”  Micro mastery is what we should be striving for in our training.  When you say someone is a great trainer, you are saying he is a Micro Master.  In training that’s the “black belt” we should be aiming for.

(Note: this video was taken when Robin was three years old.  He was not yet started under saddle.  Also, he had never been in side reins or any of the other devices that are commonly used to lunge horses.  This beautiful self-carriage was shaped entirely through clicker training.  The dressage whips that I’m using serve as targets.  They give Robin orientation points that help him maintain his balance relative to me.)

This concludes the report on Dr. Jesús Rosales’ Ruis’ 2014 presentation on Resurgence and Regression given at the Five Go To Sea conference cruise.

For information on the 2015 Five Go To Sea Alaska cruise visit fivegotosea.com

Alexandra Kurland
theclickercenter.com
theclickercentercourse.com

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 14: The Positive Side of Resurgence

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL
Part 11: Mastering Extinction
Part 12: Creativity Explored
Part 13: Degrees of Freedom
Part 14: The Positive Side of Resurgence

If you are new to this series, I suggest you begin with Part 1

Part 14: The Positive Side of Resurgence

Building Unlikely Behaviors with Resurgence
Jesús reminded us that nothing is either all good or all bad.  We want to use positive reinforcement with our animals because we see it as effective and more humane.  But positive reinforcement doesn’t always produce desirable outcomes.  In people it can lead to addictions to harmful behaviors such as over eating or gambling.

Resurgence and regression can be very negative procedures, but they can also be used to produce what might otherwise be very difficult behaviors to obtain.

toy chairJesús again used PORTL to illustrate how this can work.  In one video example, a trainer set a toy chair on the table for her learner to interact with.  The goal was to get the learner to push the chair over the table the way she might push a toy car.  The learner began to interact with the chair, but not in a way that would lead to pushing it. Why?  Because history matters. The learner is going to bring back all of her history, all of her previous repertoire of chair behaviors as she experiments.  Pushing it like a car is very unlikely because that’s not how she would have interacted with this kind of object in the past.

The same would be true if the trainer had set down a dice.  The learner would have tossed the dice or shaken it in her hand because that’s in the reinforcement history of that object.  Pushing a dice over the table like a toy car would probably be much harder to get.

Instead of trying to shape the behavior through small approximations, the trainer used resurgence.  Her first step was getting the learner to touch the chair consistently. The learner in this video was not particularly creative.  She touched the chair, but she didn’t try any other behaviors.  Getting her to push it was going to be hard.

So the trainer took the chair away and set out a toy car.  Using an object that normally would be pushed made it very easy to get the desired behavior.  The learner pushed the car over the table top. Click and treat.

This was repeated several times and then the trainer took the car away and set the chair out. The learner went back to touching it.  The chair accidentally fell over – click and treat.  The learner latched on to that, expanding her repertoire to two behaviors – touching the chair and knocking it over.  She persisted in knocking it over even when she did not get reinforced for the action.  Everything but pushing it like a car was put on extinction – meaning the trainer no longer reinforced her for these behaviors.

To avoid escalating the learner’s frustration, the trainer took the chair away and set the car out again.  The learner immediately started pushing the car over the table top. Click and treat.

To help with the generalization the trainer put a third object out – a small block.  The learner pushed the block.  Click and treat.  This was repeated several times, then the trainer took the block away and set out the car.  The car was pushed.  Click and treat.

The trainer set the chair out and the learner pushed the chair.  Job done.

Resurgence and Dog “Yoga”
Jesús next showed an example of using resurgence to train a dog to step with his hind legs onto a chair.

The dog was taught through a series of very carefully managed steps.  First, the dog learned to stand one foot each on four small plastic pods.  This alone was impressive training.  The pods were the same ones physiotherapists use to help people improve their balance and proprioception.  It took great coordination for the dog to stay balanced on the four pods.  But that was only step 1.  Next he learned to keep his front feet on the pods while he maneuvered his hind feet up onto the brick ledge of a fireplace hearth.

This was not done as a cute party trick.  The dog’s owner is a yoga teacher.  Her interest was very much the same as mine – helping her animal to maintain a healthy spine.

The last step was setting up a training session next to a chair.  The handler withheld the click, putting the dog into an extinction process.  With very little experimentation, the dog oriented himself so his hind end was to the chair.  He certainly demonstrated the flexibility of his spine by stepping up onto the chair with his hind legs so he was standing hind end up on the chair and front feet on the floor.

Generalization and Creativity
Jesús commented that if we didn’t know about resurgence we would be saying the dog generalized.  But generalization had nothing to do with it.  What we were seeing was resurgence.  Kay added that for her this process is what is meant by creativity.  It isn’t waiting and waiting for the dog to do something new.  Instead we give them a whole range of behaviors, and they come up with a new or unlikely combination.  What Jesús was showing us was a procedure for setting up the creative process.  You give the animal the repertoire, the components of more complex behaviors, and then you set up a puzzle and let extinction be the catalyst for solving it.

Coming soon: Part 15: Going Micro

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt. 13: Degrees of Freedom

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL
Part 11: Mastering Extinction
Part 12: Creativity Explored
Part 13: Degrees of Freedom

If you are new to this series, I suggest you begin with Part 1.

Part 13: Degrees of Freedom

Optimistic Puzzle Solvers
How do you help horses and handlers to become more optimistic puzzle solvers?  One way is to expand the repertoire of both the handler and the learner.  The broader and more extensive the repertoire, the more options an individual has.  If a horse knows only two choices and neither of them are working, he’s in trouble.

Jesús referred to this as being coerced by your repertoire.  Here’s the example: suppose a high school student is a great debater.  In fact he’s so good, he’s captain of the debating team. You’d expect someone like that to have a really high self-esteem. He’s so successful how could he not?

But look a little closer, and you’ll see why.  This individual is great at debating, but he’s no athlete.  He’s left out of a lot of other school events.  He doesn’t play sports.  He doesn’t go to school dances.  He has poor social skills so at lunch he’s off by himself.

Yes, in debating he wins all the prizes, but he has only that one skill.  So he’s being coerced into improving his debating skills because that’s all he can do.  He’s the best debater in the school, but that doesn’t keep him from feeling left out and miserable. With only that one skill he has only one degree of freedom.

Other members of the debating team may not be as good as he is, but they are also involved in other school activities.  Compared to him they have three or four degrees of freedom, and they are much happier.

The captain of the debating team is the best, but he’s been coerced into that position because he has no choices.  For him, as well as for our horses, the way to improve his emotional well-being is to expand his repertoire so he has more reinforcing activities available to him.

Kay Laurence confirmed this approach for dogs.  If you’re working with an aggressive dog, you want to expand his repertoire.  Teach him a dozen new behaviors: turning your head to the left, to the right, lifting a paw, walking in a circle, touching a target, etc..  Now in a threatening situation he has a dozen new ways to respond, instead of just the two or three that he started with.

Being Emotional Is Being Alive
Jesús dropped in another gem at this point by reiterating that when we talk about emotional behavior such as aggression, we are forgetting that we are always emotional. It isn’t that now we are happy, and then a switch turns off and we feel nothing.

“Living is being emotional.”

The nature and intensity of the emotions fluctuates.  We experience different degrees depending upon conditions and our reinforcement history. But thinking in terms of “emotional behavior” is too simplistic.  Emotion is part of all behavior.  It is not separate from it.

single suitcaseTraveling helps you to understand how much our emotions are a product of the habit patterns that have formed within our familiar environments and how universally present emotions are. Perhaps you are one of the huge number of people who have more to do than you could possibly accomplish in one day.  You have a family to care for, a house and barn to maintain, horses to feed and clean up after – not to mention ride.  All that and then there’s also an overfull schedule at work.  You’re always under stress, but it’s become so the norm, you don’t pay much attention to how you’re feeling.  A mildly stressed state is the normal background.

And then you treat yourself to the Five Go To Sea cruise where everything is different.  You still have a full day, with more to do and see than any one person could possibly squeeze into a day, but your normal triggers aren’t there.  The phone isn’t ringing.  You aren’t on the internet with the constant influx of work-related emails. Your co-worker’s voice coming through the office wall isn’t annoying you.  All those triggers are gone and now you get to experience who you are and how you feel without them.  You become acutely aware of just how stressed you’ve been now that you’ve stepped out of your normal habit patterns and can experience the contrast.  You’re still emotional, but now the environment is set up to trigger the kinds of supportive, pleasant emotions you want to experience.

That’s the kind of positive environment I want to create for my learners: one in which puzzle solving is fun, and both horses and handlers eagerly seek it out.

On a Caribbean cruise what do clicker enthusiats do for fun? They play PORTL games.

On a Caribbean cruise what do clicker enthusiats do for fun? They play PORTL games.

Coming Soon: Part 14: The Positive Side of Resurgence

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 11: Mastering Extinction

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL
Part 11: Mastering Extinction

If you have not yet read the previous articles, I suggest you begin with Part 1.

Part 11: Mastering Extinction

Using PORTL to Master Extinction
Extinction happens all the time.  When you withhold your click, you set up an extinction process.

If you withhold the click because you are unclear about your criteria or you’re clumsy in your handling skills, you could be setting up your learner for a macro extinction with all of the painful emotions that go along with it.

Or you could be withholding the click with very deliberate intent.  In this case you are using a micro extinction process to help shape a more complex behavior.  You are using extinction to your advantage.  The conclusion: extinction doesn’t have to be something you avoid.  It can be something you actively use to create more complex behavior patterns.

Today’s post shows how the shaping game, PORTL can help us understand how this works.

Jesús watching participants at a conference learning to play PORTL.

Jesús coaching participants at a conference as they learn how to play PORTL.

Shaping with Resurgence
Jesús used videos of PORTL experiments to illustrate what he meant.  He reminded us that there is a difference between resurgence and regression.  The first video example showed an elegant use of resurgence.

The learner was taught Behavior 1: tapping a small block.  Once that behavior was confirmed the block was removed and a toy car was placed on the table.

 

Behavior 2 was rolling the toy car over the table top.  When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it.

When that behavior appeared to be solid, the car was removed and a third object was placed on the table.  Now the behavior was lifting.

Behavior 4 was a different action.  The learner put a wooden ring on her finger.

When each of these behaviors seemed solid, the trainer reviewed one at a time what the learner was to do with each of the objects.

The trainer then placed all four objects out on the table but not in the order in which they had been taught.  There was now no reinforcement given.  The trainer was simply observing the learner’s behavior – not giving any feedback or reinforcement of any kind.  The point was to see in what order the learner would interact with each object.

The result:  The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.

So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.

The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them and then you remove reinforcement, the behaviors occur in the order in which they were trained.  This is resurgence as opposed to regression.

Shaping with Regression
Now here’s the fun part.  When you first extinguish the individual behaviors, you get the opposite result.  Now you see regression.  People will go back to the most recently learned behavior.  If that doesn’t work, they’ll go a little further back, and then a little further back – thus revealing their training history.

Jesús showed a second video, this one was exploring what happens in a shaping session where you reinforce an approximation, and then go through an extinction process so you can switch to a new behavior.  Here’s the set up:

The trainer set out one item on the table.  The learner began to manipulate it, trying to find out what was going to be clickable.  The trainer didn’t click any of this creativity.  She waited for it to extinguish and then clicked for one simple behavior – touching the object with one finger.  That was the “hot” action.

The trainer clicked and reinforced for successful approximations, then she took a break to record her data.  She continued to train in ten click units until she had achieved a high degree of consistency in touching the object with one finger.

This was the set up for the experiment.  In the next phase she set out in a circle nine or ten different objects, including the one they had been working with.  The learner began by touching the familiar object.  That got clicked and reinforced several times, then the trainer stopped reinforcing for that object.  She was using extinction to eliminate that behavior.  The learner began experimenting, touching various objects, but she only got clicked for touching the one that was immediately next to the previously hot object in the counter-clockwise direction.

The learner switched over to this object and began touching it consistently.

So now the handler stopped reinforcing for this object and only reinforced for the next object on the circle.  The learner again experimented and then discovered that the only object that she got paid for touching was the third one on the circle.

When this was consistent, the handler again stopped reinforcing for touching this object.  The learner was catching on to the overall pattern.  Now she moved more quickly to the fourth object and discovered that was the “hot” one to touch.

They continued counter-clockwise around the circle until every object had been the “hot” one once and touching it had also been extinguished.

At this point the handler stopped reinforcing for anything and observed the learner’s behavior. The result: the learner quickly switched to moving clockwise around the circle, touching  the objects in the reverse order in which she had learned them.  So she learned them originally counter-clockwise: object 1, then object 2, then object 3, then object 4, etc.

Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc.  She wasn’t getting clicked for any of these touches, but the pattern was very persistent.

What PORTL Reveals
So again: in the first video where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.

In the second video where behaviors were extinguished, the learner went through them in the reverse order.

You won’t find these distinctions in the scientific literature.  This difference in behavior relating to resurgence and regression is something Jesús and his students have been revealing by playing PORTL.

Coming Soon: Part 12: Creativity Explored

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 10: PORTL

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 10 of a 15 part series.
Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL

If you have not yet read the previous articles, I suggest you begin with Part 1.
Part 10: PORTL

PORTL’s Origins

Jesús and the son of one of the conference attendees playing PORTL during the Five Go To Sea cruise.

Jesús and the son of one of the conference attendees playing PORTL during the Five Go To Sea cruise.

PORTL evolved out of Genabacab, a table game Kay Laurence developed for teaching shaping.  Genabacab has very few instructions and really only one rule: the only person who is allowed to talk is the learner.  The trainer and spectators are not to give any verbal hints or to discuss what is going on until afterwards.

The table game lets you work out shaping plans BEFORE you go to your animal.  Do you want to learn how to attach a cue to a behavior and then change that cue to a new cue?  You can work out the process playing the table game and spare your animals the frustration of your learning curve.

Kay has described workshops at her training center where someone arrives with a “how do I teach this?” type of question. Maybe the handler wants to teach match to sample, or she wants to see if her dog can indicate which object is the biggest one in a set. Instead of going straight out to the dog and confusing it with missteps and false starts, everyone in the group will pull out their Genabacab games.  They will spend hours happily absorbed in developing the best teaching strategies for their dogs.  Their dogs, meanwhile, are spending the day relaxing while their people work away at the puzzle. It’s only once the process is well understood, that the dogs are brought in for training.

Jesús has been using Genabacab to help his students understand the concepts of learning theory.  He wants to bring the game to the scientific community as a research tool, so – with Kay’s blessing – he has renamed it.  It is now PORTL – Portable Operant Research and Teaching Laboratory.  Animal studies are increasingly difficult to do. They are expensive, and there is always the question of ethics.  How fair is it to run studies on lab rats?  PORTL is a much better solution.  You can have a question about how a particular process works, design an experiment using the PORTL game, and in the course of an evening have gathered enough data playing the game with a group of undergrads to write a paper – all without frustrating a single lab rat.  Now that’s progress!

PORTL Games
Jesús’ students meet on a regular basis to play PORTL games.  In his talk he showed some videos that illustrated beautifully how he used it to ask questions about regression and resurgence.  In one video two tasks were taught.  First, the learner was shaped to place one hand over the other – right hand over left, and then to reverse it – left hand over right.  The behavior was put on a fixed ratio of 5.  That means the learner was clicked and reinforced on every fifth swap of hands.

The second task was tapping a block.  Again, the learner was put on a fixed ratio of 5. (The learner was to tap the block five times for each click and treat.)

The trainer then increased the ratio for the tapping to 30.  The learner began  to tap the block, but now there was no click and treat after 5 taps.  The learner kept going to about 13 taps.  At that point she began to experiment.  She reverted back to swapping hands.  Then she tried a few more taps, before going back to hand swaps.  She tapped the block a few more times.  The trainer was still keeping track so each of these taps was counting towards the count of 30 she was looking for.

In the twenties the learner began to be creative.  She tried different ways to move hand over hand.  She’d go back and forth between experimenting with hand swaps and tapping the block.  Finally she reached a count of 30 at which point her handler clicked and reinforced her.  Jesús’ point is now all the extra gunk was also chained in. If the handler were to keep reinforcing the tapping of the block, she would also see the frequency of the hand swapping skyrocket.  That’s not the desired, goal behavior, but it would increase right along with the tapping.

Now you are probably thinking:  “Well that’s just poor training.  No one is going to jump from a fixed ratio of 5 to one of 30.”  My response would be to say that this can happen inadvertently. Suppose a handler has had a behavior on a high rate of reinforcement.  He’s asking the horse to carry himself in a correct bend.  He’s cueing it through gentle requests down the rein.  The horse is responding on a consistent basis, but then he’s distracted or the footing changes so he loses his balance. Whatever the reason, the handler isn’t getting the same consistent response. Instead he’s getting a string of unwanted behavior.  Sometimes the horse almost meets criterion, but not enough to click.  And then he comes through with the right answer.  The handler captures that moment with a click and a treat.  The question is: what is the long term result of that click?  Has the handler just identified a single clickable moment or has he chained in a long string of unwanted behavior?

The horse’s future responses will answer that particular question, but Jesús response in general is: if you want clean behavior, you need to learn to microshape.

Jesús made the further comment that this type of inadvertent chaining happens all too often when people are working with autistic children.  The sad thing here is the previous behavior that is being reinforced is not something harmless like hand swapping, but it’s often self-injurious behavior like head banging.

To sort out the tangle you need to analyze the whole behavior chain rather than focus in on individual behaviors.  These adjunctive behaviors can create a lot of stress. Again Jesús emphasized that’s why it is so important to understand extinction.  You need to understand it so you can master it.

Coming Soon: Part 11: Mastering Extinction

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 8: Training with High Rates of Reinforcement

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 8 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement

If you have not yet read the previous articles, I suggest you begin with Part 1.
Part 8: Training With High Rates Of Reinforcement

Raising Criteria
In a successful shaping session with horses it can seem as though they never stop eating.  That doesn’t mean that the criteria are never raised.  Quite the contrary.

In a good shaping session the next criterion you’re going to shift to is already occurring a high percentage of the time BEFORE you make that the new standard.

Nikita grown ups editedSuppose I’m working on having my horse stand politely next to me in the behavior I call: “grown-ups are talking please don’t interrupt”.  My horse is keeping his head consistently positioned so he is looking straight ahead.  I’ve decided that now I also want him to have his ears forward.  That’s a great goal, but if I abruptly stop clicking for good head position because the ears are back, guess what I’ll get – more pinned ears.  Why?  Because I’m frustrating my horse, and that emotion is expressed through pinned ears.

If I frustrate him too much, I’ll also get him swinging his head, nudging my arm, pawing etc. – all the behaviors that I thought I had extinguished as I was building my polite “grown-ups” behavior.

Using Resurgence
What is the solution?  I could begin by separating out ears from other criterion.  Every time I see this horse with his ears forward, click, I’ll reinforce him.  If I’m walking past his stall and he puts his ears forward as I go by, click, he’ll get a treat. Pretty soon I’ll see that my presence is triggering ears forward.  I’ve made it a “hot” behavior.

So now if I withhold the click in grown-ups, I’m likely to get a resurgence of the “hot” behaviors. I’m still using extinction, but I’ve set my horse up for success.  The behavior that is going to pop out is the one I’ve made “hot”, namely ears forward.

As Jesús kept saying: you have to understand extinction in order to master it.

Shaping with What is Already There
In fact, I probably won’t even shift my focus to ears forward until they already occurring at least some of the time.

I’m going to want my horse to stand in grown ups with his ears forward. That’s the goal, but at first, I’ll be happy if he simply takes his nose away from my treat pockets.  As I click him for keeping his head directly between his shoulders, some variability is going to come into the overall behavior.  Sometimes he’ll have his head slightly higher, or lower, his ears forward or back.  Initially I may be so busy monitoring the orientation of his head, I won’t even notice what he is doing with his ears.  But as his head position stabilizes and becomes more consistent, I’ll be able to take in more of the variations.

I’ll become increasingly aware of what he is doing with his ears.  If they are almost always pinned, there’s no point in making that the next criterion.  I’ll be surfing a long extinction wave before ears forward pops out.  In fact for something like ears, the more frustrated he becomes, the less likely they are to go forward.

Instead I’ll wait until his ears are popping forward frequently before I make that the clickable moment.  I’ll be withholding the click for a second or two while I wait for his ears to move.  My horse won’t be perceiving the event as anything negative.  The click will shift seamlessly to the new criterion.  That slight moment of extinction as I withhold the click causes my horse to surf through current “hot” behaviors. I’m using resurgence, but in a way that sets the horse up to have success build on success.

Coming soon: Cues and Extinction

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt 7: Emotions

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 7 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions

If you have not yet read the previous articles, I suggest you begin with Part 1.

Part 7: Emotions

Shaping With Micro Versus Macro Extinction
When someone is shaping and they want to raise the criterion, they stop reinforcing for a behavior that was just successful.  The learner goes through a resurgence/regression process.  She begins to offer other behaviors that have worked in the past.

People tend to think of extinction as happening over a long period of time, but Jesús kept emphasizing that it happens over seconds.  Two to three seconds is all you need for a mini extinction. You’ll begin to see the learner offering behavior other than the one that was previously reinforced.

Again this got my attention.  I don’t like the frustration you see when a puzzle appears to be unsolvable.  Shaping shouldn’t be marked by sharp drop offs in reinforcement. I don’t want to see macro extinctions.  If reinforcement is that sticky, it’s time to take a break.  Either put the horse away altogether while you go have a think, or regroup by shifting to another activity.  If you keep waiting, waiting, waiting until your learner finally gets close to the answer, you could lock in some unwanted behavior, and you will almost certainly lock in some unwanted emotions.

Emotions
Jesús pointed out that we use words such as “the animal was being emotional.”  But really what does that mean?   Jesús’ comment was: “we are always emotional.  It isn’t that the extinction process produces emotions.  All processes produce emotions.”

That’s such a good reminder.  We tend to think about emotions when they are the size of a five alarm fire, but really we are always “being emotional”.  There are emotions associated with all behaviors.  Ideally in training we’d like to avoid the five-alarm-fire type. That’s why it is so important to understand these processes.  The sooner you recognize that you are in an extinction process, the sooner you can do something to get out of it.

rat lever press cartoonIn extinction the individual (rat, human, horse, etc.) follows a predictable emotional pattern.

First, you see response bursting.  A rat has been reinforced consistently for pressing a lever.  Abruptly the lever pressing no longer works.  The rat will press the lever with even more energy trying to get it to work.  This has been equated with the classic hitting the button over and over again on the vending machine when your coke doesn’t fall out.

In the next stage you get angry.  Now you’re kicking the coke machine.

Next you see regression.  What behaviors have you seen modeled? What is your past history when things like this fail?

Then there is a pause followed by another period of response bursting.  Gradually the cycles become less until the individual settles into a calmer stage of acceptance.

Some psychologists have equated this pattern with the stages people go through when they are grieving.  When you lose a loved one, a job, a home, you are thrown into an extinction process.  Your loved one is gone.  No change in your behavior is going to bring back the one you’re grieving for.  Your reinforcers are gone and your behavior is ineffective.  Nothing you do will change the reality of your loss.

The stages of grief begin with denial, followed by anger, then depression, bargaining, and finally acceptance and a return to a meaningful life.

It’s interesting to see the comparison people make between these two processes. Understanding does bring with it coping skills.  If you understand the process you are in, you can keep things in perspective and find your way out of emotional tangles faster.  You can also be more understanding towards others (horse or human) if they are caught up in an extinction process.

Coming Soon: Part 8: Training With High Rates Of Reinforcement

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

5GoToSea: Pt.6: Accidental Extinction

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

This is Part 6 of a 15 Part series.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction

If you have not yet read the previous articles, I suggest you begin with Part 1.

Part 6: Accidental Extinction

Accidental Extinction
Natilie grown ups close up Extinction is not a rarity.  Extinction is going on all the time, but we aren’t always aware of it. Suppose you’re working with your horse. Perhaps you’re in the early stages of clicker training and the focus of your lesson is teaching your horse to keep his head in his own space, away from your treat pouch, a lesson I refer to as: “the grown-ups are talking, please don’t interrupt”.  What this implies is that you can stand next to your horse with your pockets full of treats while you carry on a conversation with someone, and your very polite clicker horse will be able to wait politely beside you.

To teach this behavior you’ve been asking your horse to stand quietly with his nose centered between his shoulders.  He’s been doing well.  You’re almost done with the session when your cell phone rings.   You answer it, taking your attention away from your horse.

Your horse doesn’t understand why you’ve abruptly disconnected from him.  You haven’t gone through a teaching process to explain to him that the ring tone of your cell phone is a cue for him to take a break.  Your horse doesn’t know this, and he doesn’t understand why the flow of your session has so abruptly changed.  He offers you a nice bit of grown-ups that meets all the previous criteria, but you aren’t paying any attention.  He doesn’t get clicked.  He tries harder, maybe throwing in some head lowering.  That doesn’t work either so he tries some earlier experiments – some head bobbing, some lip flapping, some gentle nudging, and finally a hard nudge followed by a nip.  That gets your attention, but now you’re thinking what an impatient, muggy horse you have!

Desperation Clicks
Your horse is offering “rude” behavior, bumping, nudging your arm, snuffling around your pockets.  He’s scrolling through the behaviors that he’s tried in the past.  You click something, anything, out of desperation.  What you are reinforcing is not just that single moment, but all the scrolling he’s been doing through his repertoire trying to get you to click. You have just chained in all those other unwanted behaviors.  chain with lockAnd just as a real chain locks things into place, so too, does this chain. You’ve just locked in those behaviors making it very likely that you’ll be seeing them again, especially under similar circumstances.  It’s going to be very hard to break that chain and discard those unwanted segments.  They’ve become an instant part of the whole sequence.

Jesús showed an interesting video from Michelle Pouliot.  Michelle is an excellent trainer.  She’s a guide dog trainer with thirty plus years of experience.  It’s been through her efforts that clicker training has been introduced into the Guide Dogs for the Blind training program.  Michelle also does freestyle with her dogs and has won many national titles, so to say that she’s a good trainer is an understatement.  I find it comforting that even trainers at this level of expertise can have an “oh oops!” happen in training.

Michelle was teaching a labrador to retrieve a dumbbell.  The dog had been successfully delivering the dumbbell to her, but now she wanted to raise the criterion and have the dog place it more firmly in her hands.  When the dog did not get reinforced for the usual behavior, it dropped the dumbbell, did a quick head bob, and then picked the dumbbell up again.  Just as Michelle clicked, the dog sat.  Oh oops!

Michelle lowered her criteria.  The dog handed her the dumbbell, but now he was also sitting as he did so.  Michelle’s hand reaching to take the dumbbell had in one click become a cue to sit.

Mini versus Maxi Extinctions
When the dog started offering behavior to get Michelle to click, that’s extinction.  We don’t tend to think of it in this way.  To develop the behavior we are training we want the offering of behavior.  Shaping depends upon differential reinforcement.  The dog offers a head bob, a paw lift, a sit.  In shaping we pick and choose from among these behaviors.  We think of extinction as something to be avoided. It’s a long drawn out process with lots of painful emotions associated with it.

What Jesús wanted us to understand was that the extinction process can occur in seconds. When you are shaping, you are working with mini extinctions.  When a learner is offering behavior, they are going through a resurgence process.  You don’t have to go hours or even minutes for the extinction process to begin.  It happens in seconds.

My ears perked up at this point in the talk.  I love this concept of mini extinctions.  It fits with microshaping and my own term – shaping on a point of contact.

Shaping on a point of contact involves pressure and release of pressure – but with this distinction.  The pressure remains at a level where it is information only.  It never escalates to become uncomfortable or fear-inducing.  It serves as a clue that helps the learner get to his click and treat faster.

Mini extinctions, micro shaping, and shaping on a point of contact are all learner-friendly because they make use of thin slicing and create high rates of reinforcement.

Microshaping
During the Five Go To Sea conference cruise, Kay Laurence gave a talk on microshaping.  She stressed that it’s not thin slicing alone that defines microshaping. It is high rates of reinforcement.  In microshaping she wants a success rate of 98% or higher. To get that you have to be very skilled at setting up the training environment. The learner is not surfing through a long series of behaviors trying to find the one that is “hot”. The design of the lesson provides very few opportunities for unwanted behavior to creep in. Instead it’s easy for the learner to offer correct responses.

Kay contrasts microshaping with what she refers to as sloppy or dirty shaping.  Here the handler lets the animal offer behavior after behavior looking for the one that will satisfy the criterion.  I’ve always been uncomfortable watching people freeshape in this clumsy fashion. They miss so many opportunities to click because they are looking for too much.  Now Jesús has helped me understand why this type of shaping makes me so uncomfortable.  Mini extinctions are part of puzzle solving.  But they are mini.  Success happens frequently so the frustration level stays low.  You could in fact see it as a positive motivator.  That little bit of: “is it this or is it that?” leads to a feeling of satisfaction each time you make the right choice.

Contrast that with macro Extinction.  Now it’s not “this, that, or maybe this other solution”. Nothing you try seems to work.  The frustration level rises to a level that takes away the fun.

Shaping Confusion

If you were the learner in a shaping game, what would you do with this tea cup?

If you were the learner in a shaping game, what would you do with this tea cup?

When you play shaping games with people and the “trainer” doesn’t have a clear plan, you’ll see this kind of frustration emerge.  Suppose the handler sets a teacup on the table.  Her goal is to get her learner to turn the tea cup upside down.  That’s a hard behavior to get because it’s not something we normally do with tea cups.  The learner gets clicked a couple of times for touching the teacup.  The teacup is clearly the “hot” item, but what is she supposed to do with it?  The learner tries turning the teacup, picking it up, passing it from hand to hand.  Nothing works.

She pretends to drink out of it, she spins it, she scoots it over the table.  Nothing gets clicked. Her frustration rises along with her unwillingness to play the game.  She’s in a macro extinction that can be painful to watch.  She goes back through the history she has with teacups.  What else can she try?  She feels like smashing the teacup over her trainer’s head, but social conventions keep her from offering that behavior!

When animals begin to scroll through behaviors they’ve learned in an attempt to get clicked, you are seeing an extinction process.  You are seeing a resurgence of previously reinforced behaviors.  In the teacup example, when the learner was no longer reinforced for just touching the teacup, when reinforcement for that behavior stopped, she tried things that she had done with tea cups or tea cuplike objects in the past.  Her handler was a new shaper.  She was outcome oriented, so she was looking for big macro responses.  She didn’t yet see the small steps that would take her easily to her goal behavior.  She didn’t yet know how to set aside the larger end goal while she taught her learner the small reaction patterns that would lead seamlessly to the desired result.

Teaching Via Reaction Patterns
In her talk on Microshaping Kay showed us some lovely examples of what it means to train for reaction patterns instead of end goals.

One of the clearest is the way Kay teaches a dog to go to a mat.  The end behavior she’s looking for has the dog going with energy to the mat, so she trains what she wants right from the start.  Instead of shaping general approximations where the dog sniffs at a mat, wanders around it, maybe sits or lies down, leaves and then comes back again, and every now and then puts a paw on it for a click and a treat, she starts out teaching an underlying base behavior.  In this case the base behavior is trotting back towards the handler after the dog has retrieved a tidbit of tossed food.

The dog retrieves it’s treat and then reorients back to the handler who is sitting in a chair.  As the dog trots towards her, the handler clicks and then tosses out another treat.  The dog turns and chases down this goody.  As soon as he has it, he reorients back towards the handler with the eager expectation that she’ll toss another treat.  As he comes back towards her, trotting with enthusiasm, she clicks again and tosses out another treat.

Shaping a dog to trot to a mat.

Shaping a dog to trot to a mat.

When this behavior is solid, she puts a mat down in the path of the dog.  As he comes trotting towards her, she clicks as his feet land on the mat.  It’s a clever way to get the behavior you want from the very first instant.

The dog experiences success after success.  He becomes a confident learner.  Instead of developing the mat behavior through trial and error learning where superstitious patterns may get linked in, he has learned it as a clean behavior.  Later if something stresses the behavior, what he will regress back to is this clean version of mat targeting, and it’s associated eager learning state.

Here’s a video of this shaping process:

Coming soon: Part 7: Emotions

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com