B.F. Skinner

by

Modern Horse Training: Connections

Everything is Connected to Everything Else – Including Dolphins and Horses

I stumbled across clicker training in 1993. A friend who bred and trained Irish Wolf hounds told me about Karen Pryor’s book, “Don’t Shoot the Dog”. I read it, loved it and wanted to learn more. What was this clicker training that she was talking about? How did it work?

I read her second training book, “Lads Before the Wind: Diary of a Dolphin Trainer” which described how she and her husband, Tap Pryor, developed Sea Life Park in Hawaii. Karen was given the task of figuring out how to train dolphins. It wasn’t anything she set out to do. The task fell to her because the people they had hired to train the dolphins weren’t getting anywhere. These trainers were trying without success to use old-style circus training methods. That didn’t work with an animal that could just swim away from them.

Karen was intrigued by the work that was coming out of B.F. Skinner’s labs. So armed with the lab notes from some of his graduate students, she set to work. She figured out how to use marker signals and positive reinforcement to train dolphins.

Her books were great. I loved both of them. I enjoyed “Lads” even more than “Don’t Shoot the Dog” because Karen shared the puzzle-solving aspect of training. But those books weren’t training manuals. They didn’t teach you HOW to train. They just teased you with the possibility that you could remove the threat of punishment that sits behind most horse training methods.

I was intrigued, but in 1993 other than Karen’s books, there weren’t any readily available resources for learning more about clicker training. Through a bit of luck, I did manage to find a VHS recording of a seminar that Karen produced. It included two short video clips showing animals being clicker trained. One was of an African bull elephant at the San Diego zoo being trained with protective contact to present his feet for trimming. The other was a 12 week old mastiff puppy who learned to sit and lie down in minutes without ever being touched.

The elephant was the most interesting. He had attacked his keepers on several occasions so the decision was made to that no one was allowed to go directly into his pen with him. That meant that for ten years he had not had any foot care. Clicker training was being used to see if they could teach him to orient to targets and present his feet through a small opening in the gate of his enclosure. During the video, you could hear Gary Priest, the director of training at that time, saying “I cannot impress upon you enough how aggressive this animal was, but he’s standing there cooperating for just the social attention and a bucket of food treats.” I watched that and thought – we in the horse world have a lot to learn. I was thinking of the twitches, the lip chains, the hobbles, etc. that I had seen people use to force horses to comply.

The other video showed the use of a treat held up above a puppy’s nose to get the puppy to sit. Within just a few clicks, the puppy was sitting, then lying down and staying down while the trainer walked around her. There was no pushing, no shoving, no use of force. It was simple, elegant training.

Those two videos were all I needed to be up and running. They gave me what I needed to go out to the barn to ask Peregrine what he thought about clicker training. He got the proverbial ball rolling, so it is fitting that the new book, Modern Horse Training, is coming out on the anniversary of his birthday, April 26.

A Perfect Fit

I could say that clicker training was a perfect fit for me. Or I could turn it around and say that I was a perfect fit for clicker training. There were no horse books out there to guide me, or even any other trainers I could visit to see how it was being used. I was on my own. But I was primed. To use the language of constructional training, I had the components that were needed to turn the idea of clicker training into a fully formed, detailed, soup to nuts training program.

So what were those components? What were the skills, the mindset, the repertoire that prepared me so well to embrace the idea of clicker training? I will say that I have met many others who shared similar components. For so many all that is missing is the understanding of marker signals. Give them that, and, like me, they are off and running. But for many others clicker training represents a huge shift in thinking. Can you really use food in training? Isn’t it a distraction? Won’t it teach your horse to bite? What do you do when your horse says: “No”? The old style of thinking dictates that you must punish unwanted behavior or your horse will become dangerous. “Fear of and fear for” becomes an underlying motivator even if it is not spoken of in that way.

We can begin with that same underlying motivator and end up with a very different result. That’s what I wrote about in yesterday’s post. You can also use treats in training and still stay wedded to the belief that unwanted behavior needs to be corrected.

Using positive reinforcement describes a procedure. What I’m addressing now is the question of what motivates your training decisions? Even kind people can end up choosing punishment because they are motivated by “fear for” risks. You’re afraid that your dog might rush out the front door and be hit by a car, so you use punishment to teach him to stay back when you open the door. That’s one example of how this plays out.

You don’t have to use punishment to solve this problem. There are other options. You begin by acknowledging that you are concerned for your dog’s safety, and then you search out solutions that are a match with your core ethics and the type of relationship you want with your dog.

The same applies to horses. Wanting to keep bad things from happening is a powerful motivator that can take us to some wonderful learner-friendly procedures.

Sometimes it’s okay to start out by running away from something. Clicker training teaches us how to reframe that so you begin to run TOWARDS the good things that you want. You stop focusing on the unpleasant outcomes that you don’t want and you teach instead all the good things you do want.

You don’t want your horse crowding into you, stepping on your toes, mugging your pockets, biting at your hands. You can certainly suppress these behaviors through the use of punishment. Or you can look at what you do want. When your horse is standing next to you, what does that look like? Can you describe what the “perfect version” of your horse would be doing? He’d be standing four on the floor, in his own space, with his head between his shoulders so his nose is well away from your pockets. The more detailed your description is, the easier it is to train what you want. Each element you describe becomes a lesson you can teach. What are his ears doing? Where is his head – level with his chest, down on the ground? You can shape all of this using a marker signal that is linked to positive reinforcement.

Those are nice sounding words, but again how do you make this work? What were some of the component skills that helped me transform clicker training from an interesting concept into a workable training program?

That’s tomorrow’s post. I’m splitting what was originally a much longer single post into two installments so it’s not too much to read in one sitting.

“Modern Horse Training: A Constructional Guide to Becoming Your Horse’s Best Friend” will be available April 26. It will be available as a hardcover, a paperback, and as an ebook. You’ll be able to oder it through my web site and also through Amazon and other booksellers.

by

JOY FULL Horses: Understanding Extinction: Part 9

Eureka Moments: What is Insight?

Using resurgence – Insight
Yesterday I shared several PORTL games developed by Dr. Jesús Rosales-Ruiz.   The games deliberately used extinction.  What was observed was this: when you have been consistently reinforcing behaviors as you establish them in repertoire, and you then remove all reinforcement for them, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

When you instead extinguish the individual behaviors during the teaching phase, you get a different result.  The student will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

So how does this help us?  How can we use this understanding to shape behavior?  To get the ideas rolling Jesús shared several video examples where resurgence was used to train complex, creative behaviors.

The first video came from Robert Epstein’s work. Epstein was B.F. Skinner’s last graduate student.  Together they were exploring the concept of “insight”.  How do we solve puzzles?  Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

Bird Brains
To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into a chamber with the box and the banana.  The banana was hung up out of reach.  The pigeon couldn’t peck the banana, so an extinction process began. There was a resurgence of previously trained behaviors.  The pigeon was able to push the box under the banana, get up on the box, and peck the banana.

https://www.youtube.com/watch?v=RWVoeMjVM7g

How did the pigeon solve this puzzle so quickly?  What is insight? What really is creativity?  Skinner and Epstein would say the pigeon could solve the problem because it had in its existing repertoire the necessary components.  Pigeons that had no experience pushing the box or jumping up on the box failed to solve the puzzle.

What is Creativity?
Jesús gives us a very process-oriented way thinking about this experiment.  This kind of complex puzzle solving was achieved through resurgence.  Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, the individual will struggle to solve the puzzle.  He will experience a much longer extinction process.  Macro extinction emotions will begin to surface, and you have to hope the subject has the persistence to become truly creative.

This is the kind of creativity that is truly stressful.  It’s much better to analyze the end goal – the complex behavior you want to train – break it down into all of it’s component tasks, and then train each of the components separately.  The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Persistence
Jesús’ comment was there is “nothing new under the sun”. The behaviors you try are all built out of things you’ve done before.  All the components of what appears to be a novel behavior have been trained in the past. So let’s consider what happens when a group of people are presented with a challenging puzzle.  When they begin experimenting and find that the usual, familiar things aren’t working, some will give up quickly.

Others will persist.  They will experiment with novel combinations of what they already know, but again most will quit if they don’t come up with a solution fairly quickly .

A few will keep trying until they stumble across a novel combination that works.  We call these people inventors and creators because they are persistent enough to find these novel combinations.  The discovery process can be a painful one, but once the new combination has been found, it’s easy for everyone else to copy the results.

I can absolutely relate to this.  Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off.  But put me in front of a computer that isn’t cooperating, and I shut down fast. There my experience has produced a different set of expectations. I’ve been in enough situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills.  I don’t have the programing background that makes wrestling with a software issue fun.  Extinction has gone too far and been too uncomfortable.  So in one situation I can be very persistent and creative.  In another I’m the one going through the classic cycle of emotions that macro extinction produces.

I know first hand both how much fun the creative process can be when the expectation of success is there.  And I also know how painful and unpleasant the extinction process is when that expectation is missing.

What I want to create for my learners is a feeling of confidence.  Whether horse or human, I want them to KNOW they can solve whatever training puzzle I throw at them. Build this expectation in early before others have taught them hard lessons about failure, and you get brilliant, enthusiastic, joyful individuals.  They are the optimists of this world.  Whether horse or human, they are fun to be around.  That’s what an understanding of these concepts helps us to create.

Coming Next: Degrees of Freedom

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

Using Clicker Training Versus Being a Clicker Trainer

JOY FULL Horses: Ten Things You Should Know About Cues: Number 9.) You Can’t Not Cue: Part 7 of 12

Using Clicker Training
Science, relationship, repertoire, persistence are the four main elements that go into the creation of clicker super glue. That was the focus of the previous post.  Put these four things together, and you will have someone who shifts from simply giving clicker training a quick look to someone who is actively using clicker training on a routine basis.  But that still doesn’t mean someone is a clicker trainer.

This is not a judgement about who is technically the better trainer.  You can be very skilled and consider yourself a user of clicker training, not a clicker trainer.  These labels refer more to the mindset that you bring to training and the impact that this has on both your training choices and your learner.

It can also be a description of where you are in the learning process.  No one starts out as a clicker trainer.  We all start out by taking a look and seeing if it is of interest.  Then we gradually move from seeing it as a tool, to seeing it more as the organizing framework for our training.

A great example of someone who actively uses clicker training – and uses it very well – but is not a clicker trainer would be Bob Bailey.  Bob has had a long and very distinguished career as a trainer.  In the fifties when open ocean work with dolphins was first being developed, he headed up the Navy’s training program.  He moved on to become the Project Manager and later Vice President and General Manager of Animal Behavior Enterprises, the company founded by Marian and Keller Breland, two of B.F. Skinner’s graduate students. In the early 1990’s when the dog community discovered clicker training, people were hungry for teachers.  They drew Bob out of retirement to give his now famous chicken training workshops.

Yes, you read that right – chicken training workshops.  Bob used chickens to teach people the science upon which clicker training is based.

Bob will tell you he uses clicker training because it is the most efficient, effective training method he knows, but if he found something that worked better, he would change in a heartbeat.  He is very much a user of clicker training.  By his own self-labeling, he is not a clicker trainer.

In a completely different category,  there are people who call themselves clicker trainers but whose understanding of what that means is light years away from what I mean.  Yes, they may click and treat, but they also cling to the need to punish their animals.  The dog gets a reward for sitting when he’s told to, but if he doesn’t sit fast enough – or worse – if he offers some other behavior, out come the corrections.  Using a clicker most definitely does not make you a clicker trainer.

The Clicker Umbrella

clicker umbrella 1
When I talk about clicker training, I often refer to the image of a huge umbrella under which a wide variety of training methods and solutions fit.  No one of these training strategies by itself defines clicker training.  You might rely heavily on targeting, but that is only one of many training strategies.  You could also use freeshaping or luring to form the behavior you want.

Pressure and release of pressure can fit comfortably under the umbrella.  If I want to figure out the answer to a treasure hunt, clues are welcome.  You’re getting warmer, you’re getting colder.  That’s the function of pressure in a clicker world.  The pressure is not escalated into a do-it-or-else threat.  It is information only.  It offers hints that help the learner get to the reinforcement faster.

If pressure remains at a level where it is information and never a threat, then even very traditional horse training techniques such as advance and retreat procedures can be modified and adapted to fit under my clicker umbrella.

So it isn’t the teaching strategy itself that determines if something fits under the clicker umbrella, but how it is used.  That includes not just pressure and release of pressure, but even targeting and feeshaping.  You can be using the tools of clicker training without really being a clicker trainer.  What does all this mean?  What is it that makes someone just a user of clicker training and not what I mean when I say someone is a clicker trainer?

Just Because You Can . . .
Ethics matter.  Here the mantra becomes:

Just because you can doesn’t mean you should.

Using a marker signal and treats, I could easily teach a horse to stay oriented between two targets.  If I slowly raise the targets up higher and higher, I can get the horse to rear.  With a little practice I could teach that horse to balance on his hind legs and walk the length of the arena.

Just because you can doesn’t mean you should.  Standing up on his hind legs like that can’t be good for the long term health of a horse’s hocks.  I might be able to teach this kind of circus-trick behavior, but I can’t imagine ever doing so.

You could easily get a yearling to jump over large fences at liberty, but again just because you can doesn’t mean you should.  The same considerations apply to older horses.  Should you be asking a horse with arthritic hocks to work at speed or to travel long distances on a horse trailer?  What we want and what our horses need are not always the same thing.

With the clicker you can train many things.  It’s not enough that you are using positive reinforcement to get a job done.  We need to consider not just HOW something is trained, but WHAT we are training.

There are lots of behaviors that look impressive, but they are hard on the individual.  It may simply be that the people who are teaching them have not fully thought out what they are doing.  They are still in the phase where they are excited by the behaviors they can train.  They aren’t yet looking at the broader picture of the animal’s long-term welfare.

Experienced clicker trainers include a consideration of balance – both physical and emotional – in everything they train. They are looking at how the behavior benefits the animal now and in the future.

just because you're using R+Good intentions are not enough.  Just because you are using positive reinforcement does not mean your animal is having a positive learning experience.  If you are fumbling around trying to get your treats out of your pocket, if your timing is off, or you are inconsistent in your criterion, your animal could be having a very frustrating time.  Instead of being clear, you’re surfing a giant extinction wave that leaves a wake of confusion behind you.

To prevent this your learner needs you to have:

  • the science to know how to create and carry out a shaping plan.
  • the relationship to care about his emotional well-being.
  • the repertoire to be adaptive to his learning needs.
  • the persistence to develop your own good handling skills.

That’s what creates clicker super glue and a complete clicker trainer.

Coming Next: More Questions

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

Are You A Clicker Trainer or a User of Clicker Training?

JOY FULL Horses: Ten Things You Should Know About Cues: Number 9.) You Can’t Not Cue: Part 4 of 12

Are You a Clicker Trainer?
I will say straight out – I am a clicker trainer.  But in 1993 when I first went out to the barn with treats and a clicker in my pocket, I was simply someone who was curious about clicker training.  I began, as we all do, by simply using clicker training.  Over time I became a clicker trainer.  What were the dots that had to connect up to turn me into a clicker trainer, and what does that mean?

There are a great many people who come across clicker training, take a quick look and never give it a try.  There are lots of reasons for this.  They may have been taught that you should never use treats in training; that the horses should work for you out of respect and because you have shown them that you are a good leader; that predators may work for rewards, but horses are grazing animals and it isn’t natural to hand feed them.

You may find yourself sputtering, wanting to say but, but, but this is all nonsense.  Save your breath.  If someone is deeply entrenched in these belief systems, no amount of evidence to the contrary is going to change their mind.  You’ll only get yourself worked up into a not very clicker-compatible argument.

If someone takes a look and walks the other way, don’t worry about it.  Clicker training doesn’t have to be everyone’s “cup of tea”.  Some people have to bump into clicker training a few times before it will attract their notice enough to give it a try.  Maybe the first horse they saw being clicker trained was still in the early stages and everything looked like a muddle.  But now they’ve seen a bit more, and they’re ready to give it a try.

What matters more than trying to argue someone into giving it a try is keeping the door open for those who get curious.

So what does finally begin to tip the balance?  What brings people to clicker training?

Why Clicker Train? The Science Foundation
For some the first attraction is that clicker training is science based.  It’s development can be traced back to B.F. Skinner’s work.  Now for some this is an instant turn off.  They’ve taken psych courses in school.  They equate Skinner with a cold and unfeeling approach to behavior.  I don’t want to get drawn into that argument.  What animal trainers took from his work can be simplified down into the ABCs of training.

That translates into this:

Antecedents are events and conditions that immediately precede Behavior.  The Behavior occurs, and it is followed by Consequences.  And it is the consequences which determine whether that behavior is more or less likely to occur again.

We tend to look at antecedents for causes.  We say “sit” and our dog sits.  It seems on the surface that it was the cue that caused the behavior.  But why did the dog respond to the cue?  Why did he sit?  Was it because he has learned that when he hears that word, if he plunks his rear end to the ground, good things happen?  You give him goodies and lots of desired attention.  That makes “sit” a true cue.

Or was it because he’s learned that if he doesn’t sit when he’s told to, he’s corrected?  You scold him as you jerk on his lead or push his rear end to the ground.  He sits the next time to avoid the negative consequences.  That makes “sit” a command.  Remember the difference?  Commands have a do it or else threat backing them up. Cues indicate opportunities for reinforcement. (Number 1: Cues Are Not Commands: Published Feb. 10, 2016: https://theclickercenterblog.com/2016/02/10/)

Reinforcers and punishers are the consequences that determine if a behavior is more or less likely to occur again.

The cues we use can be thought of as releasers.  Say “trot” to your horse and that tells him that changing gait into a trot is the fast track to reinforcement.

The cue triggers behavior.  What happens as a consequence of the behavior makes the animal more or less likely to repeat it in the future.

People often define clicker training as operant conditioning thinking they are differentiating clicker training from other forms of training.  Operant conditioning includes the study/use of punishment, as well as reinforcement.  Clicker trainers work hard to avoid the active use of punishment, but so do many good trainers.  What sets clicker training apart is the use of a marker signal paired with positive reinforcement.

Three Blind Men and the Elephant
When people talk about Skinner’s work, I am always reminded of the fable of the three blind men and the elephant.

Three blind men came upon an elephant.  The first felt the elephant’s tail.  “The elephant is like a rope,” he declared. The second blind man encountered the elephant’s leg.  “You are totally wrong.  The elephant is like a tree.”  The third blind man got a hold of the elephant’s trunk.  “What nonsense you are both talking.  The elephant is clearly like a snake!  Any fool can tell that.”

In the original fable the three blind men get into a fight because none of them could imagine that the others could be right, that depending upon their perspective they could each come to different conclusions.

What people take away from Skinner is very much like this.  Talk to some and you will hear that Skinner’s contributions to science are on a par with Darwin’s.  Others will say he held back progress in their field for decades.  For animal trainers Skinner’s work gave us the breakthrough we needed to communicate more clearly with our animals.  It gave us marker signals and with them the concept of shaping behavior.

skinner-with-dog-with-caption

The use of marker signals grew out of an unintended consequence.  When a rat pressed a lever, the automatic feeders made a clicking sound as food was released.  The click was originally just part of the apparatus, so you could say that all the innovations clicker training has brought us are the result of a happy accident.

Modern Animal Training
It is the norm to see something new, and at first to try to turn it back into something you are already familiar with.  So it is very understandable that people would come to very different conclusions about what Skinner was saying.  All of us who encounter his work bring our own perspective and biases to it.  What you take from it depends in part upon what you bring to it.

What animal trainers took from it was the power of the marker signal, and an understanding that it is consequences that drive behavior.

What has evolved is a modern science-based approach to training.  We aren’t just relying on anecdotal stories for choosing a particular training solution.  We can test our choices.  We can refer back to the studies being done by behavior analysts.  We can say, with data to back us up,  that punishment produces negative side effects

It’s the old joke – what’s the one thing three trainers can agree on?  That the fourth trainer is all wrong.  Everyone thinks their methods are the best.  With clicker training we can examine the statements we make about training.  We can design studies and produce data to help us understand why our animals respond in the way that they do.

We can look at different schedules of reinforcement, at reinforcement variability, at the effect of punishment on response, etc.  We aren’t following a particular system of training because someone tells us this is natural, or traditional, or the way it is always done.  As clicker trainers our “best practice” choices have evolved out of what research into behavior suggests really does work best.

Relationship
Science is what brought me to clicker training, but for many people that is not the principle draw.  Yes, it is reassuring that others have thought about schedules of reinforcement, etc. to develop current best practice, but what appeals to them is what grows out of this work – namely a great relationship.

Coming Next: Relationship

(And if you are wondering what happened to Poco, our ear-shy horse.  Don’t worry.  I am winding my way back to him.  When we get there, you will understand why I took this detour.)

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com