operant conditioning

by

JOY FULL Horses: Unit 10 – Part 2 of 5: What We Say

JOYFULL Horses: Ten Things You Should Know About Cues: Number 10: Playing With Chains – Cues Evolve into Chains

What We Say
It’s ten p.m., an hour at which I should be heading off to bed, but I can’t leave yet.  I’m sitting in the faculty lounge at the Clicker Expo.  We’ve just come from dinner and a presentation by this year’s guest speaker.  After a full day of presentations you would think we would all be ready to call it a night, but instead we’re just getting warmed up.

Around the table with me are Dr. Susan Friedman, Ken Ramirez,  Eva Bertilsson, Kay Laurence, Dr. Jesús Rosales-Ruiz, and Laura Monico Torelli.

We are discussing terminology.  Eva got the ball rolling with a question about chains. We are wrestling with the different definitions of chains that are in use.

chain-2

Dr. Friedman is defining a chain from the perspective of a behavior analyst.  A chain has a very narrow and very specific meaning.  For a true chain, you give one cue that starts the process.  The next behavior is triggered by an internal cue.  It’s like dominoes.  You push over the first block and all the rest follow.

This type of chain can be very elegant to watch.  Imagine a series of agility obstacles set out in your arena.  You give your horse a cue that sends him out to the first obstacle, a small jump.  Just beyond the jump is a cone.  Your horse spots the cone as he clears the jump.  The cone itself serves as the cue for him to trot over to it, and pick it up.  Nearby is a large bucket.  He walks over to the bucket, drops the cone into the bucket.  A few feet past the bucket is a large platform.  Your horse now walks over to the platform, steps up onto it with all four feet, and lifts one foot high into the air while you click and run over with his treat.

That’s a technical chain.

Now imagine a different scenario.  You send your horse out over the first jump.  Just beyond the jump are two cones, a green one and a red one.  As your horse jumps, you shout “green”.  You’ve added a cue to tell your horse which cone he’s to pick up.  He heads straight over to the green cone, but now there are more choices.  Instead of one bucket, there are two identical ones, except one has a symbol of a circle painted on it, and the other a triangle.  As he picks up the green cone, you shout “circle”.  He walks over to the correct bucket and drops the cone in.

After this he again has more choices.  There are two platforms, one to the right and one to the left of the buckets.

You shout “Left”, and he walks over to the platform that’s off his left shoulder and steps up on it.

If you are using scientific terminology, this very sophisticated series of behaviors is not a chain because you are cueing each one.  It would be considered a sequence.

Our discussion rolled on around these two terms.  We all understood the distinctions.  The question was how fluid and flexible should we be with the language we use.

The Meaning of Words
In the field of learning theory scientists took for their own use many terms which already had a common-usage meaning.  Punishment is a great example.  When someone says we need to punish a child, a criminal, a terrorist, another country, the meaning is clear.  There is a moral element to it.  You don’t simply want to stop the behavior.  You want to impose a penalty.  You want the person to suffer in some way, to “pay” for his offense.  You are punishing the individual, not the behavior.

When a behavioral analyst uses the term, the meaning is very different.  There are no moral overtones of retribution.  If you smack a horse for biting, and the behavior decreases, you can say that the smack punished the biting behavior.  If the biting continues, the smack did not punish the behavior.  It may have annoyed or even frightened the horse, but if the behavior of biting didn’t decrease, the smack wasn’t a punisher.

When scientists take words that are already in common usage and redefine them, we can get a muddled result.  We also have confusion when scientists use words that we’re sort of familiar with, but not really.

A great example is operant conditioning.

That’s the big umbrella under which clicker training sits.  Operant sounds like operator.  And conditioning we understand from fitness programs.  But what do those two words put together really mean?

Look at what else happens when scientists start combining words we thought we understood.

Consider the four quadrants of operant conditioning: there’s positive and negative reinforcement, positive and negative punishment.

Positive punishment!?  Really.

Okay, the scientists explain.  The positive means simply that something has been added.  You’re adding something the horse doesn’t want and that stops the behavior, at least for the moment.  You add the smack of your hand when your horse bites you.

That’s clear enough, except it’s hard not to feel the harsh “take that” edge when you even just think about smacking your horse.  We can say we understand the plus and minus of the terms, but we still experience emotions we’ve come to associate with the words: positive equals good, negative equals bad.  Of course people get confused by these terms!  They understand them intellectually, but they experience them emotionally.  The only term that matches up and creates no conflict in meaning is “positive reinforcement”.  The rest get us into a real “knickers in a twist” state of confusion.

Negative Reinforcement
I was listening to the conversation, but I was also keeping an eye on my watch.  Eleven o’clock.  I had presentations to give the following day.  I should be calling it a night.  I decided to stay just a few more minutes.

Eva was asking more questions.  Now we were talking about negative reinforcement, a subject that always gets my attention given it’s connection to horse training.

When horses are handled with conventional training methods, rope handling is a very clear example of negative reinforcement.  The horse can avoid/escape the threat of escalating pressure by moving in the direction the handler wants.  As the horse learns to obey, the pressure diminishes to a subtle command.  The work looks soft, but the threat of escalation remains.  The soft command tells the horse how to avoid the escalating pressure.

Often people watch the finished result and think the trainer is very soft and kind.  This is very much a case of don’t judge a book by it’s cover.  The handler can look gentle because the horse understands the threat of escalating pressure that’s hidden inside every soft request.

That’s very straight forward.  If the handler is skilled, many horses thrive in this kind of system.  They know what they need to do to stay out of trouble. There’s no guess work. The commands are clear, the consequences are swiftly applied. Respond well, the pressure goes away. Fail to respond, and it escalates.  If you can figure out what is wanted – and if you can physically do it – you can stay out of trouble.

It’s easy to understand this kind of handling.  It’s textbook negative reinforcement.  And it’s also standard-issue horse training.

So what do we call it when the pressure doesn’t increase? When there is no threat of escalation, what is it?

I’ve always kept the use of the term negative reinforcement when I write about clicker-compatible rope handling.  I do this in part because I want to remember our history.  I want to remember where so many of the techniques that we use evolved from.  I want to remember so I won’t ever be tempted to go back there.

I have always combined pressure and release of pressure with the clicker.  You could say that I am simply piggy backing the clicker onto existing training systems, and that’s not really clicker training.

Perhaps, but it is a bridge.  If I am working with a rider who has spent years perfecting her horse-handling skills, I don’t want to say: “Throw all that away.  You won’t be using leads, or reins, or anything else you’re familiar with.”  That’s a great way to lose someone before they’re even out of the starting gate.

But if I say the communication system you know still works, we’re just going to teach it very differently, that makes more sense.  There’s still a huge learning curve, but I’m not going to begin by “throwing the baby out with the bath water.”

By the way do you know the derivation of that expression?  Before the modern era of indoor plumbing, baths were a rarity.  You brought water in and heated it for one bath.  The patriarch of the household took his bath first, followed in rank by everyone else.  The children would be the last ones to bathe.  By the time it was the turn of the youngest babies, the water would be murky brown.  You literally had to be careful not to throw the baby out with the bath water!

This derivation comes courtesy of the historian, Lucy Worsley and her wonderful book, “If Walls Could Talk, An Intimate History of the Home”.

Just as we still take baths – but my how they’ve changed – we still use lead ropes and other pressure cues in clicker training. But again – how things change when you take the threat away and make them clicker compatible!

Coming Next: Procedure versus The Emotional Effect

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

Are You A Clicker Trainer or a User of Clicker Training?

JOY FULL Horses: Ten Things You Should Know About Cues: Number 9.) You Can’t Not Cue: Part 4 of 12

Are You a Clicker Trainer?
I will say straight out – I am a clicker trainer.  But in 1993 when I first went out to the barn with treats and a clicker in my pocket, I was simply someone who was curious about clicker training.  I began, as we all do, by simply using clicker training.  Over time I became a clicker trainer.  What were the dots that had to connect up to turn me into a clicker trainer, and what does that mean?

There are a great many people who come across clicker training, take a quick look and never give it a try.  There are lots of reasons for this.  They may have been taught that you should never use treats in training; that the horses should work for you out of respect and because you have shown them that you are a good leader; that predators may work for rewards, but horses are grazing animals and it isn’t natural to hand feed them.

You may find yourself sputtering, wanting to say but, but, but this is all nonsense.  Save your breath.  If someone is deeply entrenched in these belief systems, no amount of evidence to the contrary is going to change their mind.  You’ll only get yourself worked up into a not very clicker-compatible argument.

If someone takes a look and walks the other way, don’t worry about it.  Clicker training doesn’t have to be everyone’s “cup of tea”.  Some people have to bump into clicker training a few times before it will attract their notice enough to give it a try.  Maybe the first horse they saw being clicker trained was still in the early stages and everything looked like a muddle.  But now they’ve seen a bit more, and they’re ready to give it a try.

What matters more than trying to argue someone into giving it a try is keeping the door open for those who get curious.

So what does finally begin to tip the balance?  What brings people to clicker training?

Why Clicker Train? The Science Foundation
For some the first attraction is that clicker training is science based.  It’s development can be traced back to B.F. Skinner’s work.  Now for some this is an instant turn off.  They’ve taken psych courses in school.  They equate Skinner with a cold and unfeeling approach to behavior.  I don’t want to get drawn into that argument.  What animal trainers took from his work can be simplified down into the ABCs of training.

That translates into this:

Antecedents are events and conditions that immediately precede Behavior.  The Behavior occurs, and it is followed by Consequences.  And it is the consequences which determine whether that behavior is more or less likely to occur again.

We tend to look at antecedents for causes.  We say “sit” and our dog sits.  It seems on the surface that it was the cue that caused the behavior.  But why did the dog respond to the cue?  Why did he sit?  Was it because he has learned that when he hears that word, if he plunks his rear end to the ground, good things happen?  You give him goodies and lots of desired attention.  That makes “sit” a true cue.

Or was it because he’s learned that if he doesn’t sit when he’s told to, he’s corrected?  You scold him as you jerk on his lead or push his rear end to the ground.  He sits the next time to avoid the negative consequences.  That makes “sit” a command.  Remember the difference?  Commands have a do it or else threat backing them up. Cues indicate opportunities for reinforcement. (Number 1: Cues Are Not Commands: Published Feb. 10, 2016: https://theclickercenterblog.com/2016/02/10/)

Reinforcers and punishers are the consequences that determine if a behavior is more or less likely to occur again.

The cues we use can be thought of as releasers.  Say “trot” to your horse and that tells him that changing gait into a trot is the fast track to reinforcement.

The cue triggers behavior.  What happens as a consequence of the behavior makes the animal more or less likely to repeat it in the future.

People often define clicker training as operant conditioning thinking they are differentiating clicker training from other forms of training.  Operant conditioning includes the study/use of punishment, as well as reinforcement.  Clicker trainers work hard to avoid the active use of punishment, but so do many good trainers.  What sets clicker training apart is the use of a marker signal paired with positive reinforcement.

Three Blind Men and the Elephant
When people talk about Skinner’s work, I am always reminded of the fable of the three blind men and the elephant.

Three blind men came upon an elephant.  The first felt the elephant’s tail.  “The elephant is like a rope,” he declared. The second blind man encountered the elephant’s leg.  “You are totally wrong.  The elephant is like a tree.”  The third blind man got a hold of the elephant’s trunk.  “What nonsense you are both talking.  The elephant is clearly like a snake!  Any fool can tell that.”

In the original fable the three blind men get into a fight because none of them could imagine that the others could be right, that depending upon their perspective they could each come to different conclusions.

What people take away from Skinner is very much like this.  Talk to some and you will hear that Skinner’s contributions to science are on a par with Darwin’s.  Others will say he held back progress in their field for decades.  For animal trainers Skinner’s work gave us the breakthrough we needed to communicate more clearly with our animals.  It gave us marker signals and with them the concept of shaping behavior.

skinner-with-dog-with-caption

The use of marker signals grew out of an unintended consequence.  When a rat pressed a lever, the automatic feeders made a clicking sound as food was released.  The click was originally just part of the apparatus, so you could say that all the innovations clicker training has brought us are the result of a happy accident.

Modern Animal Training
It is the norm to see something new, and at first to try to turn it back into something you are already familiar with.  So it is very understandable that people would come to very different conclusions about what Skinner was saying.  All of us who encounter his work bring our own perspective and biases to it.  What you take from it depends in part upon what you bring to it.

What animal trainers took from it was the power of the marker signal, and an understanding that it is consequences that drive behavior.

What has evolved is a modern science-based approach to training.  We aren’t just relying on anecdotal stories for choosing a particular training solution.  We can test our choices.  We can refer back to the studies being done by behavior analysts.  We can say, with data to back us up,  that punishment produces negative side effects

It’s the old joke – what’s the one thing three trainers can agree on?  That the fourth trainer is all wrong.  Everyone thinks their methods are the best.  With clicker training we can examine the statements we make about training.  We can design studies and produce data to help us understand why our animals respond in the way that they do.

We can look at different schedules of reinforcement, at reinforcement variability, at the effect of punishment on response, etc.  We aren’t following a particular system of training because someone tells us this is natural, or traditional, or the way it is always done.  As clicker trainers our “best practice” choices have evolved out of what research into behavior suggests really does work best.

Relationship
Science is what brought me to clicker training, but for many people that is not the principle draw.  Yes, it is reassuring that others have thought about schedules of reinforcement, etc. to develop current best practice, but what appeals to them is what grows out of this work – namely a great relationship.

Coming Next: Relationship

(And if you are wondering what happened to Poco, our ear-shy horse.  Don’t worry.  I am winding my way back to him.  When we get there, you will understand why I took this detour.)

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com