B.F. Skinner

by

JOY FULL Horses: Understanding Extinction: Part 9

Eureka Moments: What is Insight?

Using resurgence – Insight
Yesterday I shared several PORTL games developed by Dr. Jesús Rosales-Ruiz.   The games deliberately used extinction.  What was observed was this: when you have been consistently reinforcing behaviors as you establish them in repertoire, and you then remove all reinforcement for them, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

When you instead extinguish the individual behaviors during the teaching phase, you get a different result.  The student will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

So how does this help us?  How can we use this understanding to shape behavior?  To get the ideas rolling Jesús shared several video examples where resurgence was used to train complex, creative behaviors.

The first video came from Robert Epstein’s work. Epstein was B.F. Skinner’s last graduate student.  Together they were exploring the concept of “insight”.  How do we solve puzzles?  Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

Bird Brains
To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into a chamber with the box and the banana.  The banana was hung up out of reach.  The pigeon couldn’t peck the banana, so an extinction process began. There was a resurgence of previously trained behaviors.  The pigeon was able to push the box under the banana, get up on the box, and peck the banana.

How did the pigeon solve this puzzle so quickly?  What is insight? What really is creativity?  Skinner and Epstein would say the pigeon could solve the problem because it had in its existing repertoire the necessary components.  Pigeons that had no experience pushing the box or jumping up on the box failed to solve the puzzle.

What is Creativity?
Jesús gives us a very process-oriented way thinking about this experiment.  This kind of complex puzzle solving was achieved through resurgence.  Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, the individual will struggle to solve the puzzle.  He will experience a much longer extinction process.  Macro extinction emotions will begin to surface, and you have to hope the subject has the persistence to become truly creative.

This is the kind of creativity that is truly stressful.  It’s much better to analyze the end goal – the complex behavior you want to train – break it down into all of it’s component tasks, and then train each of the components separately.  The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Persistence
Jesús’ comment was there is “nothing new under the sun”. The behaviors you try are all built out of things you’ve done before.  All the components of what appears to be a novel behavior have been trained in the past. So let’s consider what happens when a group of people are presented with a challenging puzzle.  When they begin experimenting and find that the usual, familiar things aren’t working, some will give up quickly.

Others will persist.  They will experiment with novel combinations of what they already know, but again most will quit if they don’t come up with a solution fairly quickly .

A few will keep trying until they stumble across a novel combination that works.  We call these people inventors and creators because they are persistent enough to find these novel combinations.  The discovery process can be a painful one, but once the new combination has been found, it’s easy for everyone else to copy the results.

I can absolutely relate to this.  Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off.  But put me in front of a computer that isn’t cooperating, and I shut down fast. There my experience has produced a different set of expectations. I’ve been in enough situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills.  I don’t have the programing background that makes wrestling with a software issue fun.  Extinction has gone too far and been too uncomfortable.  So in one situation I can be very persistent and creative.  In another I’m the one going through the classic cycle of emotions that macro extinction produces.

I know first hand both how much fun the creative process can be when the expectation of success is there.  And I also know how painful and unpleasant the extinction process is when that expectation is missing.

What I want to create for my learners is a feeling of confidence.  Whether horse or human, I want them to KNOW they can solve whatever training puzzle I throw at them. Build this expectation in early before others have taught them hard lessons about failure, and you get brilliant, enthusiastic, joyful individuals.  They are the optimists of this world.  Whether horse or human, they are fun to be around.  That’s what an understanding of these concepts helps us to create.

Coming Next: Degrees of Freedom

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

Using Clicker Training Versus Being a Clicker Trainer

JOY FULL Horses: Ten Things You Should Know About Cues: Number 9.) You Can’t Not Cue: Part 7 of 12

Using Clicker Training
Science, relationship, repertoire, persistence are the four main elements that go into the creation of clicker super glue. That was the focus of the previous post.  Put these four things together, and you will have someone who shifts from simply giving clicker training a quick look to someone who is actively using clicker training on a routine basis.  But that still doesn’t mean someone is a clicker trainer.

This is not a judgement about who is technically the better trainer.  You can be very skilled and consider yourself a user of clicker training, not a clicker trainer.  These labels refer more to the mindset that you bring to training and the impact that this has on both your training choices and your learner.

It can also be a description of where you are in the learning process.  No one starts out as a clicker trainer.  We all start out by taking a look and seeing if it is of interest.  Then we gradually move from seeing it as a tool, to seeing it more as the organizing framework for our training.

A great example of someone who actively uses clicker training – and uses it very well – but is not a clicker trainer would be Bob Bailey.  Bob has had a long and very distinguished career as a trainer.  In the fifties when open ocean work with dolphins was first being developed, he headed up the Navy’s training program.  He moved on to become the Project Manager and later Vice President and General Manager of Animal Behavior Enterprises, the company founded by Marian and Keller Breland, two of B.F. Skinner’s graduate students. In the early 1990’s when the dog community discovered clicker training, people were hungry for teachers.  They drew Bob out of retirement to give his now famous chicken training workshops.

Yes, you read that right – chicken training workshops.  Bob used chickens to teach people the science upon which clicker training is based.

Bob will tell you he uses clicker training because it is the most efficient, effective training method he knows, but if he found something that worked better, he would change in a heartbeat.  He is very much a user of clicker training.  By his own self-labeling, he is not a clicker trainer.

In a completely different category,  there are people who call themselves clicker trainers but whose understanding of what that means is light years away from what I mean.  Yes, they may click and treat, but they also cling to the need to punish their animals.  The dog gets a reward for sitting when he’s told to, but if he doesn’t sit fast enough – or worse – if he offers some other behavior, out come the corrections.  Using a clicker most definitely does not make you a clicker trainer.

The Clicker Umbrella

clicker umbrella 1
When I talk about clicker training, I often refer to the image of a huge umbrella under which a wide variety of training methods and solutions fit.  No one of these training strategies by itself defines clicker training.  You might rely heavily on targeting, but that is only one of many training strategies.  You could also use freeshaping or luring to form the behavior you want.

Pressure and release of pressure can fit comfortably under the umbrella.  If I want to figure out the answer to a treasure hunt, clues are welcome.  You’re getting warmer, you’re getting colder.  That’s the function of pressure in a clicker world.  The pressure is not escalated into a do-it-or-else threat.  It is information only.  It offers hints that help the learner get to the reinforcement faster.

If pressure remains at a level where it is information and never a threat, then even very traditional horse training techniques such as advance and retreat procedures can be modified and adapted to fit under my clicker umbrella.

So it isn’t the teaching strategy itself that determines if something fits under the clicker umbrella, but how it is used.  That includes not just pressure and release of pressure, but even targeting and feeshaping.  You can be using the tools of clicker training without really being a clicker trainer.  What does all this mean?  What is it that makes someone just a user of clicker training and not what I mean when I say someone is a clicker trainer?

Just Because You Can . . .
Ethics matter.  Here the mantra becomes:

Just because you can doesn’t mean you should.

Using a marker signal and treats, I could easily teach a horse to stay oriented between two targets.  If I slowly raise the targets up higher and higher, I can get the horse to rear.  With a little practice I could teach that horse to balance on his hind legs and walk the length of the arena.

Just because you can doesn’t mean you should.  Standing up on his hind legs like that can’t be good for the long term health of a horse’s hocks.  I might be able to teach this kind of circus-trick behavior, but I can’t imagine ever doing so.

You could easily get a yearling to jump over large fences at liberty, but again just because you can doesn’t mean you should.  The same considerations apply to older horses.  Should you be asking a horse with arthritic hocks to work at speed or to travel long distances on a horse trailer?  What we want and what our horses need are not always the same thing.

With the clicker you can train many things.  It’s not enough that you are using positive reinforcement to get a job done.  We need to consider not just HOW something is trained, but WHAT we are training.

There are lots of behaviors that look impressive, but they are hard on the individual.  It may simply be that the people who are teaching them have not fully thought out what they are doing.  They are still in the phase where they are excited by the behaviors they can train.  They aren’t yet looking at the broader picture of the animal’s long-term welfare.

Experienced clicker trainers include a consideration of balance – both physical and emotional – in everything they train. They are looking at how the behavior benefits the animal now and in the future.

just because you're using R+Good intentions are not enough.  Just because you are using positive reinforcement does not mean your animal is having a positive learning experience.  If you are fumbling around trying to get your treats out of your pocket, if your timing is off, or you are inconsistent in your criterion, your animal could be having a very frustrating time.  Instead of being clear, you’re surfing a giant extinction wave that leaves a wake of confusion behind you.

To prevent this your learner needs you to have:

  • the science to know how to create and carry out a shaping plan.
  • the relationship to care about his emotional well-being.
  • the repertoire to be adaptive to his learning needs.
  • the persistence to develop your own good handling skills.

That’s what creates clicker super glue and a complete clicker trainer.

Coming Next: More Questions

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

Are You A Clicker Trainer or a User of Clicker Training?

JOY FULL Horses: Ten Things You Should Know About Cues: Number 9.) You Can’t Not Cue: Part 4 of 12

Are You a Clicker Trainer?
I will say straight out – I am a clicker trainer.  But in 1993 when I first went out to the barn with treats and a clicker in my pocket, I was simply someone who was curious about clicker training.  I began, as we all do, by simply using clicker training.  Over time I became a clicker trainer.  What were the dots that had to connect up to turn me into a clicker trainer, and what does that mean?

There are a great many people who come across clicker training, take a quick look and never give it a try.  There are lots of reasons for this.  They may have been taught that you should never use treats in training; that the horses should work for you out of respect and because you have shown them that you are a good leader; that predators may work for rewards, but horses are grazing animals and it isn’t natural to hand feed them.

You may find yourself sputtering, wanting to say but, but, but this is all nonsense.  Save your breath.  If someone is deeply entrenched in these belief systems, no amount of evidence to the contrary is going to change their mind.  You’ll only get yourself worked up into a not very clicker-compatible argument.

If someone takes a look and walks the other way, don’t worry about it.  Clicker training doesn’t have to be everyone’s “cup of tea”.  Some people have to bump into clicker training a few times before it will attract their notice enough to give it a try.  Maybe the first horse they saw being clicker trained was still in the early stages and everything looked like a muddle.  But now they’ve seen a bit more, and they’re ready to give it a try.

What matters more than trying to argue someone into giving it a try is keeping the door open for those who get curious.

So what does finally begin to tip the balance?  What brings people to clicker training?

Why Clicker Train? The Science Foundation
For some the first attraction is that clicker training is science based.  It’s development can be traced back to B.F. Skinner’s work.  Now for some this is an instant turn off.  They’ve taken psych courses in school.  They equate Skinner with a cold and unfeeling approach to behavior.  I don’t want to get drawn into that argument.  What animal trainers took from his work can be simplified down into the ABCs of training.

That translates into this:

Antecedents are events and conditions that immediately precede Behavior.  The Behavior occurs, and it is followed by Consequences.  And it is the consequences which determine whether that behavior is more or less likely to occur again.

We tend to look at antecedents for causes.  We say “sit” and our dog sits.  It seems on the surface that it was the cue that caused the behavior.  But why did the dog respond to the cue?  Why did he sit?  Was it because he has learned that when he hears that word, if he plunks his rear end to the ground, good things happen?  You give him goodies and lots of desired attention.  That makes “sit” a true cue.

Or was it because he’s learned that if he doesn’t sit when he’s told to, he’s corrected?  You scold him as you jerk on his lead or push his rear end to the ground.  He sits the next time to avoid the negative consequences.  That makes “sit” a command.  Remember the difference?  Commands have a do it or else threat backing them up. Cues indicate opportunities for reinforcement. (Number 1: Cues Are Not Commands: Published Feb. 10, 2016: https://theclickercenterblog.com/2016/02/10/)

Reinforcers and punishers are the consequences that determine if a behavior is more or less likely to occur again.

The cues we use can be thought of as releasers.  Say “trot” to your horse and that tells him that changing gait into a trot is the fast track to reinforcement.

The cue triggers behavior.  What happens as a consequence of the behavior makes the animal more or less likely to repeat it in the future.

People often define clicker training as operant conditioning thinking they are differentiating clicker training from other forms of training.  Operant conditioning includes the study/use of punishment, as well as reinforcement.  Clicker trainers work hard to avoid the active use of punishment, but so do many good trainers.  What sets clicker training apart is the use of a marker signal paired with positive reinforcement.

Three Blind Men and the Elephant
When people talk about Skinner’s work, I am always reminded of the fable of the three blind men and the elephant.

Three blind men came upon an elephant.  The first felt the elephant’s tail.  “The elephant is like a rope,” he declared. The second blind man encountered the elephant’s leg.  “You are totally wrong.  The elephant is like a tree.”  The third blind man got a hold of the elephant’s trunk.  “What nonsense you are both talking.  The elephant is clearly like a snake!  Any fool can tell that.”

In the original fable the three blind men get into a fight because none of them could imagine that the others could be right, that depending upon their perspective they could each come to different conclusions.

What people take away from Skinner is very much like this.  Talk to some and you will hear that Skinner’s contributions to science are on a par with Darwin’s.  Others will say he held back progress in their field for decades.  For animal trainers Skinner’s work gave us the breakthrough we needed to communicate more clearly with our animals.  It gave us marker signals and with them the concept of shaping behavior.

skinner-with-dog-with-caption

The use of marker signals grew out of an unintended consequence.  When a rat pressed a lever, the automatic feeders made a clicking sound as food was released.  The click was originally just part of the apparatus, so you could say that all the innovations clicker training has brought us are the result of a happy accident.

Modern Animal Training
It is the norm to see something new, and at first to try to turn it back into something you are already familiar with.  So it is very understandable that people would come to very different conclusions about what Skinner was saying.  All of us who encounter his work bring our own perspective and biases to it.  What you take from it depends in part upon what you bring to it.

What animal trainers took from it was the power of the marker signal, and an understanding that it is consequences that drive behavior.

What has evolved is a modern science-based approach to training.  We aren’t just relying on anecdotal stories for choosing a particular training solution.  We can test our choices.  We can refer back to the studies being done by behavior analysts.  We can say, with data to back us up,  that punishment produces negative side effects

It’s the old joke – what’s the one thing three trainers can agree on?  That the fourth trainer is all wrong.  Everyone thinks their methods are the best.  With clicker training we can examine the statements we make about training.  We can design studies and produce data to help us understand why our animals respond in the way that they do.

We can look at different schedules of reinforcement, at reinforcement variability, at the effect of punishment on response, etc.  We aren’t following a particular system of training because someone tells us this is natural, or traditional, or the way it is always done.  As clicker trainers our “best practice” choices have evolved out of what research into behavior suggests really does work best.

Relationship
Science is what brought me to clicker training, but for many people that is not the principle draw.  Yes, it is reassuring that others have thought about schedules of reinforcement, etc. to develop current best practice, but what appeals to them is what grows out of this work – namely a great relationship.

Coming Next: Relationship

(And if you are wondering what happened to Poco, our ear-shy horse.  Don’t worry.  I am winding my way back to him.  When we get there, you will understand why I took this detour.)

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com