micro extinction

by

JOY FULL Horses: Understanding Extinction: Part 14

Our Creative Horses
Yesterday I shared with you the story of Robin’s “pose”.  The use of resurgence has helped us develop a much more systematic way of creating unlikely behaviors.  Because we understand the process better, we can be more deliberate in it’s use.  I ended the post by saying: “The end result may look like magic, but there is good science behind it.”

When we open up our training in this way and turn our learners into more active participants, we often find that they are even more creative than we are.  Once again Robin provided me with a great example of this.

When Robin was three I took him to the Equine Affaire to be my demo horse.  I wanted to show people what freeshaping via clicker training looks like.  I didn’t want them just to see the end product of freeshaping.  I wanted them to see me teach Robin a completely novel behavior.  The problem was he already had a pretty extensive repertoire. I was stumped for ideas, but I thought the easiest solution would be to use a prop.  One of my clients had been teaching his horse to flip a hula hoop up over his head.  I thought I could make a start on that with Robin.

Robin had been our first equine retriever.  Picking things up was solidly in repertoire.  I figured if I put the hula hoop on the ground, he would try to pick it up.  I’d be able to reinforce that and build it into Robin holding it longer which might over three days of demos lead to him flipping it over his head.  Such was my level of creativity, that’s all I could think of to work on with a hula hoop.

So during our demo, I brought out the hula hoop and tossed it out on the ground.  I was still explaining freeshaping to the audience so I wasn’t focusing yet on Robin.  While I was talking, he walked over to the hoop and stood with his front feet planted in the middle of it just as he would have stood on a mat.  Before I could respond to him, he reached down, picked up one side of the hoop and began walking himself forward foot by foot with the hoop!  That was his level of creativity!

The Creative Process
Here are the steps the horses have been teaching us:

First, you build a strong history of reinforcement for the component behaviors.

You change the situation somewhat so mild extinction comes into play.

You get a resurgence of these previously reinforced behaviors and new combinations emerge.  That’s creativity.  The most fun for me is seeing what the horses invent. As we have seen, they are often so much more creative than their human partners!

Familiar Landscapes
Kay Laurence might say we were seeing familiar landscapes with fresh eyes.

Dr. Jesús Rosales-Ruiz would say you have to understand the process of extinction before you can master it.  If you understand it, you’ll avoid situations that create macro extinction processes and all the frustration that goes along with them.  Instead you’ll use micro extinctions to build complex behaviors.

I would say that monitoring the level of extinction your learner is experiencing is a keys-to-the-kingdom part of good training.

I’ve just spent a couple of days working with a group of horses I have come to know well. One of them is a retired performance horse. Without going into a lot of details, I would describe him as an emotionally fragile horse. He’s easily worried.  If he thinks he has the right answer, he’s a superstar, but I always have to be careful how far I stretch him into new behaviors.  If he thinks he might get something wrong, he worries.  He’s come out of a training environment in which he had to perform correctly or his rider could get seriously hurt. I suspect he was punished for mistakes which accounts for his worry.

Mastering Micro
His back was looking prematurely aged so I wanted to teach him Robin’s “pilates pose”.  I had already shown him that he could get reinforced for lifting his back up and releasing at the poll.  In this particular session I was holding out for slightly better versions. As I withheld my click, I saw him experimenting.  Was it higher with his poll? Was it more lift of his back? What did I want?

The shifts he was giving me represented micro changes.  They were all within a clickable range.  Clicking him for any of these variations would have been fine, but I was waiting fractionally to see what else would pop out.

I was using micro extinctions to create the next step.  And because I was thinking about this in terms of extinction, I was monitoring closely his emotional responses.  I did not want him to become macro worried.  We were always just a second or two from a click so I could let him experiment without risking the emotional fallout of a larger extinction process.

Micro Masters
Micro is so very much the key.

Macro extinctions are frustrating.  Micro extinctions are part of good teaching.

Macro shaping can be confusing.  Micro shaping is elegant.

Macro negative reinforcement is literally painful. Micro negative reinforcement is clear communication. It is a conversation with cues exchanged in both directions.

When you go micro, your learner is always just a second or two away from a reinforceable moment.  You can cue another behavior, or you can simply click and treat. Either way, you are saying: “Yes! Great idea!” Micro Mastery is what we should be striving for in our training.  When you say someone is a great trainer, you are really saying that individual is a Micro Master.  In training that’s the “black belt” we should be aiming for.

robin-pg-lying-down-micro-masters

With this last section we come to the end of my JOY FULL Horses book – almost.  What remains is one final chapter and that’s what’s coming next.

Coming Next: Doorways

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY Full Horses: Understanding Extinction: Part 8

Mastering Extinction
Extinction happens all the time.  When you withhold your click, you set up an extinction process.

If you are unclear about your criteria or clumsy in your handling skills, you could be setting up your learner for a macro extinction process with all of the painful emotions that go along with it.

Or you could be using a micro extinction strategy to help shape a more complex behavior.  In this case you are using extinction to your advantage.  Extinction doesn’t have to be something you avoid.  It can be something you actively use to create more complex behavior patterns.

In yesterday’s post I described the PORTL games that Dr. Jesús Rosales-Ruiz  uses to help his students understand principles of behavior.  In his talks he shares some fascinating PORTL experiments to illustrate the difference between resurgence and regression.

Experiment One: Resurgence
The learner was taught a series of behaviors:

Behavior 1: tapping a small block. Once that behavior was confirmed, the block was removed and a toy car was placed on the table.

Behavior 2 was rolling the toy car over the table top.  When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it.  Pushing a car is an easy guess for what you would do with this kind of object.

When that behavior appeared to be solid, the car was removed and a third object, a key, was placed on the table.  Now the behavior was lifting.  Fingering a key is a normal response to this kind of object so it was easy to get the learner first to touch the key and then to lift it up off the table.  Once the learner was consistently lifting the key, that object was removed and a fourth one was introduced.

Behavior 4 involved the learner putting a wooden ring on her finger.  The learner quickly figured this out and began to consistently offer this behavior.

When each of these behaviors seemed solid – tapping the block, pushing the car, lifting the key, putting a ring on her finger – the trainer reviewed, one at a time, what the learner was to do with each of the objects.

The trainer then placed all four objects out on the table, but not in the order in which they had been taught.  The trainer observed the learner’s behavior.  She did not give any feedback or reinforcement of any kind.  The point was to see in what order the learner would interact with each object.

The result:  The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.

So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.

The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them, and then you remove reinforcement, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

Now here’s the fun part.  When you instead extinguish the individual behaviors, you get the opposite result.  Now you see regression.  The individual will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back – thus revealing his training history.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

These differences are illustrated in the second experiment.

Experiment Two: Regression
After a series of behaviors have been learned, this experiment again puts the learner through an extinction process.  In the initial set up each time the learner is moved on to a new task, an extinction process is used to eliminate the previous behavior.  Here’s the experiment:

The trainer sets out one item on the table.  The learner begins to manipulate it, trying to find out what is going to be clickable.  The trainer doesn’t click any of this creativity. She waits instead for it to extinguish and then clicks for one simple behavior – touching the object with one finger. That is the “hot” action.

The trainer clicks and reinforces for successful approximations until she has achieved a high degree of consistency in touching the object with one finger.

This was the set up for the experiment.  In the next phase she sets ten different objects out in a circle, including the one they had just been working with.  The learner begins by touching the familiar object.  That gets clicked and reinforced several times, then the trainer stops reinforcing for that object.  She is using extinction to eliminate that behavior.  The learner begins by experimenting, touching various objects, but she only gets clicked for touching the one that was immediately next to the previously hot object in a counter clockwise direction.

The learner switches over to this object and begins touching it consistently.

So now the handler stops reinforcing for this object and only reinforces for the next object on the circle.  The learner again experiments and then discovers that the only object that she gets paid for touching is the third one on the circle.

When this is consistent, the handler again stops reinforcing for touching this object.  The learner is catching on to the overall pattern. Now she moves more quickly to the fourth object and discovers that is the “hot” one to touch.

They continue counter clockwise around the circle until every object has been the “hot” one once and touching it has also been extinguished.

At this point the handler stops reinforcing altogether and simply observes the learner’s behavior.  The result: the learner quickly switches to moving clockwise around the circle, touching the objects in the reverse order in which she learned them.  So she learned them originally counter clockwise: object 1, then object 2, then object 3, then object 4, etc.

Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc.  She isn’t getting clicked for any of these touches, but the pattern is very persistent.

So again: in the first experiment where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.

In the second experiment where behaviors were extinguished, the learner went through them in the reverse order.

You won’t find these distinctions in the scientific literature. These two extinction outcomes, resurgence versus regression, are something Jesús and his students have been revealing by playing PORTL games.

Mind Games
Again Play is the key here.  PORTL may have a serious purpose behind it, but these are games.  All the creativity that comes with play is woven into these experiments.  It may turn out that others playing with similar set ups will have different results.  That’s a good thing.  That simply raises more questions, more puzzles to solve.

Do you have a question about how something works? Great. Design an experiment, test it a few times to work out the kinks in the procedure, and then invite your friends over for a pizza and PORTL party.  In the course of an evening you could have enough data to write a paper!

I do like the new twist Jesús has given to this version of the training game.  As he has pointed out, we’ve been using lab rats to learn about human behavior.  Now we are using humans to model animal behavior. Turnabout is fair play.  Much better to frustrate an undergrad than some poor lab rat!

Coming Next: Eureka Moments!  What is Insight?

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com

by

JOY FULL HORSES: Understanding Extinction – Part 5

Using “Hot” Behaviors

The Measure of Success
When horses are engaged in a successful shaping session, it can seem as though they never stop eating.  If you aren’t familiar with clicker training it can look as though the handler is constantly clicking and treating.  Don’t they ever stop feeding?  How is this going to work?  How do you raise criteria if you’re always feeding?

In a good shaping session the next criterion you’re going to shift to is already occurring a high percentage of the time BEFORE you make it the new standard.  Suppose I’m working on grown-ups, and I’ve decided that I want my horse to have his ears forward.  That’s a great goal, but if I abruptly stop clicking for good head position because the ears are back, guess what I’ll get – more pinned ears.  Why? Because I’m frustrating my horse and that emotion is expressed through pinned ears.

I’ll also get him swinging his head, nudging my arm, pawing etc., all the behaviors that I thought I had extinguished as I was building my grown-ups.

Using “Hot” Behaviors
What is the solution?  I could begin by separating out ears from other criterion. During casual exchanges when we aren’t in a formal training session, every time I see my horse with his ears forward, click, I’ll reinforce him.  If I’m walking past his stall and he puts his ears forward, click, he’ll get a treat. Pretty soon I’ll see that my presence is triggering ears forward.  I’ve made it a “hot” behavior.

So, now if I withhold the click in grown-ups, I’m likely to get a resurgence of “hot” behaviors.  I’m still using extinction, but I’ve set my horse up for success. The behavior that is going to pop out is the one I’ve recently made “hot” – in this case ears forward.

Click For What You Already Have
I won’t even shift my focus to ears forward until they are already occurring at a high frequency.  My goal is to have him standing beside me with his ears forward, but initially I’m happy if he simply takes his nose away from my arm.

As I click him for keeping his head directly between his shoulders, some variability is going to come into the overall behavior.  Sometimes he’ll have his head slightly higher, or lower, his ears forward or back.  I may be so busy monitoring the orientation of his head, I won’t even notice what he is doing with his ears.

As his head stabilizes and his overall orientation becomes more consistent, I’ll be able to take in more of these subtle variations. The movement of his ears pricking forward will catch my attention.  I’ll become increasingly aware of what he is doing with his ears.  If they are almost always pinned, there’s no point in making ears forward the next criterion.  I’ll be surfing a long extinction wave before ears forward pops out. In fact for something like ears, the more frustrated he becomes, the less likely they are to go forward.

So I’ll “prime the pump” instead.  I’ll make ears forward a hot behavior.  Now when he’s in grown-ups, if I make ears forward the next criterion, I’ll be withholding my click for only a second or two.  My horse won’t be perceiving the event as unpleasant or frustrating. The click will shift seamlessly to the new criterion.  That slight moment of extinction causes my horse to surf through current “hot” behaviors. I’m using resurgence, but in a way that sets the horse up to have success build on success.

Coming Next: Cues and Extinction

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com                    theclickercentercourse.com