December 13, 2016 by theclickercenter

JOY FULL Horses: Understanding Extinction: Part 12

Mastering Micro: Building Unlikely Behaviors with Resurgence
Nothing is either all good or all bad.

We want to use positive reinforcement with our animals because we see it as being both effective and more humane. But the associations created through positive reinforcement can create addictions to harmful behaviors. Think about the way advertisers manipulate our behavior to encourage smoking or overeating.

Resurgence and regression can be very negative procedures, but they can also be used to produce what might otherwise be very difficult behaviors to obtain.

If you aren’t sure how you can turn what seems like a negative procedure into a positive teaching strategy, PORTL can once again help to illustrate how this works.

Here’s the set up:

The trainer sets a toy chair on the table for her learner to interact with. The goal is to get the learner to push the chair over the table the way she might push a toy car.

We’ll now observe quietly in the background while the learner begins to interact with the chair. The trainer could get lucky. The learner might begin offering the behavior she’s after within the first couple of clicks. But with this learner there’s no sign of any chair pushing behavior. Why?

History matters.

The learner is going to draw on all of her previous repertoire of things she has done with chairs. In this case we have a learner who was scolded as a child for pushing her chair over the floor, so she’s not very likely to offer this type of behavior with the toy chair.

A history of punishment has played a role in depressing chair pushing behavior for this learner, but pushing would also have been an unlikely behavior if the trainer had set down a dice. The learner would have tossed the dice or shaken it in her hand because that’s what you do with this kind of object. Pushing a dice over the table like a toy car is not an obvious behavior to try.

Through a series of small approximations, the trainer could try to shaping the behavior she wants. Her first step would be reinforcing the learner for touching the chair.

The learner in this case is not particularly creative. She offers simple touches, but nothing else. Again, the trainer may be dealing with a history of punishment. Her learner doesn’t have a lot of experience being reinforced for trying things. In fact, quite the opposite – she may have been punished for stepping “outside the lines”. She is like so many of our animal learners – hesitant, lacking in confidence, and not showing any outward signs of curiosity. In her first few attempts she touches the chair, but she doesn’t try any other behaviors. Getting her to push the chair is going to be hard.

So the trainer takes the chair away and sets out a toy car. Using an object that normally would be pushed makes it very easy to get the desired action. The learner pushes the car over the table top. Click and treat.

This is repeated several times, and then the trainer takes the car away and sets the chair out. The learner goes back to touching it. The chair accidentally falls over – click and treat. The learner latches on to that, expanding her repertoire to two behaviors – touching the chair and knocking it over.

We see this so many times with our animal learners. One click and suddenly you’ve locked in a behavior you don’t want. With a creative learner this isn’t a problem. You can quickly shift the behavior into something you want, but with these “one trick ponies” you have to be so very careful what you click. In this case the learner persists in knocking the chair over even when she is no longer getting reinforced for the action.

Her trainer makes a quick decision and decides to put everything but pushing the chair like a car on extinction. Her learner is clearly becoming frustrated. To avoid having her shut down completely, the trainer takes the chair away and sets the car out again. The learner immediately starts pushing the car over the table top. Click and treat.

To help with the generalization the trainer puts a third object out – a small block. The learner pushes the block. Click and treat. This is repeated several times, then the trainer takes the block away and sets out the car. The car is pushed. Click and treat.

The trainer sets the chair out, and the learner pushes the chair. Job done.

Resurgence and Dog “Yoga”
Using the car in this way is an elegant teaching strategy. Often when we come up with these clever ways of helping our learner to be successful, we know that it works, but we don’t really have good explanations for why. Understanding resurgence helps us with the why in this case. And it helps us to be more deliberate in the use of this kind of teaching strategy. Here’s another example.

One of Kay Laurence’s students taught her dog to step up with his hind legs onto a chair. It was elegant training, a beautiful example of setting the learner up for success. In his talk on extinction, Dr. Jesús Rosales-Ruiz helped us to see that it was also a great example of using resurgence.

Here’s the lesson: First, the dog learned to stand one foot each on four small plastic pods. This alone was impressive training. The pods were the same ones physiotherapists use to help people improve their balance and proprioception. It took great coordination for the dog to stay balanced on the four pods. But that was only step 1. Next he learned to keep his front feet on the floor while he maneuvered his hind feet up onto the brick ledge of a fireplace hearth.

Adding in the precision of the pods came next. Now the dog wasn’t just standing with his front paws on the floor and his hind end up on the ledge. He was also balancing on all four pods.

This was not done as a cute party trick. The dog’s owner is a yoga teacher. Her interest was very much the same as mine – helping her animal learner maintain a healthy spine. In this orientation she could ask her dog for weight shifts that contribute to a flexible spine.

The last step was setting up a training session next to a chair. The handler withheld the click, putting the dog into an extinction process. With very little experimentation, the dog oriented himself so his hind end was to the chair. He certainly demonstrated the flexibility of his spine by stepping up onto the chair with his hind legs so he was standing hind end up on the chair and front feet on the floor.

Generalization and Creativity
Jesús commented that if we didn’t know about resurgence we would simply be saying the dog generalized. That’s not a sufficient explanation. What we were seeing was a great example of resurgence. PORTL has given us a better understanding of how to encourage this kind of problem solving. When we want to train for this type of generalization, knowing about the “why” of resurgence helps us to be more deliberate and efficient in our training.

It isn’t positive reinforcement by itself that creates a positive learning experience. An eagerness for learning comes from being a successful puzzle solver. That success in turn comes from the kind of efficient, clean training that the clever use of resurgence encourages.

These examples give us a great perspective on creativity. When we’re training, we aren’t waiting and waiting for our animals to do something we can reinforce. Instead we can “seed” the behaviors we want them to draw on. Then we set up the conditions and let them have the pleasure of discovering for themselves new or unlikely combinations.

We have a procedure for setting up the creative process. You give your learner the repertoire, the components that form more complex behaviors, and then you set a puzzle and let extinction be the catalyst for solving it.

Coming Next: The “Pose”

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends. But please remember this is copyrighted material. All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra Kurland, via theclickercenter.com

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training. If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com theclickercentercourse.com

December 10, 2016 by theclickercenter

JOY FULL Horses: Understanding Extinction: Part 9

Eureka Moments: What is Insight?

Using resurgence – Insight
Yesterday I shared several PORTL games developed by Dr. Jesús Rosales-Ruiz. The games deliberately used extinction. What was observed was this: when you have been consistently reinforcing behaviors as you establish them in repertoire, and you then remove all reinforcement for them, you get a resurgence of these previously reinforced behaviors. They reoccur in the order in which they were trained.

When you instead extinguish the individual behaviors during the teaching phase, you get a different result. The student will go back to the most recently learned behavior. If that doesn’t work, he’ll go a little further back, and then a little further back.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses. The most recently taught behavior reappears first.

So how does this help us? How can we use this understanding to shape behavior? To get the ideas rolling Jesús shared several video examples where resurgence was used to train complex, creative behaviors.

The first video came from Robert Epstein’s work. Epstein was B.F. Skinner’s last graduate student. Together they were exploring the concept of “insight”. How do we solve puzzles? Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

Bird Brains
To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into a chamber with the box and the banana. The banana was hung up out of reach. The pigeon couldn’t peck the banana, so an extinction process began. There was a resurgence of previously trained behaviors. The pigeon was able to push the box under the banana, get up on the box, and peck the banana.

How did the pigeon solve this puzzle so quickly? What is insight? What really is creativity? Skinner and Epstein would say the pigeon could solve the problem because it had in its existing repertoire the necessary components. Pigeons that had no experience pushing the box or jumping up on the box failed to solve the puzzle.

What is Creativity?
Jesús gives us a very process-oriented way thinking about this experiment. This kind of complex puzzle solving was achieved through resurgence. Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, the individual will struggle to solve the puzzle. He will experience a much longer extinction process. Macro extinction emotions will begin to surface, and you have to hope the subject has the persistence to become truly creative.

This is the kind of creativity that is truly stressful. It’s much better to analyze the end goal – the complex behavior you want to train – break it down into all of it’s component tasks, and then train each of the components separately. The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Persistence
Jesús’ comment was there is “nothing new under the sun”. The behaviors you try are all built out of things you’ve done before. All the components of what appears to be a novel behavior have been trained in the past. So let’s consider what happens when a group of people are presented with a challenging puzzle. When they begin experimenting and find that the usual, familiar things aren’t working, some will give up quickly.

Others will persist. They will experiment with novel combinations of what they already know, but again most will quit if they don’t come up with a solution fairly quickly .

A few will keep trying until they stumble across a novel combination that works. We call these people inventors and creators because they are persistent enough to find these novel combinations. The discovery process can be a painful one, but once the new combination has been found, it’s easy for everyone else to copy the results.

I can absolutely relate to this. Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off. But put me in front of a computer that isn’t cooperating, and I shut down fast. There my experience has produced a different set of expectations. I’ve been in enough situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills. I don’t have the programing background that makes wrestling with a software issue fun. Extinction has gone too far and been too uncomfortable. So in one situation I can be very persistent and creative. In another I’m the one going through the classic cycle of emotions that macro extinction produces.

I know first hand both how much fun the creative process can be when the expectation of success is there. And I also know how painful and unpleasant the extinction process is when that expectation is missing.

What I want to create for my learners is a feeling of confidence. Whether horse or human, I want them to KNOW they can solve whatever training puzzle I throw at them. Build this expectation in early before others have taught them hard lessons about failure, and you get brilliant, enthusiastic, joyful individuals. They are the optimists of this world. Whether horse or human, they are fun to be around. That’s what an understanding of these concepts helps us to create.

Coming Next: Degrees of Freedom

theclickercenter.com theclickercentercourse.com

December 9, 2016 by theclickercenter

JOY Full Horses: Understanding Extinction: Part 8

Mastering Extinction
Extinction happens all the time. When you withhold your click, you set up an extinction process.

If you are unclear about your criteria or clumsy in your handling skills, you could be setting up your learner for a macro extinction process with all of the painful emotions that go along with it.

Or you could be using a micro extinction strategy to help shape a more complex behavior. In this case you are using extinction to your advantage. Extinction doesn’t have to be something you avoid. It can be something you actively use to create more complex behavior patterns.

In yesterday’s post I described the PORTL games that Dr. Jesús Rosales-Ruiz uses to help his students understand principles of behavior. In his talks he shares some fascinating PORTL experiments to illustrate the difference between resurgence and regression.

Experiment One: Resurgence
The learner was taught a series of behaviors:

Behavior 1: tapping a small block. Once that behavior was confirmed, the block was removed and a toy car was placed on the table.

Behavior 2 was rolling the toy car over the table top. When the car was brought out for the first time, there was a small extinction burst of tapping the car, but the learner quickly shifted to pushing it. Pushing a car is an easy guess for what you would do with this kind of object.

When that behavior appeared to be solid, the car was removed and a third object, a key, was placed on the table. Now the behavior was lifting. Fingering a key is a normal response to this kind of object so it was easy to get the learner first to touch the key and then to lift it up off the table. Once the learner was consistently lifting the key, that object was removed and a fourth one was introduced.

Behavior 4 involved the learner putting a wooden ring on her finger. The learner quickly figured this out and began to consistently offer this behavior.

When each of these behaviors seemed solid – tapping the block, pushing the car, lifting the key, putting a ring on her finger – the trainer reviewed, one at a time, what the learner was to do with each of the objects.

The trainer then placed all four objects out on the table, but not in the order in which they had been taught. The trainer observed the learner’s behavior. She did not give any feedback or reinforcement of any kind. The point was to see in what order the learner would interact with each object.

The result: The learner went first to object 1/behavior 1, then moved to object 2/behavior 2, then object 3/behavior 3/and finally object 4/behavior 4.

So even though that wasn’t the left to right order in which the objects were set out, that was the order in which the learner interacted with them.

The conclusion: when you have not gone through an extinction process for the behaviors you are using, when you have instead reinforced them, and then you remove reinforcement, you get a resurgence of these previously reinforced behaviors. They reoccur in the order in which they were trained.

Now here’s the fun part. When you instead extinguish the individual behaviors, you get the opposite result. Now you see regression. The individual will go back to the most recently learned behavior. If that doesn’t work, he’ll go a little further back, and then a little further back – thus revealing his training history.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses. The most recently taught behavior reappears first.

These differences are illustrated in the second experiment.

Experiment Two: Regression
After a series of behaviors have been learned, this experiment again puts the learner through an extinction process. In the initial set up each time the learner is moved on to a new task, an extinction process is used to eliminate the previous behavior. Here’s the experiment:

The trainer sets out one item on the table. The learner begins to manipulate it, trying to find out what is going to be clickable. The trainer doesn’t click any of this creativity. She waits instead for it to extinguish and then clicks for one simple behavior – touching the object with one finger. That is the “hot” action.

The trainer clicks and reinforces for successful approximations until she has achieved a high degree of consistency in touching the object with one finger.

This was the set up for the experiment. In the next phase she sets ten different objects out in a circle, including the one they had just been working with. The learner begins by touching the familiar object. That gets clicked and reinforced several times, then the trainer stops reinforcing for that object. She is using extinction to eliminate that behavior. The learner begins by experimenting, touching various objects, but she only gets clicked for touching the one that was immediately next to the previously hot object in a counter clockwise direction.

The learner switches over to this object and begins touching it consistently.

So now the handler stops reinforcing for this object and only reinforces for the next object on the circle. The learner again experiments and then discovers that the only object that she gets paid for touching is the third one on the circle.

When this is consistent, the handler again stops reinforcing for touching this object. The learner is catching on to the overall pattern. Now she moves more quickly to the fourth object and discovers that is the “hot” one to touch.

They continue counter clockwise around the circle until every object has been the “hot” one once and touching it has also been extinguished.

At this point the handler stops reinforcing altogether and simply observes the learner’s behavior. The result: the learner quickly switches to moving clockwise around the circle, touching the objects in the reverse order in which she learned them. So she learned them originally counter clockwise: object 1, then object 2, then object 3, then object 4, etc.

Now she was touching them clockwise: object 10 – object 9 – object 8 – object 7, etc. She isn’t getting clicked for any of these touches, but the pattern is very persistent.

So again: in the first experiment where the behaviors were taught, but not extinguished, the learner went through them in the order in which they had originally been learned.

In the second experiment where behaviors were extinguished, the learner went through them in the reverse order.

You won’t find these distinctions in the scientific literature. These two extinction outcomes, resurgence versus regression, are something Jesús and his students have been revealing by playing PORTL games.

Mind Games
Again Play is the key here. PORTL may have a serious purpose behind it, but these are games. All the creativity that comes with play is woven into these experiments. It may turn out that others playing with similar set ups will have different results. That’s a good thing. That simply raises more questions, more puzzles to solve.

Do you have a question about how something works? Great. Design an experiment, test it a few times to work out the kinks in the procedure, and then invite your friends over for a pizza and PORTL party. In the course of an evening you could have enough data to write a paper!

I do like the new twist Jesús has given to this version of the training game. As he has pointed out, we’ve been using lab rats to learn about human behavior. Now we are using humans to model animal behavior. Turnabout is fair play. Much better to frustrate an undergrad than some poor lab rat!

Coming Next: Eureka Moments! What is Insight?

theclickercenter.com theclickercentercourse.com

December 2, 2016 by theclickercenter

JOY Full Horses: Understanding Extinction: Part 2

Yesterday’s post ended with a quote from Dr. Jesús Rosales-Ruiz: “If you don’t understand extinction, you won’t be able to master it.” Today’s post will begin to unravel what that means.

Regression and Resurgence
I’ve talked about regression in previous posts. Now we need to add in resurgence and the distinction between them.

In regression you revert back to previously extinguished behaviors.

In resurgence you revert back to previously reinforced behavior.

This isn’t just semantics. According to Jesús regression and resurgence emerge out of different training strategies and produce different outcomes.

Regression can be defined as: “If the present behavior is not capable of getting reinforcement, one reverts to older forms of response which were once effective, but which have previously been extinguished.” The order in which this unfolds is significant.

Under stress you will revert to an older way of behaving. If that behavior is not reinforced, you’ll go through another extinction process. You’ll revert back to even older behaviors. You’ll keep trying things and trying things, until you either give up entirely, or you are pushed to creativity. This can be a stressful process which is why some people equate creativity with an unpleasant experience. If you were to suggest to them that they take a creative writing class, they would be running for the hills! In their experience there’s nothing fun about being creative. How very sad!

Extinction History
Regression emerges because a behavior which normally earns reinforcement is no longer working. Often we think of extinction as simply a procedure that’s intended to reduce behavior. You don’t like a dog’s barking so you never reinforce it in the hope that the behavior will go away. This simplistic view misses an important key to understanding how to use extinction. A dog that isn’t barking is still doing something. What is the “something” that takes the place of the barking? The behaviors that emerge in an extinction process are not random. Understanding the order lets you master the process.

Jesús described extinction as the mirror image of reinforcement.

Extinction tells you what was reinforced in the past.

Reinforcement tells you what behaviors you are building for the future.

I wrote about this is previous posts. (https://theclickercenterblog.com/2016/11/17/) When you are first learning about clicker training, if your handling confuses the horse and puts him into an extinction process, you are revealing his past. You may see his frustration expressed as pawing, pinned ears, even occasionally biting. Don’t blame yourself for the outburst. You didn’t create the behavior you’re now dodging. Turn your spotlight instead on his past. That’s where the behavior was learned.

The Catalyst, Not the Cause
You may be the catalyst, but you are not the cause. That’s good news. You don’t have to take his behavior personally. The cause sits not in the present, but in the past. It’s only natural to become worried by the emotional reaction you’re seeing. People sometimes inadvertently end up compounding the problem. If their handling skills are clumsy or they don’t yet know how to manage the environment, they can put the horse into even more of an extinction process.

I’ve seen this in beginner handlers. They don’t yet understand how much a lack of clear criteria can impact a learner. Everything starts out so wonderfully. The horse gets clicked and reinforced for touching a target. What fun! But then the handler gets distracted. She misses three or four clickable moments. Those missed clicks can put the horse into an extinction process that leads to emotional outbursts. That’s where video cameras can be so useful. Video helps the handler see the training from the horse’s point of view. It reveals the good tries he’s offering and helps the handler understand more clearly what needs to be reinforced.

The solution to the extinction puzzle lies in embracing clicker training. Through clicker training you’ll be building a repertoire of behaviors that gives the horse alternatives to his old patterns. Instead of reverting back to behaviors you don’t want, now the extinction process will be popping out behaviors you’ve planted, behaviors you like and that you can reinforce. Suddenly, you aren’t in an extinction process anymore. You’re back on track with high rates of reinforcement.

This will take bit more unraveling of the extinction puzzle to understand.

Coming Next: Extinction Reveals the Past

theclickercenter.com theclickercentercourse.com

December 1, 2016 by theclickercenter

JOY FULL Horses: Understanding Extinction: Part 1

Today’s post begins the final chapter of JOY Full Horses, but don’t worry about the book ending too soon. I still have a month’s worth of posts to go. The first installment went up January 3, 2016. I thought maybe I’d be done around April. Ha! I hadn’t realized how much work was going to be involved in putting up each post. My goal is to finish by January 3, 2017. We’ll see if I make it. In the meantime, if you have friends who would enjoy reading the JOY Full Horses blogs, do share the links. The more the merrier in this Holiday season!

Enjoy this next installment.

Extinction and Shaping
In the last couple of posts I’ve been sharing with you a presentation given by Dr. Jesús Rosales-Ruiz on extinction. Many clicker trainers will say they never use extinction. I know I work hard to set up my training so horses aren’t put into the kind of guessing game that leads to outbursts of frustration and aggression. That’s something I very much want to avoid. But that doesn’t mean I don’t use extinction. That’s what Dr. Jesús Rosales-Ruiz makes so clear when he talks about resurgence and regression.

To the people who say they never use extinction, his response is: “What do you mean you never use extinction! Extinction is at the heart of shaping. Shaping is differential reinforcement. It’s the interplay between positive reinforcement and extinction. So if someone says they aren’t using extinction, probably they don’t understand what they are saying.”

That’s such a wonderfully blunt and typical Jesús comment. Thankfully he doesn’t leave people just with that. He goes on to explain what he means. At the core of his presentations is this statement: “If you don’t understand extinction, you won’t be able to master it.”

So what does it means to master extinction, and how do you put it to use in a positive way? That’s what I’ll be exploring in the coming posts.

Coming Soon: Regression and Resurgence

theclickercenter.com theclickercentercourse.com

November 19, 2016 by theclickercenter

JOY FULL Horses: Animal Emotions

Emotions: To Feel or Not To Feel – That Is The Question
When you begin talking about animal emotions, emotions run high. The belief systems that have grown up around emotions are truly amazing. In the past people have denied that animals even feel emotions. They’ll tell you animals may feel pain, but they aren’t really aware that they feel pain.

Wait a minute. What are they saying!? That just made my head spin.

What nonsense. Clearly these people have never been on a thoroughbred. Thoroughbreds are wonderfully emotional creatures. That’s their charm. They let you know everything they are feeling – the excitement, the fear, the worry, the joy. They truly “wear their emotions on their sleeve”. To say that these wonderful horses are not aware of their emotions is nonsense.

With his work on the seven core Affective circuits, Jaak Panksepp has helped bring the discussion of emotions “out of the closet”. Suddenly talking about emotions is the “in” thing. If an animal is being “too emotional”, people will tell you you’re clearly doing something wrong in your training.

Wait a minute. What did you say? Too emotional.

Words are amazing. They show us our belief systems.

Too emotional. What does that mean?

One of the roles of a behavioral analyst is to make us think about the words we use. Dr. Jesús Rosales-Ruiz would say we are always emotional. Emotions can be pleasant or unpleasant. You can feel excited and agitated, calm and serene, but they are all emotions.

The Emotions of Extinction
We can look at a particular process, such as extinction. Suppose you have been consistently reinforcing your horse for standing on a mat. He has your undivided attention, so the clicks have been very consistent. But now you’re interrupted. Your friend has asked you to watch how her horse is trotting. She’s not sure if he’s lame.

Your attention shifts away from your own horse. He’s still standing on the mat, but now he’s not being reinforced. You’ve just put him into the early stages of an extinction process.

While you’re focusing on your friend’s lame horse, your own horse is going through his most recent repertoire of behaviors. What is going to work to get you paying attention to him? He puts his ears forward, he poses, he drops his head, he paws, he nudges your arm.

While he’s presenting those obvious behaviors, he’s also experiencing emotions. He’s feeling confused, then frustrated, then possibly angry. If the extinction process continues on long enough, he may begin to feel helpless because nothing is working. Finally, he’ll become resigned as he gives up and settles into a more subdued state of acceptance.

Extinction’s Emotional Pattern
We see this extinction process as a negative thing because it “produces emotions.”

Jesús reminds us that ALL processes produce emotions. We tend to think about emotions when they are the size of a five alarm fire, but really we are always “being emotional”. There are emotions associated with ALL behaviors. Ideally in training we’d like to avoid the five-alarm-fire type. That’s why it is so important to understand these processes. The sooner you recognize that you are in an extinction process, the sooner you can do something to get out of it.

In extinction the individual (rat, human, horse, etc.) follows a predictable emotional pattern.

First, you see response bursting.

rat-pressing-lever Here’s what that means: You are observing a rat that has been reinforced consistently for pressing a lever. Abruptly the lever pressing no longer produces the expected result.

What does the rat do? It presses the lever with even more energy trying to get it to work. This has been equated with the classic hitting the button over and over again on the vending machine when your coke doesn’t fall out.

In the next stage you get angry. Now you’re kicking the coke machine.

Next you see regression. Behaviors which have been useful to you in the past reappear. What have you seen modeled? What is your past history when things like this fail?

Then there is a pause followed by another period of response bursting. Gradually the cycles become less pronounced. Each phase becomes smaller both in scale and duration until the individual settles into a calmer stage of acceptance.

Grief
Some psychologists have equated this pattern with the stages people go through when they are grieving. When you lose a loved one, a job, a home, you are thrown into an extinction process. Your loved one is gone. The reinforcers associated with that individual are gone, and your behavior is ineffective. Nothing you can do will change the reality of your loss.

The stages of grief begin with denial, followed by anger, then depression, bargaining, and finally acceptance and a return to a meaningful life.

It’s interesting to see the comparison people make between the process of grief and the process of extinction. Understanding does bring with it coping skills. If you understand the process you are in, you can keep things in perspective and find a faster way out of the worst of the emotional tangles. You can also be more understanding towards others (horse or human) if they are caught up in an extinction or grief process.

One of my Click That Teaches coaches, Cindy Martin wrote:

“Your description of the process people and horses go through, when things don’t work the way they expected, was so accurate and yet so full of empathy. The more I do clicker training, and teach and share clicker training, the more I realize there are some very profound lessons in the process; forgiveness, compassion, consideration. Those occupy a deeper layer, beneath the observation, handling skills, planning and preparation. Lately, I’ve been describing this type of clicker training, the kind that emphasizes details, and consideration for the learner, as ‘thoughtful clicker training.’”

When people ask Kay Laurence how she trained a particular behavior, the answer she often gives is: thoughtfully. As we gain more of an understanding of this work, we converge along similar paths. They all lead in the same direction – toward an ever deepening appreciation of others – whatever the species.

Coming Next: Understanding Extinction to Master Extinction

theclickercenter.com theclickercentercourse.com

November 18, 2016 by theclickercenter

JOY FULL Horses: Leaving History Behind

Emitted and Permitted Behaviors
It’s all too easy to find yourself in an extinction process. You thought you could hold out for just a few more steps of beautiful trot, but your horse got distracted and now he’s not giving you anything you like. In fact, the more you withhold your click, the worse it gets. Now he’s regressing back to behaviors that you thought were long gone. What are the keys to unraveling this regression mess?

The first is to tighten up your training and learn how to set up the environment so the behavior you want is the behavior that is most likely to occur. In his presentation on regression and resurgence, Dr. Rosales-Ruiz made the distinction between emitted and permitted behaviors.

When behavior is emitted, you are waiting to see what the learner offers. When behavior is permitted, you set up the environment so the behavior you want is the behavior that is most likely to occur.

If you’re waiting, waiting, waiting for the dog to sit or the horse to step on a mat, you may see lots of experimenting before you get something you want to click. All that experimenting can end up being part of your final behavior.

When your horse isn’t certain what to do next, waiting for something to click can lead to frustration. He’ll start trying previously learned, but unwanted behavior.

To avoid this kind of confusion and frustration we start horses off with very simple, easily isolated behaviors such as targeting and backing. We set up the environment so the behavior is likely to occur. You aren’t surfing an extinction wave of behaviors. Your horse doesn’t have to do a lot of guessing. The right answer is obvious and easy.

Data Collecting
When I’m working with novice horses and novice trainers, I have people put just a few treats into their pockets. Limiting the treats means you’re limiting the amount of training you can do all in one go. Before your horse can get too confused or frustrated, you’re stepping away to get another round of treats. You’re also assessing what just occurred in that session. That first targeting session is just data collecting.

You’re finding out if that’s a good starting point, or perhaps you need to find a different lesson. A horse that is very shut down, or becomes easily stressed when he’s not told exactly what to do, may need you to start with an even simpler step than targeting. This is a horse who may need his introduction into clicker training very carefully structured.

You may need to begin by feeding without making the food contingent on anything. You reach into your pocket and hand him a treat. Repeat this. When he’s at ease with the hand feeding, you can click and feed, click and feed. He won’t have time in between mouthfuls to do very much so you don’t have to worry about introducing unwanted behaviors.

When you click and you see him looking for the food, you can begin to make the click contingent on a specific behavior. In the early days of clicker training this was called charging the clicker. Normally, we can go straight to targeting or some other very simple lesson, but for some horses this is an important beginning step.

Tuning Up the Handler’s Skills
Designing an appropriate lesson plan is just part of the solution. You also need to have clean handling and good timing. Clicking late, clicking the wrong thing, clicking because you haven’t clicked for a while – all of these things will confuse your learner and lock in more unwanted behavior. So work on your handling skills. Use your video camera, practice in front of a mirror, borrow a friend to be your “horse”.

When your handling is quiet, clean, organized, and second nature, that’s what your training will become – quiet, clean, organized, and second nature.

Building Your Repertoire
Good handling is part of the solution. Another is developing a broad repertoire of behaviors. The more skills you teach your horse, the more options he’ll have besides the one you don’t want. Instead of falling back into old habits of biting or spooking, he’ll respond with the newer, more recently reinforced behaviors you’ve taught him.

This is where the phrase “trust the process” begins to make sense. We’ve all read the stories. Someone has been struggling with a horse, not seeing much progress, and then suddenly the pieces all fall into place. Instead of snapping at his handler, the horse is backing up and dropping his head. Instead of pulling away, he’s offering beautiful lateral flexions.

The older repertoire is still there. Given the right triggers, you might still see him regressing back into “childhood”, just as we sometimes find ourselves regressing back and behaving like our four year old selves in need of a nap. But you’ve given him more tools. That broader repertoire gives him more options. Now when he’s uncertain, he’ll go first into head lowering instead of snapping at you.

Coming Next: Animal Emotions

theclickercenter.com theclickercentercourse.com

November 17, 2016 by theclickercenter

JOY FULL Horses: Resurgence and Regression

Reverting to Past Behaviors
Imagine you have joined us for the Five Go To Sea conference cruise. You have just come from breakfast which you enjoyed on a terrace overlooking the open waters of the Carribbean. You have now settled yourself comfortably in the Reflection’s conference room to listen to Dr. Jesús Rosales-Ruiz’s lecture on resurgence and regression.

Jesús began by sharing the story of a professor who was attending a conference in Mexico. She got trapped in an elevator. At first she tried pushing all the buttons, calling out for help, things we would all think to do. Two hours later, when they finally got the elevator working again and the doors open, they found her huddled in the corner of the elevator calling for her mother – and her mother had been dead for years.

What does this story tell us? We regress in predictable patterns that reveals our history.

When a behavior that was being reinforced no longer works, you enter an extinction process in which you regress back to previously learned behavior. When the first behaviors you try don’t work, you go back another step and then another.

As Jesús said, very tongue in cheek, during the extinction process we see behavior that was modeled for us in our childhood. If you want to learn about someone’s early family dynamics, watch what happens to them when they are under stress. If one of his students is acting out, he tells them – “Don’t blame me. Blame your parents. You’re simply presenting behavior that was modeled for you in childhood.”

So extinction can reveal history. That’s definitely a gem to take away from our Caribbean treasure trove and carry back to our horses.

Extinction Reveals Your Horse’s Past
When a horse is first learning about clicker training, much of what he knows no longer applies. You’re holding a target up for him to touch. A lot of horses figure out quickly how the game is played, but some get confused. Suppose you’re working with a horse you recently adopted from a horse rescue. He isn’t sure what he’s supposed to do. Consider the dilemma he finds himself in. You only mean well, but he doesn’t know that. Past experience has told him wrong answers get punished, but the few things he knows how to do aren’t working. He is plunging head long into an extinction process.

The extinction process can reveal a horse’s training history. It helps us to understand the “childhood” our horses have had. Did your horse have a fair introduction to people, or are there issues you need to know about?

In most cases when you introduce a horse to the clicker, it’s smooth sailing. The horse quickly figures out the game. You may have to go through a little bit of explaining around the food, but for most horses this moves along without any major hiccups. You hold a target up, he investigates it, click, you give him a treat. Easy. Unless he’s one of those horses who has been punished for showing any self-expression.

If your horse has learned that being “well behaved” means he doesn’t offer any behavior you haven’t asked for, he’ll be good at following orders, but not taking the initiative. In fact your “well-behaved” horse may have learned that offering behavior is dangerous. The best way to avoid punishment is to wait to be told what to do.

This is why I put well-behaved in quotes. Is he well mannered in the way a clicker-trained horse can be? Or is he simply not offering much in the way of behavior? There’s a huge difference. In the first, the personality is expressed. In the later, it is suppressed.

When you hold out the target, a suppressed horse may be stuck for answers. He doesn’t know what you want. The “right answers” that normally work don’t seem to apply in this new situation. This horse finds himself in a difficult position. He knows he’s supposed to do something, but past experience tells him if he guesses wrong, he’ll be punished. He’s not sure what the answer is so he’s plunged into an extinction process.

Extinction follows a predictable pattern. At first he may try offering the one or two things that might possibly fit this situation. When those don’t work, he’ll shift rapidly from feeling frustrated and worried to being aggressive. That’s the next, predictable stage in the extinction process. Your “well behaved” horse is suddenly coming at you with teeth bared.

It’s easy to blame clicker training or the treats for this sudden turnaround in behavior, but I’ve always seen it very differently. I’ve always said that what is happening is the training history of the horse is being revealed. Jesús’ presentation on resurgence and regression confirmed this. It helped me understand even more clearly this dynamic. Sadly, there are all too many horses who have been at the receiving end of excessive punishment. Often you don’t know which is the horse who really is sweet and well behaved, and which is shut down through punishment. This is one of the reasons I put so much structure around the beginning steps of clicker training. The support of these lessons helps insulate the punished horses from their history.

Well Behaved or Shut Down?
Often what we refer to as “well behaved” horses (and people) are really individuals whose behavior and personality have been shut down through the use of corrections. They have learned to wait to be told what to do. Offering behavior, and expressing their personality has been punished. Give them a command, and they will respond promptly. They can seem like such perfect horses. Safe, easy to direct. But put them into a situation where they don’t know the answer – in fact they really don’t even understand the question – and you will begin to see things unravel. As the extinction process unfolds, they will take you back through the stair steps of how they have been treated, and often the story they tell is not a pretty one.

Clicker training did not cause these outbursts. When these horses are not sure of the “safe” answer, they’ll began to regress back through their training history. You are seeing the behavior that others “swept under the carpet” by suppressing it with punishment.

When you are brand new to clicker training, and, especially if you are also new to horses, this can be a hard dynamic to understand. What you hear about clicker training is how much fun it is, how much horses enjoy it. So you give it a try. But instead of smooth sailing, your horse falls apart. Instead of having a wonderful time, you’re dodging teeth.

You’ve been promised a dream horse and all you have is a nightmare. How could you not blame clicker training? But just as equally, how can you go back? How can you return to the use of punishment to suppress the behavior you’re now dealing with?

You keep hearing from others that you need to trust the process. That can seem like a hard choice, especially when you don’t really understand what the process is, but what other choice is there? You don’t want to go back to your correction-based training, so you plunge ahead, clicker in hand.

Coming Next: Leaving History Behind

theclickercenter.com theclickercentercourse.com

June 9, 2015 by theclickercenter

An Invitation

The series on Resurgence has generated a lot of interest, and I know from my in-box, also a lot of questions. So I’ve invited Dr. Jesús Rosales-Ruiz and Kay Laurence to join me for an on-line question and answer session. I hope you will join us, as well.

When: June 21, 2015

What Time? 1 pm Eastern Time Zone (Check on line for time zone converters if you aren’t sure of the time difference.)

Cost: $35.00

Space is limited. If you would like to attend, do sign up early. The meeting is limited to a maximum of 25 people.

How do you sign up? Email me at: kurlanda@crisny.org. I will send you a paypal invoice for the meeting.

What do you need to participate? For optimum audio quality, a headset is recommended.

What should I bring? Lots of questions!

Alexandra Kurland
theclickercenter.com

June 8, 2015 by theclickercenter

5GoToSea: Part 15: Micro Masters

Resurgence and Regression: Understanding Extinction So You Can Master It

From a presentation given by Dr. Jesús Rosales-Ruiz during the 2014 Five Go To Sea Conference cruise.

Part 1: The Elevator Question
Part 2: The Translation to Horses: Is Personality Expressed or Suppressed?
Part 3: Unraveling the Regression Mess
Part 4: Extinction and Shaping
Part 5: Extinction Reveals The Past
Part 6: Accidental Extinction
Part 7: Emotions
Part 8: Training With High Rates Of Reinforcement
Part 9: Cues and Extinction
Part 10: PORTL
Part 11: Mastering Extinction
Part 12: Creativity Explored
Part 13: Degrees of Freedom
Part 14: The Positive Side of Resurgence
Part 15: Micro Masters

If you have not read the previous installments of this series, I suggest you begin with Part 1. Part 1 was published on May 21, 2015.

Part 15: Micro Masters

The “Pose”
Jesús closed his presentation with two horse examples. The first was Robin’s “pose”. I’ve told the story of the “pose” many times. I’ll keep it brief here. Robin first learned a stationary “pose”. It originally was a by-product of cleaning up his treat taking manners when he was two years old. During the process he started “posing”, arching his neck and looking like a very pretty dressage horse. I liked the look so I continued to reinforce it. It became a default behavior. In the absence of any other active cue from me, if Robin posed, I would click and reinforce him. I became the cue for the behavior.

Offering “the pose” meant that if Robin wanted to interact with me and engage in the clicker game, he had a sure fire way of doing so. Even if I was busy doing barn chores, if I saw him posing, I would click and reinforce him. I never wanted him to feel like the proverbial toddler who is banging the kitchen pots and pans to get his mother’s attention. If Robin wanted attention from me, he had a behavior which he could use to satisfy his need for social interaction.

Because Robin wasn’t ignored, he didn’t go through an extinction process. I didn’t see a regression into the unwanted behaviors that macro extinctions can cause. Instead I was able to reinforce a behavior I liked, one that was a useful warm up for our formal training sessions. For his part Robin was confident that I would engage with him when he asked for attention.

Reinforcing him for the stationary pose went on through the winter. I didn’t have any plans for developing the behavior. It was simply something I liked. It was Robin who was the creative one!

It must have been late March. I was lunging him in the arena one evening. He was giving me a ho hum trot. There was nothing there I could reinforce. Robin went once around the circle, twice, three times without reinforcement. Normally I would have been clicking and reinforcing him at a much higher rate, but given the plow horse trot I was presented with, there was nothing there I wanted to say yes to.

At the time I would not have described it in these terms, but I was putting him into an extinction process. I could see him searching, trying to decide what to do. On the third time round he had the answer. He would try his pose. But in order to pose and still stay in the trot, he had to add energy. Within one stride he transformed into magazine-cover magnificence. I captured the moment with a click and the rest is history. The “pose” has evolved into a major component of my work. Robin showed us that we could indeed shape self carriage. What began as a happy accident for Robin has become a deliberate and very systematically trained behavior in other horses.

Our Creative Horses
When I first told this story to Jesús, he commented that the pose came out because of resurgence. At the time, I didn’t understand the significance of what he was saying, but I remembered what he said. And Jesús remembered the story. It got him thinking about the procedure and how we might use it to make deliberate use of resurgence. The result: we now have a systematic way of creating unlikely behaviors. The end result can look like magic, but there is good science behind it. Here are the steps:

First, you build a strong history of reinforcement for the component behaviors.

Next, you change the situation somewhat so extinction comes into play.

This generates a resurgence of previously reinforced behaviors. The result: new combinations emerge. That’s creativity. The most fun for me is seeing what the horses invent. They are often so much more creative than their human partners!

Seeing Familiar Landscapes with Fresh Eyes
Kay Laurence might say we are seeing familiar landscapes with fresh eyes.

Jesús would say you have to understand the process of extinction so you can master it. If you understand it, you won’t be frustrating your animals. Instead, you’ll know how to use extinction to generate complex behaviors.

I would say that monitoring the level of extinction your learner is experiencing is a keys-to-the-kingdom part of good training. I recently spent a couple of days working with a group of horses I have come to know well. One of them is a retired performance horse. Without going into a lot of details, I would describe him as an emotionally fragile horse. He’s easily worried. If he thinks he has the right answer, he’s a superstar, but I always have to be careful how far I stretch him into new behaviors. If he thinks he might get something wrong, he worries. He’s come out of a training environment in which he had to perform correctly or his rider could get seriously hurt. I suspect he was corrected for mistakes which accounts for his worry.

Mastering Micro
This past weekend I was working among other things on this horse’s pose. He’s very much got the idea that he gets reinforced for lifting up through his topline and releasing at the poll. I was holding out for slightly better versions. As I withheld my click, I saw him experimenting. Was it higher with his poll? Was it more lift of his back? What did I want?

The shifts he was giving me represented micro changes. The variations were all within a clickable range. Clicking him for any of these variations would not have been wrong, but I was waiting fractionally to see what else would pop out. I was using micro extinctions to create the next step. And because I was thinking about this in terms of extinction, I was monitoring closely how this related to his emotional level. I did not want him to become macro worried.

We were always just a second or two away from a click so I could let him experiment within a micro extinction without risking the emotional fallout of a larger extinction process.

Micro is so very much the key.

Macro extinctions are painful. Micro extinctions are part of good shaping.

Macro shaping can be frustrating. Micro shaping is elegant.

Macro negative reinforcement is literally painful. Micro is again good shaping.

When you go micro, your learner is always just a second or two away from a reinforceable moment. You can cue another behavior. You can click and treat. Either way, you are saying: “Yes! Great idea!” Micro mastery is what we should be striving for in our training. When you say someone is a great trainer, you are saying he is a Micro Master. In training that’s the “black belt” we should be aiming for.

(Note: this video was taken when Robin was three years old. He was not yet started under saddle. Also, he had never been in side reins or any of the other devices that are commonly used to lunge horses. This beautiful self-carriage was shaped entirely through clicker training. The dressage whips that I’m using serve as targets. They give Robin orientation points that help him maintain his balance relative to me.)

This concludes the report on Dr. Jesús Rosales’ Ruis’ 2014 presentation on Resurgence and Regression given at the Five Go To Sea conference cruise.

For information on the 2015 Five Go To Sea Alaska cruise visit fivegotosea.com

Alexandra Kurland
theclickercenter.com
theclickercentercourse.com

Please note: If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites:

theclickercenter.com theclickercentercourse.com