The Goat Diaries – Still Catching Up

The Goat Palace – Foot Targets and Puzzle Moments

This will teach me to take time off from these Goat Diaries for the Holidays.  It feels as though I have entered Alice in Wonderland territory – always running and never getting anywhere.  By the time I catch up, I will be behind again.  Oh well.  We’ll see how far I get today.

I think the easiest approach is to describe where we are and not worry too much about the details of how we got there.  In my previous post I wrote about stillness: in stillness comes understanding.  I needed stillness, especially with Thanzi.  She’s so used to getting what she wants by pushing her way in.  Touching a target was easy.  That part was great, but after the click she was pushing in trying to go directly to the treats.

Thanzi pushing for treat.png

Thanzi has just touched her target, but now she’s trying to go directly to the treats.

In loopy training both sides of the click have to be clean for a loop to be clean.  The food delivery was a long way from being what I wanted.  I could move on with the targeting, but I would be dragging along with it all the unwanted food manners.  So I changed tactics and introduced both ladies to platforms.

Platforms give the feet a place to be.  I could click them for being on the platform. That was the behavior side of the click.  On the food delivery side I positioned myself so they remained on the platform while they got their treat. That meant staying very close to them which made my treat bowl even more of a target, but at least I had their feet more or less planted.  It was funny watching how much they could stand elephant-on-a-drum at the edge of the platform.  They would look as though they couldn’t possibly keep from tumbling forward, but they spread their toes and somehow managed to stay on.

When I first set out the platforms,  Trixie predictably was suspicious of the platforms.  She able to be by herself in the hallway, but she was uneasy about moving too far away from Thanzi.  She certainly wasn’t ready yet to go as far as the second platform which was set further down the hallway.  Both platforms were initially something to avoid. She was the most horse-like of all the goats.  It wasn’t that she was afraid to step on raised surfaces, but she was definitely suspicious of anything new in her environment.

Thanzi was the complete opposite.  When I opened the gate, she would dash out.  She would then stick to me like glue.  If I went to the far end of the hallway, she dashed right along with me.  She certainly had no concerns about being away from Trixie.  Instead it was hurry, hurry, hurry to stay glued to me.

The platforms, however, were also something to be avoided.  At first, I just worked on targeting and ignored the platforms.  Thanzi was getting pretty solid with the targeting so it was time to make the platforms part of the lesson.  I had my green target stick with me. I had her orient to it a couple of times. Click.  I fed so she had to back up away from me.  Then I kept changing my position.  To get to the target, the most direct route was to step up onto the platform.  This is so very different from the way I would do it with horses!

Thanzi put her front feet up on the platform, and then it was easy.  She stepped up all four onto the platform.  Click then treat while her feet were on the platform.  Finally, with her feet still, I could work on grown-ups. I held the target stick straight down in a neutral position and waited for her to take her nose away from my pocket. Click – treat.  It helped so much having her feet anchored on the platform.

In stillness comes understanding.

The platform is a foot target.  Keeping her planted on one spot helped her to notice what the rest of her body was doing.  I was standing right beside her, close enough for her to easily reach my pocket.  If I had stepped away even a little bit, my movement would have drawn her off the platform.  When she was pushing at my pocket, no treats, but as soon as her nose moved away from my pocket, click, I fed her.

When she wasn’t on the platform, her feet moved along with her nose so it wasn’t clear what she was being clicked for.  Here it was so much clearer.  I clicked because you moved your nose.  That sound you just heard is a predictor that treats are coming.  Thanzi was putting two and two together fast.

With people we often find rules restricting.  The more rules we have, the more we feel caged in.  But here rules were liberating.  Stand on this platform, look straight ahead, and you are in control. You can make this very odd person reach into her pocket and hand you a much desired piece of squash.  For Thanzi understanding created even more of an eagerness to play the game. That eagerness was expressed not with anxiety, but with an ever-growing confidence.

Once Thanzi was on a platform, she was super at staying on. I would click and treat for a short round, then I walked to the other platform holding the target out for her to follow. Thanzi raced after me, but she didn’t automatically jump up onto the platform.  She’d by-pass it on her way to get to me.  I had to maneuver myself so the most direct route to me was via the platform.  Click as she stepped up, and then another round of clicking for all four feet on the platform and her head in good “grown-ups” position.

Trixie’s sessions were similar.  They were just done in slow motion compared to Thanzi.  I noticed with her that when I switched sides, she completely lost track of the target. She had not yet generalized so I could present the target from different orientations.  I had to be on her left side, and the target needed to appear as expected.

Certainly with horses if I don’t vary my presentation, I can end up with a very one-sided learner. Trixie’s reaction to the target was more than this.  It made me wonder if her nervous personality made it harder for her to notice patterns. And then does that contribute to her feeling even more nervous because it is harder for her to predict what is going to happen? She starts out nervous and becomes more nervous because all that worry gets in the way of making connections.

Or do nervous individuals start out having more trouble making connections, and that’s what gets the ball rolling.  They start to feel anxious because they don’t understand why things are happening the way they are.  Which comes first the chicken or the egg?  Either way, it’s a snowball scenario.

As very young horses, Robin and Panda were both superb at making connections and they are both supremely confident. Thanzi is so like them.  She’s very quick and very confident. If we collected data, would this correlation between pattern recognition and confidence hold?

Is this one of the reasons clicker training helps nervous learners? By slowing things down for Trixie I am helping her make connections.  She was understanding that A leads to B in a predictable way.  That was making her bolder and more confident in her responses.

Does predictability help reduce generalized anxiety by giving you more control over what is going to happen?  If I don’t understand how A and B are connected, it’s much harder to adjust my behavior to avoid the things I don’t like or to get to the things I want.  When you’re frazzled and worried about every little thing, all that noise makes it harder to understand what is really happening.  You see connections that don’t exist, and you miss the reliable predictors.  When the world seems to be built on shifting sands, of course you will be more nervous.  It remains to be seen if the stability that platforms offer will help Trixie to become an increasingly confident learner.

With both goats I had the beginning of platform training.  They would both step up onto a platform and stay there while I clicked and treated them.  But I wasn’t convinced that they were really aware that there was something special about being on a platform.  Once on, they tended to stay on, but when I took them off with a target, I had to direct them back to the platform.  Neither one of them went to a platform on her own.

So I switched tactics.  I set a food bowl down just far enough in front of the platform that they would have to take a step off to get to the bowl.  The question was what would they do after they got the treat.  Thanzi’s answer was she consistently backed herself onto the platform, looked regal, got clicked, dashed forward to get her treat and then backed herself onto the platform.

All the backing to deliver the treat had primed her well.  She easily solved this puzzle.  Trixie was similar, just slower.  So now I had two goats who could get themselves back onto a platform after getting their treats.  Progress!  For Thanzi in particular it was time to create a real puzzle moment for her.

Kay Laurence talks about puzzle moments.  This is when you set a test for your learner to see if what they have learned matches what you thought you were teaching.  When you play the table games, you often have times when the person gives you the correct answer.  You think they have the concept you were trying to teach.  Maybe you want them to touch only the yellow object.  You’re setting two objects out on the table, one yellow and one purple, and they are consistently touching the one you want.  Click and treat.  So now you add to the choices by setting out three objects, and they touch the purple one!  Surprise, surprise.  What is going on?

They’ve been operating under a different rule.  Maybe the yellow object was always the first one you put out, or the one you put closest, or the smallest, or the biggest.  There are lots of different variables to choose from in any system.  Your learner found a rule that worked to produce the correct answer – until it didn’t.  You can go on for quite a long time thinking your learner understands what you want, but unless you test it, you really don’t know for sure.

Maybe Thanzi had no idea that the platform was significant.  Maybe she was backing up after she got her treat because she’d discovered that backing up got me to click.  The platform just happened to be in the way.  That was certainly part of how I got her onto the platform in the first place.  The question now was had the platform itself become significant?  Did she understand that being on the platform mattered, not just backing up?

I had been gradually moving the food bowl further and further out from the platform.  That meant a couple of times after she got her treat, Thanzi was no longer lined up directly with the platform.  When she backed, she ended up broadside to it.   What would she do?

I love watching puzzle moments.  This is when you can really see your learner processing what is going on.  When you train in tight loops, a rhythm emerges.  It’s get the right answer, repeat.  Get the right answer, repeat.  You slide the criterion along so smoothly that your learner really doesn’t notice that you’ve been gradually making the lesson harder.  There’s no break in the rhythm until you stumble across a puzzle moment.  Now there’s a pause.  This is when you want to be very still.  No prompting.  No helping the learner.  No giving away the answer and depriving them of ownership of the solution.  You wait and watch.  If it’s clear your learner needs help, you offer it.  You give the clue that will make solving the puzzle possible, but first you wait to see if that extra clue is needed.

The reinforcer for waiting was seeing Thanzi step sideways up onto the platform.  Click and treat.  She was understanding.  Several more times in that session she got off line from the platform and each time she very deliberately deviated from where she was to step up onto it.  Her actions told me that she understood the platform was indeed significant.

So now she was ready for the next puzzle moment.  She was solidly on the platform.  I could click, and then walk to her to give her a treat.  She would wait for me to bring the treat to her.

And I could also drop the treat into the food bucket.  She would leave the platform, get the treat, and then back up onto the platform.  I was impressed by how well she understood the two forms of treat delivery.  There was no confusion between them.  She was reading my body language well, and she was waiting on the platform when that was indicated.

I was also impressed by how quickly I had been able to open up space between us and how solid she was about staying on the platform.  What a smart goat!

So now I presented her with a puzzle moment that made her head hurt.

I tossed the treat into her food bowl.  The platform was behind her.  As she lifted her head, I began to take a further step back in the opposite direction from the platform.  The food was going away!

In the world in which she had lived up until now you followed food, and you made sure you were the first with your head in the bucket.  In this alternate universe that she now found herself in, backing up away from the food bucket got you treats.  She had worked her mind around that concept.  Going back to the platform got me to reach into the metal bowl I was holding and hand her a treat.  But I had never moved the bowl away.  Now it was leaving!

The question was could she back up as I backed up?  Could she allow the gap between us to keep expanding.  Would she be able to cope with this truly inside out world?  I backed just a little and waited.  It was so clear her head was hurting with all the computing she was doing.  As smart as she is, she wasn’t used to having to think so hard.  No, she just couldn’t do it.  Not yet.  Not this time.  I helped by stepping slightly forward.  She went onto the platform.  Click and treat.

I did another couple of rounds where things were kept as she expected them to be. I clicked and reinforced her a couple of times while she stayed on the platform.  Then I clicked and dropped a treat in the bucket.  She came forward to get her treat.  As she lifted her head out of the bucket, I took a step back.

And this time she could find the answer.  She let me take the food bowl further away as she backed up to the platform.  What a truly fast learner she is!  What a smart goat.  No wonder she’s such a powerful leader.  When she stood up on her platform waiting for me to click, she looked so regal.  I hope she was as pleased as I was by how well she had solved that puzzle!

So that is pretty much where I am with the four I am working with.  Elyan and Pellias are learning about working together and sharing.  It sounds like kindergarten.  Always I am reminded of Robert Fulghum’s charming book, “Everything I Ever Needed to Know I Learned in Kindergarten”.  The ladies are learning about platforms.   Marla has continued to work with Galahad on targeting.  She was away over the Thanksgiving Holiday.  When she got back, she also introduced him to platforms.  So all the goats have taken a major step forward in their basic education.

Coming next we’ll see where all of these good puzzle pieces are taking us.





JOY FULL Horses: Understanding Extinction: Part 9

Eureka Moments: What is Insight?

Using resurgence – Insight
Yesterday I shared several PORTL games developed by Dr. Jesús Rosales-Ruiz.   The games deliberately used extinction.  What was observed was this: when you have been consistently reinforcing behaviors as you establish them in repertoire, and you then remove all reinforcement for them, you get a resurgence of these previously reinforced behaviors.  They reoccur in the order in which they were trained.  

When you instead extinguish the individual behaviors during the teaching phase, you get a different result.  The student will go back to the most recently learned behavior.  If that doesn’t work, he’ll go a little further back, and then a little further back.

In resurgence the behaviors occur in the order in which they were taught, so the oldest behavior in the cluster occurs first.

In regression the order reverses.  The most recently taught behavior reappears first.

So how does this help us?  How can we use this understanding to shape behavior?  To get the ideas rolling Jesús shared several video examples where resurgence was used to train complex, creative behaviors.

The first video came from Robert Epstein’s work. Epstein was B.F. Skinner’s last graduate student.  Together they were exploring the concept of “insight”.  How do we solve puzzles?  Are we truly creating something that has not existed before, or is creativity a product of combining known components to solve a novel puzzle?

Bird Brains
To explore this question Epstein taught a pigeon three component behaviors: pecking a banana, climbing on a box, and pushing the box towards a target.

The pigeon was then put into a chamber with the box and the banana.  The banana was hung up out of reach.  The pigeon couldn’t peck the banana, so an extinction process began. There was a resurgence of previously trained behaviors.  The pigeon was able to push the box under the banana, get up on the box, and peck the banana.

How did the pigeon solve this puzzle so quickly?  What is insight? What really is creativity?  Skinner and Epstein would say the pigeon could solve the problem because it had in its existing repertoire the necessary components.  Pigeons that had no experience pushing the box or jumping up on the box failed to solve the puzzle.

What is Creativity?
Jesús gives us a very process-oriented way thinking about this experiment.  This kind of complex puzzle solving was achieved through resurgence.  Set up the underlying components well, add in a bit of extinction, and “creativity” pops out.

If you leave out one of the components, the individual will struggle to solve the puzzle.  He will experience a much longer extinction process.  Macro extinction emotions will begin to surface, and you have to hope the subject has the persistence to become truly creative.

This is the kind of creativity that is truly stressful.  It’s much better to analyze the end goal – the complex behavior you want to train – break it down into all of it’s component tasks, and then train each of the components separately.  The result will be brilliant looking pigeons that solve in minutes what we might otherwise think would be an impossible puzzle for them.

Jesús’ comment was there is “nothing new under the sun”. The behaviors you try are all built out of things you’ve done before.  All the components of what appears to be a novel behavior have been trained in the past. So let’s consider what happens when a group of people are presented with a challenging puzzle.  When they begin experimenting and find that the usual, familiar things aren’t working, some will give up quickly.

Others will persist.  They will experiment with novel combinations of what they already know, but again most will quit if they don’t come up with a solution fairly quickly .

A few will keep trying until they stumble across a novel combination that works.  We call these people inventors and creators because they are persistent enough to find these novel combinations.  The discovery process can be a painful one, but once the new combination has been found, it’s easy for everyone else to copy the results.

I can absolutely relate to this.  Give me a horse puzzle to solve, and I can be very persistent. My life experience has taught me that persistence pays off.  But put me in front of a computer that isn’t cooperating, and I shut down fast. There my experience has produced a different set of expectations. I’ve been in enough situations where errors in a software program have made a problem unsolvable, at least for my level of computer skills.  I don’t have the programing background that makes wrestling with a software issue fun.  Extinction has gone too far and been too uncomfortable.  So in one situation I can be very persistent and creative.  In another I’m the one going through the classic cycle of emotions that macro extinction produces.

I know first hand both how much fun the creative process can be when the expectation of success is there.  And I also know how painful and unpleasant the extinction process is when that expectation is missing.

What I want to create for my learners is a feeling of confidence.  Whether horse or human, I want them to KNOW they can solve whatever training puzzle I throw at them. Build this expectation in early before others have taught them hard lessons about failure, and you get brilliant, enthusiastic, joyful individuals.  They are the optimists of this world.  Whether horse or human, they are fun to be around.  That’s what an understanding of these concepts helps us to create.

Coming Next: Degrees of Freedom

Remember, if you are new to the JOY Full Horse blog, click on the JOY Full Horses tab at the top of this page to find the full table of contents and links to each of the articles I have published so far.

I hope you will want to share these articles by sending links to this blog to your friends.  But please remember this is copyrighted material.  All rights are reserved. Please do not copy any of the “JOY Full Horses” articles without first getting written permission from Alexandra  Kurland, via

Also note: these articles are not intended as an instruction guide for introducing your horse to clicker training.  If you are new to clicker training and you are looking for how-to instructions, you will find what you need at my web sites: