Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Support retention and transfer

by Leave a Comment

In a discussion we were having with David Ganulin on marketing, my colleague Matt Richter ended up talking about how many ‘team building’ activities don’t work. The typical model is an event where folks get together off-campus and face challenges together. They have to work together to overcome the challenges. Yet, Matt’s claim was that the empirical evidence was that the results didn’t transfer back to the workplace. What does it take? How can we support transfer to achieve persistent results?

The classic model is the ‘ropes course’. Folks have to work together to get everyone safely across some challenge. By working together to achieve success, you should build team cohesion and respect the different capabilities of your colleagues. Yet, investigations suggest that what’s learned doesn’t carry back to the workplace. People who got along, when they get back to the workplace, can be surprised and disappointed that the same conflicts exist.

What’s happening, of course, is context-specificity. The resulting benefits worked in the context of the team-building, but it’s not the same context as work. Just like the ‘brain training’ exercises didn’t transfer to other tasks, so to any learning is likely to dissipate quickly and still not transfer to another context. What do we need to do, then, to generate retention over time and support transfer to the workplace as well?

For one, we need more than one practice. I just read the results of interesting research suggesting two stages of memory. The first stage says initial memories can last briefly, but for sustained retention, you need a second stage of retrieval practice. Yes, we should know that, but too often we don’t practice it! (Which also suggests that a test at the end of a learning event may not be a good indicator!) Also, I’ll suggest, if we want appropriate transfer, we have to engineer it.

How do you engineer transfer? I’ll posit two steps. For one, you need experience across several different contexts. So, do task A together, then B which is widely different, then C, which is different again. You could do a task that requires different physical attributes (tall, small, strong, heavy), and then one that requires different creative approaches (art, music, prose). Along the way, you reinforce a particular team approach that works across contexts. You facilitate reflection, as well, on what’s common.

Matt went further, suggesting that then you need to take that facilitation back to the workplace, and I’ll agree that it’d be ideal. If you then brought the models back to the workplace and facilitated their application to situations at work, you could extend the internalization and appropriate re-contextualization of the learning.

One-shot events are unlikely to generate the sustained transfer you need, at least not without specific design and support. If you’re not trying to achieve retention (over time after the event until needed) and transfer (to all appropriate and no inappropriate) situations, why bother? If you do want retention and transfer (and you should), design for it. Specifically engineer to support retention and transfer. Use spaced repetition with increased challenge to achieve the former. Use contextual variance and reflection facilitation to support the latter. When you do, you’ll have outcomes worth the investment.

Misplaced organizational focus?

by 3 Comments

Conjunctions are interesting learning opportunities. When two things provide different facets, particularly on something you’ve been thinking about, it’s serendipitous. In this case, two widely different readings triggered some reflections asking whether perhaps we’ve a misplaced organizational focus.

So, I’ve been a bit concerned about the rabid interest in generative AI. Not that I think it’s inherently bad, despite its flaws. Instead, my concern is the uses it’s put to. If you think about the classic engineering proposition – cheap, fast, or good; pick 2 – you know you can apply AI to any of the areas. Always, however, it seems that the focus is on cheap and fast. Which concerns me. There’s substantial evidence that our L&D efforts aren’t having an impact. Thus, doing bad faster and cheaper is still bad!

Part of this, it seemed to me, to stem from a rabid focus on short-term returns. I read The Japan That Can Say No many moons ago, and became convinced that a purely financial focus isn’t in the long-term interests of organizations. Now, there’re reinforcement!

First, in Australian news was a report about how a famous economist was rethinking the role of economics. While I didn’t agree with all of it, one aspect that resonated was captured in these bits:

“…we have largely stopped thinking about ethics and about what constitutes human well-being. We are technocrats who focus on efficiency…We often equate well-being with money or consumption, missing much of what matters to people.”

The juxtaposition happened with this quote aggregated by Learnnovators and posted to LinkedIn:

” The early signals of what A.I. can do should compel us to think differently about ourselves as a species. …Those skills are ones we all possess and can improve, yet they have never been properly valued in our economy or prioritized in our education and training…”
– Aneesh Raman, VP, Workforce Expert at LinkedIn & Maria Flynn, President & CEO of Jobs for the Future (JFF)

The overlap, to me, has to do with the undervaluing of what humans bring to the economic table. Efficiency isn’t the only good. Pushing L&D to do ‘box ticking’ learning design faster and cheaper isn’t consonant with recognizing what gives our work meaning. Besides undervaluing what learning design could and should be, it’s disrespectful to the learners and the organization.

I think that what’s driving organizations should be how they contribute to society as a whole. The means to that end is creating an internal environment conducive to supporting people, individually and collectively, to contribute their best in ways that respect what we offer. There are things technology can do that, frankly, we as people shouldn’t. Similarly, there are things we can do that we shouldn’t abrogate. To paraphrase the meme, I don’t want people doing menial tasks leaving the creativity to machines.

A holistic synergy, each doing what they do best to augment the other, alone and together, is optimal. Our economics should support that as well, and to the extent our structures don’t, it may be time to rethink them. Otherwise, it’s a misplaced organizational focus. Thoughts?

Engineering solutions

by 1 Comment

Every once in a while, I wonder what I’m doing (ok, not so infrequently ;). And it’s easy to think it’s about applying what’s known about learning to the design of solutions. However, it’s more. It is about applying science results to designing improvements, but, it’s broader than learning, and not just individual. Here are some reflections on engineering solutions.

As I’ve probably regaled you with before, I was designing and programming educational computer games, and asking questions like “should we use spacebar and return, or number keys to navigate through menus?” (This was a long time ago.) I came across an article that argued for ‘cognitive engineering’, applying what we knew about how we think to the design of systems. Innately I understood that this also applied to the design of learning. I ended up studying with the author of the article, getting a grounding in what was, effectively, ‘applied cognitive science’.

Now, my focus on games has been on them as learning solutions, and that includes scenarios and simulation-driven experiences. But, when looking for solutions, I realize that learning isn’t always the answer. Many times, for instance, we are better off with ‘distributed‘ cognition. That is, putting the answer in the world instead of in our heads. This is broader than learning, and invokes cognitive science. Also, quite frankly, many problems are just based in bad interface designs!  Thus, we can’t stop at learning. We truly are more about performance than learning.

In a sense, we’re engineers; applying learning and cognitive science to the design of solutions, (just as chemical engineering is about applying chemistry). Interestingly, the term learning engineering has another definition. This one talks about using the benefits of engineering approaches, such as data, and technology-at-scale, to design solutions. For instance, making adaptive systems requires integrating content management, artificial intelligence, learning design, and more.

Historically, our initial efforts in technology-facilitated learning did take teams. The technology wasn’t advanced enough, and it took learning designers, software engineers, interface designers and more to generate solutions like Plato, intelligent tutoring systems, and the like.  I’ve argued that Web 1.0 took the integration of the tech, content design, and more, which usually was more than one person could handle. Now, we’ve created powerful tools that allow anyone to create content. Which may be a problem! The teams used to ensure quality. Hopefully, the shift back comes with a focus on process.

We can apply cognitive science to our own design processes. We’ve evolved many tools to support not making reliable mistakes: design processes, tools like checklists, etc. I’ll suggest that moving to tools that make it easy to produce content haven’t been scaffolded with support to do the right thing. (In fact, good design makes it hard to do bad things, but our authoring tools have been almost the opposite!)  There’s some hope that the additional complexity will focus us back on quality instead of being a tool for quantity. I’m not completely optimistic in the short term, but eventually we may find that tools that let us focus on knowledge aren’t the answer.

I’m thinking we will start looking at how we can use tools to help us do good design. You know the old engineering mantra: good, fast, and cheap, pick 2. Well, I am always on about ‘good’. How do we make that an ongoing factor? Can we put in constraints so it’s hard to do bad design? Hmm… An interesting premise that I’ve just now resurrected for myself. (One more reason to blog!) What’re your thoughts?

 

Why DEI?

by 1 Comment

At the event I attended a bit ago, one of the discussions was on Diversity, Equity, and Inclusion (DEI). I attended, to hear what was up. There were discussions of how to instigate DEI, but one thing I felt was missing, so of course I chimed in at the end. Actually, I learned something else as well, so that’s worth reciting to. So, why DEI?

There are, of course, lots of good reasons. For one, the privileges I’ve had haven’t been shared. Folks often come from less opportune backgrounds than others have had the advantage of. Moreover, such advantage hasn’t been accounted for before they get to work. Unfortunately, schools and social welfare haven’t adequately addressed this We have racism, and misogyny, and other forms of discrimination to deal with. ‘Us against them’ isn’t a healthy perspective. However, perhaps you wonder, why should organizations be a source of remedy?

My argument it pretty simple, really. Research says that we get better results when we have diversity in looking for solutions. There’s a pretty simple explanation why, too. What we’re doing, when looking for answers (research, design, trouble-shooting) is searching a potential solution space. It’s easy to not explore thoroughly. I’ve talked about brainstorming, for instance, as something we can do badly or well. That’s about process. But there’s more.

Garvin, Edmondson, & Gino wrote about learning organization dimensions, and one of the four aspects of a supportive learning environment was “appreciation of differences”. I want to emphasize it’s not ‘tolerating diversity’, it’s valuing it! In exploring that space of solutions, the more diversity in the group, the more likely we are to cover a big range. (There’re caveats, of course, particularly that all have to share a commitment to finding an answer.) Homogeneity is the enemy here!

Of course, this means equity in treatment, and inclusion. If you’re excluding people, you’re not taking advantage of diversity. If you’re not promoting equity, the injustices perpetuate. The only good way to get people to feel good about diversity if it is equitable and inclusive.

Interestingly, one of the hosts mentioned that there’s separate evidence of value. This was something I hadn’t heard. Apparently, having more diversity in the room makes people more diverse in their thinking. That is, even before getting people to generate ideas, people’s attitudes are more diverse because of the observed variety. I haven’t been able to confirm this, but I have no reason not to believe it, and it’s an interesting (and valuable) result.

Now, as said, there are lots of good reasons. But one that is very pragmatic is that you get better solutions when different viewpoints are incorporated. We should be looking at complementary and varied viewpoints. That involves bringing different people together that have something to offer, and just being different is one! Celebrate that!

So, that’s why DEI in my mind; done right, the outcome is better!  Overall, we fare better when we work in the ways that align with how our brains operate. That’s alone and together. Let’s do the best for us and our organizations.

Domain-independent coaching?

by Leave a Comment

At an event this past week, I sat in on a discussion of coaching. Asking folks what coaching was, there were lots of responses about ‘establishing rapport’, ‘asking questions’, etc. I admit I was a wee bit curious amongst all this, thinking about specifics. Which prompted some reflections. My question is about whether there can be domain-independent coaching.

To start, I was thinking about how to develop people just after a learning ‘event’ or experience. They’ve been developed to a certain level, and then we’d like to continue their development. To do so, I thought feedback would be useful, and specifically tying the learning to any relevant task, and providing feedback to fine-tune their performance. Specifically, this requires knowing the domain they’re learning about, observing their performance (in some way), and identifying ways in which they went right, or wrong. That, in my mind, requires specific knowledge about how the mental models play out in context. This, for example, is what we see in sports coaching.

As context, I remember talking to a very smart individual who runs a business that does coaching as a service, at scale. To do this, they have to have folks who know coaching, but pragmatically can’t necessarily know the domain. I was curious how this could work, but empirically it does. Coupled with the responses of folks around the table, I had to reconcile my specifics with a more general approach.  How can this work?

Of course, I started thinking about the trajectory of learners. They start as novices in any particular domain, then proceed to become practitioners, and can become experts. As they progress, they need less specifics. If you look at situated leadership as a model, you go from providing direction and support, to eventually removing the (domain-specific) direction, then the support, as they become capable. Thus, coaching can move to asking about how they’re feeling about it, and to apply their own knowledge to the situation. That is, you can start asking about the process and their thoughts rather than focusing on specifics.

Of course, to me, if you apply the domain-independent coaching at the wrong time, you can delay (or extinguish) their development. On the other hand, continuing with micromanaging performance can be similarly restricting. So, I reckon you can shift to domain-independent coaching, after you have developed a minimum viable level of capability.  That’s my reconciliation; what’re your thoughts?

We can be logical

by Leave a Comment

So, I’ve been on a bit of a crusade saying we’re not formal logical reasoning beings. And, I do think it’s important to emphasize this in the face of some legacy beliefs. On the other hand, I think there’s evidence that we can be logical. So, how do we reconcile this?

The reason I push against a belief that we’re logical is that too often we are designing as if that’s the case. We see it in way too many policies, practices, and the like. Yet, as has been documented, that’s not our default.

On the other hand, we can be effective reasoners. We have created complex mathematics, advanced science, and generally improved our situation. Something is going on. But what?

Well, Kahneman talks about how we, effectively, have two systems, fast and slow. The slow one takes cognitive effort, so we tend to avoid it. The fast one, then, is default. It’s based upon instinct. Which can be good in two situations: one, where our instincts are likely to be right (e.g. dealing with biologically primary information) or where we have expertise. It can also be bad, where we use it inappropriately.

On the other hand, we can use the slow route. It’s hard, but it works.  This is where we reason things out. (We have to be careful, because being hard, we can depend on it inappropriately.) We can use cognitive support, and complementary skills, but we can document the situation, explore alternatives, trial solutions, and reason our way to good decisions.

And we should! Frankly, I’d rather have in office a policy wonk building coalitions of expertise than a solitary ‘profile’ claiming solutions across the board. I want evidence-based approaches, not simplistic and wrong answers to complex problems!

So, we can be formal logical reasoning beings. Under the right circumstances, with the right support. We should automate what we can so we build the necessary expertise, and provide the conditions for good decisions. That can sometimes be fast, and sometimes be slow, but better to be right than to be expedient. Not perfect, of course, but I’m suggesting we err on the side of likelihood.

That’s my view, at any rate. We can be logical, and that’s a matter of design. We should evaluate and optimize situations so we get the best decisions. That recognizes when training is helpful, when performance support can be used, and when we should support good innovation (problem-solving, research, design, etc). So let’s take a healthy informed look at how we make decisions, and increase the likelihood of good ones. That’s my decision, at any rate. What’s yours?

Do you feel lucky?

by Leave a Comment

roulette wheelOne of the things that I feel is undervalued is the role of luck. We hear about how the successful – the winners in business – get that way by virtue of their intelligence and diligence. Yet, if you think about it, lots of folks are smart and work hard. Yet not all succeed. Which made me wonder just how much of success is luck. I asked Siri (I was on a walk) and got the link to an article where they actually researched this. As to the answer, do you feel lucky?

The article starts with a suite of evidence. I know I’m mighty lucky to have been born as a white male in California, had both parents, was able to secure a really good education, and more. The data says that all these things are boons to the likelihood of success. There were also all sorts of weird variations (including middle initials contributes to success?).

Further, the article reports on how two researchers ran some simulations. They had characters with varying degrees of ‘talent’, and then also some good and bad luck. What happened, of course, is that the folks with a combination of luck and ‘talent’, did best. Talent alone didn’t do it, nor luck. In fact, the most talented didn’t succeed the most. “The most successful agents tended to be those who were only slightly above average in talent but with a lot of luck in their lives.”

The research goes further. It’s typical, in academia, that folks who get grants then are more likely to get subsequent grants. Which, it turns out, isn’t the best option. A different simulation by other researchers suggested random was better!  And, arguably, the best policy was giving everyone the same amount!

When we take this back to the real world, what seems to be important is that luck plays a big role in success. Those folks at the very top appear to have been very lucky. Further, their future success isn’t guaranteed (note that currently there’s a prime example of over-valuing previous success). If you’re smart, and dedicated, you’re more likely to do well, but you can also be subject to the slings and arrows of fortune which can similarly contribute.

I think we should be wary of rewarding past success with greater opportunity. We should also be wary of any assessment of how smart someone must be, just because they are successful. There are a lot of factors that contribute to success (for instance, research suggests, that being taller and having a deeper voice, increases the likelihood of doing well in business). They do say luck favors the prepared mind, so do work hard. But you’re also dependent on the vicissitudes of fate. Do you feel lucky?

For ‘normals’

by 5 Comments

So, I generally advocate for evidence-based practices. And, I realized, I do this with some prejudice. Which isn’t my intent! So, I was reflecting on what affects such decisions, and I realized that perhaps I need a qualification. When I state my prescriptions then, I might have to add “for ‘normals'”.

First, I have to be careful. What do I mean by ‘normal’? I personally believe we’re all on continua on many factors. We may not cross the line to actively qualify as obsessive-compulsive, or attention-deficit, or sensorily-limited. Yet we’re all somewhere on these dimensions. Some of us cross some or more of those lines (if we’re ever even measured; they didn’t have some of these tests when I was growing up). So, for me, ‘normal’ are folks who don’t cross those lines, or cope well enough. Another way to say it is ‘neurotypical’ (thanks, Declan).

What prompted this, amongst other things, is a colleague who insisted that learning styles did matter. In her case, she couldn’t learn unless it was audio, at least at first. Now, the science doesn’t support learning styles. However, if you’re visually-challenged (e.g. legally blind), you really can’t be a visual learner. I had another colleague who insisted she didn’t dream in images, but instead in audio. I do think there are biases to particular media that can be less or more extreme. Of course, I do think you probably can’t learn to ride a bicycle without some kinesthetic elements, just as learning music pretty much requires audio.

Now, Todd Rose, in his book The End of Average, makes the case that no one is average. That is, we all vary. He tells a lovely story about how an airplane cockpit carefully designed to be the exact average actually fit no one! So, making statements about the average may be problematic. While we’ve had it in classrooms, now we also have the ability to work beyond a ‘one-size fits all’ response online. We can adapt based upon the learner.

Still, we need to have a baseline. The more we know about the audience, the better a job we can do. (What they did with cockpits is make them adjustable. Then, some people still won’t fit, at least not without extra accommodation)  That said, we will need to design for the ‘normal’ audience. We should, of course, also do what we can to make the content accessible to all (that covers a wide swath by the way). And, while I assume it’s understood, let me be explicit here that I am talking “for ‘normals'”. We should ensure, however, that we’re accommodating everyone possible.

Facilitating in the dark

by 1 Comment

I recently spoke to the International Association of Facilitators – India, having chosen to focus on transfer. My intent was for them to be thinking about ensuring that the skills they facilitate get applied when useful. My preparation was, apparently, insufficient, leaving me to discover something mid-talk. Which leads me to reflecting on facilitating in the dark.

So, I’m not a trained facilitator (nor designer, nor trainer, nor coach). While I’ve done most of this (with generally good results),I’m guided by the learning science behind whatever. So, in this case, I thought they were facilitating learning by either serving as trainers or coaches. Imagine my surprise when I found out that they largely facilitate without knowing the topic!

In general, to create learning experiences, we need good performance objectives. From there, we design the practice, and then align everything else to succeed on the final practice. We also (should) design the extension of the learning to coaching past any formal instruction, and generally ensuring that the impact isn’t undermined.

How, then, do you get models, examples, and provide feedback on practice if you don’t know the domain? What they said was that they were taking it from the learners themselves. They would get the learners together and facilitate them into helping each other, largely. This included creating an appropriate space.

To me, then, there are some additional things that need to be done. (And I’m not arguing they don’t do this.) You need to get the learners to:

  • articulate the models
  • provide examples
  • ensure that they articulate the underlying thinking
  • think about how to unpack the nuances
  • ensure sufficient coverage of contexts
  • provide feedback on others’ experiences

This is in addition to creating a safe space, opening and closing the experience, etc.

So it caused me to think about when this can happen. I really can’t see this happening for novices. They don’t know the frameworks and don’t have the experience. They need formal instruction. Once learners have had some introduction and practice, however, this sort of facilitation could work. It may be a substitute for a community of practice that might naturally provide this context. You’d just be creating the safe space in the facilitation instead of the community.

The necessary skills to do this well, to be agile enough mentally to balance all these tasks, even with a process, is impressive.  I did ask whether they ended up working in particular verticals, because it does seem like even if you came in facilitating in the dark, you couldn’t help learn while doing the facilitation. There did seem to be some agreement.

Overall, while I prefer people with domain knowledge doing facilitation, I can see this. At least, if the community can’t do it itself. We don’t share enough about learning to learn, and we could. I do think a role for L&D is to spread the abilities to learn, so that more folks can do it more effectively. The late Jay Cross believed this might be the best investment a company could make!

Nonetheless, while facilitating in the dark may not be optimal, it may be useful. And that, of course, is really the litmus test. So it was another learning opportunity for me, and hopefully for them too!

 

Myths are models

by 4 Comments

A recent LinkedIn post talked about how models are good, but myths are bad. Which was a realization for me. I’ve kept myths and models largely separate in my mind, but I realize that’s not the case. Myths are models, just wrong ones. And, I suppose, we need to deal with them as such. (Also, folks hang on to myths and models if they’re tied up with identity, but we should still be able to deal with the logical rationale.)

So, I’m an advocate for mental models. There are a variety of reasons, personal, pragmatic, and principled. Personally, I was gifted a book on mental models by my workmates as I left for graduate school. Pragmatically, they’re useful. On principle, they’re how we reason about the world. Heck, our brains are constantly building them!

The important aspects of models are that they’re predictive (and explanatory). That is, they tell us the outcomes of actions in particular situations. They are models of a small bit of the world, and are used to understand a perturbation of the model. They’re causal, in that they talk about how the world works, and conceptual in that they talk about the elements of the world. They’re incomplete, in that they only need to account for the parts of the world relevant to the particular situation.

Examples include using an analogy of water flowing in pipes for thinking about electric circuits. Or how advertisements use association with valued things or people to induce a positive affect. You can use them to explain what happened, or what will happen. It’s the latter that’s important for the purposes of providing a basis for guiding decisions, and thus their role in learning.  They guide us in deciding how to take actions under different circumstances.

Models can be good or bad. The old ‘planets circling a sun’ model of electrons in orbit around a nucleus of protons and neutrons turned out to be inaccurate as our understanding increased. We then moved to probability clouds as a better model. Many of our mistakes come from using the wrong model, for a variety of reasons. We can mistake the situation, or think a model is accurate and useful.

We should avoid models that aren’t appropriate for the situation. Myths are models that aren’t appropriate for any situation. So, for instance, learning styles, generations, ‘attention span of a goldfish’, and ‘images are processed 60K faster than prose’ are examples of myths. They lead people to make decisions that are erroneous, such as providing different learning prescriptions. They are models, because they do categorize the world and lead to prescriptions about what to do. They’re myths, because their implications will lead to decisions that waste time and money.

As the saying goes, “all models are wrong, but some are useful”. They’re wrong because they’re only part of the world. The good ones give us useful predictions, The bad ones lead us to make bad decisions. The useful ones are to be lauded, shared, and used. Myths, however, should be debunked and avoided. Myths are models, but not all models are good. It’s important that I remember that!