Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Experts and Explanations

by 1 Comment

blueprint pencil rulerI’ve been going through several different forms of expert documentation. As a consequence, I’ve been experiencing a lot of the problems with that! Experts have trouble articulating their thinking. This requires some extra work on the part of those who work with them, whether instructional designers, technical writers, editors, whoever. There are some reliable problems with experts and explanations that are worth reviewing.

The start of the problem is that the way we acquire expertise is to take our conscious thinking and automatize it, basically. We want to make our thinking so automatic that we no longer have to consciously think about it. So, we basically compile it away. Which creates a problem. For one, what we process into memory may not bear a close resemblance to what we have heard and applied. That is, the semantic language we use to guide our practice and internalize may not be what we store as we automate it.

It’s also the case that we lose access to that compiled away expertise. There’s evidence of this, for one from the results of research by the Cognitive Technology group at the University of Southern California showing experts can’t access about 70% of what they do! Another piece of evidence is the widespread failure of so-called ‘expert systems’ in the 80s, resulting in the AI winter. Whether the locus of the problem is in what actually gets stored, or access to it, the result is that  what we were told to do, and say we do, may not actually be close to what we actually do.

Another problem is that experts also lose touch with what they grappled with as novices. What they take for granted isn’t even on the radar of novices. So it’s difficult to get them to provide good support for acquiring skills or understanding. Their attempts at explanations for reference of instruction fail.

All told, this leads to systematic gaps in content. I’ve been seeing this manifest in explanations that may say what to do, but not why or how. There may be a lack of examples, and the thinking behind the examples I  do see isn’t there.  There’s also a lack of visual support. They’re not including diagrams when it’s conceptual relationships that need understanding. They’re also not including images when context is needed. They shouldn’t necessarily be blamed, because they don’t need the support and can’t even imagine that others do!

It’s clear that experts should not be the ones doing the explanations. They’re experts, and they have valuable input, but there needs to be a process to avoid these problems. We need tech writers, IDs, and others to work with experts to get this right. Too often we see experts being tasked with doing the explanations, and we live with the consequences.

What to do? One step is to let experts know that their expertise is in their domain, but the expertise in extracting that expertise and presenting it lies in others. To do so convincingly, you’ll need the science about why. For another, know techniques to unearth that underlying thinking. Also allow time in your schedule for this to happen. Don’t think the SME can just give you information; you’ll have to process what you get to rearrange it into something useful. You may also need some sticks and carrots.

As I wrestle with the outputs of experts, here’s my plea. There are wonderful ways experts and explanations can work out, but don’t take it for granted. Don’t give experts the job of communicating to anyone but other experts, or to experts on working with experts to get explanations. Fair enough?

Examples before practice

by 4 Comments

I’ve been wrong before, and I’ll be wrong again, and that’s ok <warning: link is NSFW>. It’s like with science: if you change your mind, you weren’t lying before, you’ve learned more now. So I’ve been wrong about the emphasis between practice and examples. What I’ve learned is that, in general, practice isn’t the only area of importance, and the benefits of examples before practice.

So, as part of the Learning Development Accelerator‘s YOK (You Oughta Know) series, I got the chance to interview John Sweller. I’ve known John, I’m very honored to say, from my days at UNSW. I was aware of his reputation as a cog sci luminary, but he also turned out to be a really  nice person. He’s the originator of Cognitive Load Theory (CLT), and he was kind enough to agree to talk about it.

As background, he’s tapped into David Geary’s biologically primary and biologically secondary learning. The core idea is that some things we’ve evolved to learn, like speaking. Then there are things we’ve developed intellectually, like reading and writing, that aren’t natural. Instruction is to assist us to acquire the latter.  The latter typically has high ‘element interactivity’, whereby there are complex interrelationships to master. That is, it’s complex.

CLT posits that we have limited cognitive capacity, and overwhelming that capacity interferes with learning. The model talks about two types of load. The first is intrinsic load, that implied by the learning task. The second is extrinsic load, coming from additional factors in the particular situation. The premise is that learning complex things (biologically secondary) has such a high intrinsic load that we really need to focus on managing load so we can gradually acquire the entailed relationships.

There are a number of implications of CLT, but one is about the value of worked examples. An important element is showing the thinking  behind the steps. A second empirical result is that worked examples  are better than practice! At least, initially, for novices. Yet this upends one of my recommendations, which is generally that the most important thing we can do to improve our learning is focus on better practice. I still believe that, but now with the caveat after worked examples.  

Now, he didn’t tell us when that happens, e.g. when you switch from worked examples to practice. However, like the answer to how much spacing needed for the spaced practice effect, I suspect the answer is ‘it depends’. There’s the ‘expertise reversal’ effect that says as you gain experience, the value of worked examples falls and the value of practice raises. That point, I’d suggest, is dependent on the prior knowledge of the learners, the complexity of the material, the scope, and more.

I’m now recommending, particularly for new material, that improving the learning outcomes includes meaningful practice  after  quality worked examples. That’s my new, better, understanding. Make sense?

As an aside, I talked about CLT in my most recent book, on learning science, with a diagram. In it, I only included intrinsic and extrinsic, as those two seemed critical, yet the classic theory also includes  germane intrinsic load. One of the audience members asked him about that, and John opined that he probably needn’t have included germane. Vindication!

Good and bad advice all in one!

by 2 Comments

I was asked to go to read an article and weigh in. First, please don’t do this if you don’t know me. However, that’s not the topic here, instead, I want to comment on the article. Realize that if you ask me to read an article, you’re opening yourself up to my opinion, good  or bad. This one’s interesting, because it’s both. Then the question is how do you deal with good and bad advice all in one.

This article is about microlearning. If you’ve been paying attention (and there’s no reason you should be), I’ve gone off on the term before. I think it’s used loosely, and that’s a problem because there are separate meanings, which require separate designs, and not distinguishing them means it’s not clear you know what you’re talking about. (If someone uses the term, I’m liable to ask which they mean! You might do the same.).

This article starts out saying that 3-5 minute videos are  not  microlearning. I have to agree with that. However, the author then goes on to document 15 points that are important about microlearning. I’ll give credit for the admission that there’s no claim that this a necessary and complete set. Then, unfortunately, I also have to remove credit for providing no data to  support the claims!  Thus, we have to evaluate each on it’s own merits.  Sorry, but I kinda prefer some sort of evidence, rather than a ‘self-evident’ fallback.

For instance, there’s a claim for brevity. I’ve liked the admonition (e.g. by JD Dillon) that microlearning should be no longer, and no shorter, than necessary. However, there’s also a claim here that it should be “3 – 10 minutes of attention span”. Why? What determines this? Human attention is complex, and we can disappear into novels, or films, or games, for hours. Yes, “Time for learning is a critical derailer”, but…it’s a factor of how important, complex, and costly if wrong the topic is. There’s no one magic guideline.

The advice continues in this frame: there’re calls for simplicity, minimalism, etc. Most of these are good principles,  when appropriately constrained. However, arbitrary calls for “one concept at a time is the golden rule”  isn’t necessarily right, and isn’t based on anything other than “our brains need time for processing”. Yes, that’s what automation is about, but to build chunks for short term memory, we have to activate things in juxtaposition. Is that one concept? It’s too vague.

However, it could be tolerated if some of the advice didn’t fall prey to fallacious reasoning. So, for instance, the call for gamification leans into “Millennials and Gen Z workforce” claims. This is a myth. Gamification itself is already dubious, and using a bad basis as an assumed foundation exacerbates the problem.  There are other problems as well. For one, automatically assuming social is useful is a mistake. Tying competition into the need to compete is a facile suggestion. Using terms like ‘horde’ and ‘herd’ actually feels demeaning to the value of community. A bald statement like “Numbers speak louder than words!” similarly seems to suggest that marketing trumps matter. I don’t agree.

Overall, this article is a mixed bag. So then the question arises, how do you rate it? What do you do? Obviously, I had to take it apart. The desire for a comment isn’t sufficient to address a complex suite of decent principles mixed up with bad advice and justified (if at all) on false premises. I have to say that this isn’t worth your time. There’s better advice to be had, including on microlearning. In general, I’ll suggest that if there’s good and bad advice all in one, it’s overall bad. Caveat emptor!

Generic Thinking Skills?

by 3 Comments

Recently, a colleague asked a few of us about our views on critical thinking skills. This is actually a contentious topic. There are broad claims of the need for them, increasingly, even showing up in job advertisements. On the other hand, researchers and others have weighed in against them, saying that expertise is the only lever. I tend to lump critical thinking skills in with the broader issue of generic thinking skills, so what are the issues?

Upfront, I’ll admit that I like the concept of generic thinking skills. Say, for instance,  learning-to-learn skills. That is, domain independent skills that lead to better approaches. It seems to make sense that, in the absence of specific knowledge, some general approaches are more useful than others. For instance, faced with a new domain, I’d be inclined to expect that systematic experimentation and observation would be better than random trial and error.

On the other hand, prominent psychologists like John Sweller and Paul Kirschner have said that domain-specific skills are the only way to bet. There is significant evidence that expertise matters in successful approaches to problem-solving, and others. While we have some innate skills for domains that are biologically primary, learning in other domains requires expertise.

Is there, then, any evidence for generic skills? Based on Micki Chi’s work on the value of self-explanation, Kate Bielaczyc and others have found that instruction on systematically explaining steps in examples help, across domains. In my own Ph.D. thesis, I trained folks on analogical reasoning skills, and found improvement (for component skills that weren’t a) already ceilinged or b) were perceived to be immutable, across different problem types.

How, then, do we reconcile these conflicting viewpoints? My (self- :) explanation is that it’s a matter of degree, a continuum rather than a dichotomy. The more domain knowledge you possess, the more likely you are to find a good answer. However, what if you’re in a new domain where you don’t have relevant expertise to hand? In that case, I’ll suggest that there are benefits to some approaches over others, and training those general skills is justifiable. That is, general skills are weaker than domain specific skills, but general skills are better than nothing!

We know that there are practices that improve outcomes. For instance, I’ve written about how to, and not to, do brainstorming. Similarly, I believe Harold Jarche’s Seek-Sense-Share model works across domains. Systematic creativity is  not an oxymoron!  That’s the story I’m holding on to about generic thinking skills. What are your thoughts on the topic?

Accreditation?

by 1 Comment

As occasionally happens, I was asked a question on LinkedIn. In this case, it was about my thoughts on accreditation. Also, as occasionally happens, I thought that I’d share my thoughts in this forum, and look for feedback to improve my thinking. So here’re some thoughts on accreditation. I welcome yours!

First, let’s be clear, I am not an expert on accreditation. I haven’t accredited anything, for one ;). I  did look into it, at one point many years ago. I’ve also served on independent board of directors or advisory boards for several entities. In the former case, we have a legal responsibility to provide guidance. In the latter case, we provide the best guidance, but of course the organization isn’t obliged to comply. The former, in particular, serves as a quality check, or a form of accreditation.

When I looked into accreditation for educational institutions, the requirement isn’t about the actual curriculum, but instead that there is a library and that there are processes for review and revision of course offerings. That is, it was about the support for learning and quality processes, not the actual offering. This creates a process support that should ensure quality, yet also the ability to apply this to institutions with a wide variety of offerings.

Institutions can also seek accreditation by organizations in particular areas of curriculum. Offerings in computer science, business, and others for instance, receive review and then can receive approval by bodies that represent the particular field. This depends on the quality of the organization doing the accreditation and their processes, of course.

There’s also accreditation on the quality of the educational process. You can also be reviewed and accredited on the basis of your pedagogy, for instance your online teaching approach. It depends, of course, on what they stipulate as quality, but that’s always going to be the case.

There are, of course, dubious accreditations. It’s not unknown for an organization or collection thereof to establish their own accrediting body that basically rubber stamps the organization(s). Caveat emptor.

In general, I think that having a scrutable external validation is a good check on quality. Whether that makes sense is probably an issue of scope. A small offering of a particular course might benefit from an independent advisory board, which provides some oversight. The larger the organization and the scope of activity, the greater the need for some external validity check.

From the other side, I think a certificate or credentials help the learner signify what they’ve accomplished.  However, without accreditation or at least a scrutable process, how do you know the skill/knowledge is appropriate and accurate? I think accreditation has the potential to be a ‘reality check’ on any offering.

Learning or Performance Strategy

by 1 Comment

Of late, I’m working in a couple of engagements where the issue of learning and performance strategy have come up. It has prompted some thoughts both on my part and the part of my clients. I think it’s worth laying out some of the issues and thinking, and of course I welcome your thoughts. So here are some reflections on whether to use learning or performance strategy as an organizing concept.

In one case, an organization decreed that they needed a learning strategy. Taken with my backwards design diagram  from the learning science book, I was tasked with determining what that means. In this case, the audience can’t be mandated with classes or tutorials. So really, the only options are to support performance in the moment and develop them over time. Thus we focus on job aids and examples. I think of it as a ‘performance strategy’, not a learning one.

In the other case, an organization is executing on a shift from a training philosophy to a performance focus. Which of course I laud, but the powers-that-be expect it to yield less training without much other change. Here I’m pushing for performance support, and the thinking is largely welcome. However, it’s a mindset shift for a group that previous was developing training.

I general, I support thinking that goes beyond the course, and for the optimal execution side of a full ecosystem, you want to look at outcomes and let that drive you. It includes performance consulting, so you’re applying the  right solution to performance gaps, not the convenient one (read: ‘courses’ ;). Thus, I think it makes more sense to talk performance strategy than learning one.

Even then, the question becomes what does such a strategy really entail, whether learning  or performance. Really, it’s about having a plan in place to systematically prioritize needs and address them in effective ways. It’s not  just design processes that reflect evidence-informed principles, though it includes that. It’s also, however, ways to identify and track problems, attach organizational costs and solution costs, and choose where to invest resources. It includes front-end analysis, but also ongoing-monitoring.

It also involves other elements. For one, the technology to hand; what solutions are in use and ensuring a process of ongoing reviews. This includes both formal learning tools including the LMS and LXP, but also informal learning tools such as social media platforms and collaborative documents. Another issue is management: lifecycle monitoring, ownership, and costs.

There’s a lot that goes into it, but being strategic about your approach keeps you from just being tactical and missing the forest for the trees. A lot of L&D is reactive, and I am suggesting that L&D needs to be come proactive. This includes going from courses to performance, as a first step. The next step is to facilitating informal learning and driving innovation in the organization. Associated elements include meaningful measurement  and truly understanding how we learn for a firm basis upon which to ground both formal  and informal learning. Those are my thoughts a learning or performance strategy, what am I missing?

What makes a good book?

by 1 Comment

I was in contact with a person about a potential book, and she followed up with an interesting question: what’s the vision I have for publishing? She was looking for what I thought was a good book. Of course, I hadn’t really articulated it! I responded, but thought I should share my thinking with you as well. In particular, to get your thoughts!  So, what makes a good book? (I’m talking non-fiction here, of course.)

My first response was that I like books that take a sensible approach to a subject. That is, they start where the learner is and get them realizing this is an important topic. Then the book walks them through the thinking with models and examples. Ultimately, a book should leave them equipped to do new things. In a sense, it’s the author leading the reader through a narrative that leaves them with a different and valuable view of the world.

I think these books can take different forms. Some shake up your world view with new perspectives, so for example Don Norman‘s Design of Everyday Things or Todd  Rose‘s The End of Average. Another types are  ones that provide deep coverage of an important topic, such as Patti Shank‘s  Write Better Multiple-Choice Questions.  A third type are ones that lead you through a process, such as Cathy Moore‘s Map It. These are rough characterizations, that may not be mutually exclusive, but each can be done to fit the description above.

To me the necessary elements are that it’s readable, authoritative, and worthwhile. That is, first there’s a narrative flow that makes it easy to process. For instance, Annie Murphy Paul’s The Extended Mind takes a journalistic approach to important phenomena.  Also, a book needs an evidence-base, grounding in documented experience and/or science. It can re-spin topics (I’m thinking here about Lisa Feldman Barrett’s  How Emotions Are Made), but must have a viable reinterpretation. Finally, it has to be something that’s worth covering. That may differ by reader, but it has to be applicable to  a field. You should leave with a new perspective and potentially new capabilities.

That’s what came off the top of my head. What am I missing in what makes a good book?

Courtesy, shopping carts, and the organization

by 1 Comment

There’s a popular meme on the internet that I think is kind of apt. It asks whether you’re the type of person who returns your shopping cart. I think this is an important concept, because it illustrates to me one of the facets that make societies, and organizations, work. So let me talk about courtesy, shopping carts, and the organization.

The meme is basically saying do you return the cart, or leave it near your car. You’ve seen the results: parking spaces blocked by an abandoned cart, carts pulled up on curbs into planters, etc. This, to me, is like whether you bring a bag to clean up after your dog (and place it an appropriate receptacle). Or dump your ashtray and car trash on the side of the street. It’s about recognizing a) that there aren’t necessarily folks who have this as a job, and b) it interferes with innocent others, and c) therefore it undermines a pleasant environment. It may have to do with what your cultural expectations are, but I’ll suggest it’s worth the small effort.

Why does this matter? Because it seems to me that societies work better when folks are courteous. When folks respect one another, they find ways to make things work. When they don’t have that respect, they find or stumble into ways to aggravate situations. Now, I get that sometimes being discourteous is a way to get revenge against a real or perceived injustice. Yet, I suggest there are better ways to register your discontent that more accurately target the perpetrators of the injustice. Random acts of discourtesy can lead to perceptions that you’re just a jerk. Because some of it is people thinking that they don’t have to care about other people. I fear it’s getting worse.

The issue I want to address here, rather than a general societal rant, is about what this means in organizations. What does courtesy have to do with working life? I suggest it has to do with creating an environment in which people can work together for organizational success in an optimal way. That is, if we’re helping make the workplace pleasant, we’re making it effective. This means things like offering to help when you’ve useful information to provide. It means paying attention to the organizational norms. I suggest it also means pointing out when those norms aren’t best for positive interaction. Others: Refilling the printer paper tray when you’ve used the last. Taking notes for someone who  has to miss the meeting. Mot missing a meeting if you’ve no real excuse. Keeping meetings on time and on point. Not holding a meeting when there’s a better way.

There are a lot of little courtesies in everyday life. You may have a grudge or grievance, but deal with it appropriately. A lack of courtesy because you’re upset about something else isn’t appropriate or helpful. I reckon it just makes you look like a jerk. That’s my take, what are your thoughts on courtesy, shopping carts, and the organization?

The Performance Ecosystem and L&D

by 2 Comments

On LinkedIn recently, a survey in a post asked whether L&D should simply become performance consulting (Y/N). In the ensuing discussion, a comment was made that the binary discussion was flawed, and that a richer picture was possible. I was extremely pleased when she referred to my  Revolutionize Learning & Development book, and posted a diagram from it. I backed her comment, but it occurs to me that there’s more here, and of course I have a connection. So here’re some thoughts on the Performance Ecosystem and L&D.

To start, she cited how I wanted to move to Performance and Development. Indeed, I’ve posted about it, and included a diagram. In it, performance consulting  is represented, but as she noticed, there’s more. I think performance consulting is great, but…it’s not everything. To me, it only addresses the ‘optimal execution’ side of the picture, and ignores the ‘continual innovation’ opportunity.

To be fair, suggesting that L&D take responsibility for informal learning could be considered a stretch. My argument is simply that informal learning has practices and policies that can optimize outcomes,  and that it’s a necessary component of success going forward. (I note that problem-solving, design, research, and innovation all start without a known answer, so they’re learning too!) It’s not necessarily L&D’s role,  but who else (should) know more about learning?

So, innovation is an opportunity. A big one, I suggest. It’s a chance to move to the most valuable role in the organization, going forward. Orgs  need to innovate, and facilitating the best innovation is going to be a critical role. Why  not L&D? Yes, we have to get out of our comfort zone, start working with other business units, and most importantly know learning. So? We should anyway!

The infrastructure necessary is what I call the performance ecosystem. It’s about formal learning, but also more. That includes social, and information and learning resources. It includes facilitation as well as performance interventions. It’s about technology, but how to use it in ways that align with our brains.

The interesting issue for me is how to awaken this awareness. I  suggest  mobile is a gateway to the appropriate thinking. I wrote about mobile before writing the Revolution book (as my then-publisher required), but even there I laid out the case how mobile was not (just) about formal learning. Indeed, when you look at the way people use mobile, it’s very different. It’s also a digital platform, which means that it supports multiple outcomes.

Thus, mobile thinking is a way to break through the mindset of courses, and start looking at the bigger picture of technology supporting how we think, work, and learn to the success of our organizations. Which is why I’m happy to say that I’ll again be running the mobile course with Allen Academy, starting next week. Through 18 Jan, they’re offering this as a two-fer, so you get both the mobile and the learning science course for one low price! Together, you’re addressing my silly clip about L&D, both doing courses well  and going beyond them.

If you want to get your mind around the performance ecosystem and L&D, I suggest that mobile learning is a effective vehicle. You get both some deep advice about mobile, but it also generalizes to digital technology overall. The course itself looks at formal learning, performance support, informal learning, and more, as well as strategic issues. Coupled with learning science, this is a real grounding in the most important opportunities and necessities facing L&D today. Whether you call it P&D or L&D, these are core concepts. Hope to see you there!

 

Happy New(s) Year!

by 1 Comment

It’s the new year, and I’ve been hinting for a while that something’s up. Well, now I can announce it. The new year seems like the right time for news, so Happy New(s) Year!

While Quinnovation will continue, I have joined another initiative as well. If you’ve been paying attention, and I hope you have, two summers ago Will Thalheimer & Matt Richter ran a new and different event, the L&D Conference. It had two sessions of things, so it could reach most of the globe, and they were recorded so you could watch after the fact. There were also some asynch workshops. There was an emphasis on evidence-informed speakers and content.

It was successful enough that it convinced Matt and Will that there was the opportunity for a society promoting the same values. At the beginning of 2021, they began this initiative, the Learning Development Accelerator  (LDA). The society has had regular events, articles, also workshops. They also ran the conference again.

However, Will got a full-time job with Tier 1. Thus, trying to keep up with demand was, well, trying. After one unsuccessful attempt, they settled on a second choice to replace Will. I’ve agreed. Thus, I will now be working with Matt (who’s heroically shouldered most of the work) to keep the society going. It’s started, and continued, as all virtual. Which helps support the goal of being quite global. The other emphasis continues, to be evidence-based. I can align with that. I think it’s our obligation as professionals. We need to promote those who are translating research into evidence-informed practices.

There’s a second initiative of LDA as well. Matt thinks LDA should publish, and had offered me the chance to serve as initial publisher (more like Editor-In-Chief). So LDA Press is kicking off. We’re going to have to start slow to grow, but stay tuned for initial publications. The goal is to fill in the gaps of the books we should have, and provide a better relationship with authors. We’re already talking to some folks.

That’s pretty much it. I’ll continue to speak, write, run workshops, and assist clients as Quinnovation as well, but I’ll also be putting energy into what I think is an important contribution. Of course, it’s also about learning, stepping out of my comfort zone. My focus will be on trying to help introduce members to people and ideas they should know. The more we know, the better we can do our job! So that’s my Happy New(s) Year, and wishing you and yours all the best for the coming year.