Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Working with SMEs

by 1 Comment

In a recent post, I talked about how expertise is compiled away, and the impact on designing learning and documentation. Someone, of course, asked  how do you then work with SMEs to get the necessary information. Connie Malamed, one of our recognized research translators, has recently written about getting tacit knowledge, but I also want to address the more usual process.  I thought I’d written about it somewhere, but I can’t find it. So here are some thoughts on working with SMEs.

First, I’ve heard from several folks experienced in this that any one SME may not have both necessary elements. One element is to have a good model to guide the performance. The second elements is the ability to articulate that model. Their solution is to work with SMEs in groups. Guy Wallace (Eppic), Roger Schank (Socratic Arts), and Norman Bier (CMU) have all mentioned to me that they’ve found utility in getting SMEs together as a group and having that knowledge negotiation unpack the necessary learnings. They’re all folks worth listening to. You have to manage the process right, of course, but if you can do it, it’s useful.

I suggest that you also want several  different types of SMEs. You want not only the top performers, and theoretical experts, you also want just-past novices (also attributable to Guy) and supervisors. Theorists can give you models, while top performers can talk about the practical implications. Novices can let you know what they found hard to understand, and supervisors provide insight into what performers typically do wrong. All are helpful information for different parts of the learning.

Another trick I use is to focus on decisions. I argue that making better decisions will be more important to organizations than the ability to recite knowledge. SMEs  do have access to all the knowledge they learned, and it’s easy for them to focus on that. That’s where you get ‘info dump’ courses and bullet point-laden slides. By using decisions as a focus, you cut through the knowledge. “What decisions will they make differently/better as a result of this knowledge?” is a helpful question.

You can use questionnaires as well. Asking specifically about the elements: models, misconceptions, consequences, can be a good preliminary step before you actually talk to them. Or have a template for content for them to fill out. Any guidance and structure helps keep them focused.

Another preparatory step is to create a draft proposal of the information. You’ll likely be getting a dump of PDFs and PPTs). Process that material, and make your first, best, guess. It’s easier to critique than generate, so if you’re willing to be wrong (and why not), you can have them shoot holes in what you did. You’ll have focused on decisions (right?), and they’ll fix it, but you’ll have biased them for action.

Of course, you want to ensure you test for confirmation. You should circulate what you have learned, and get validation. You’ll need to have clear objectives that operationalize your learnings. You then should prototype and test what you’ve developed and see if it actually changes the behavior in useful ways. Ensure that your focus actually leads to the necessary change.

There are other elements you want from SMEs, such as their personal interest. However, it’s critical that you get them to focus on behavior change. It’s not easy, but it’s part of the job. Working with SMEs, correctly, is key to designing learning experiences that address real needs. These are my thoughts, what are yours?

Experts and Explanations

by 1 Comment

blueprint pencil rulerI’ve been going through several different forms of expert documentation. As a consequence, I’ve been experiencing a lot of the problems with that! Experts have trouble articulating their thinking. This requires some extra work on the part of those who work with them, whether instructional designers, technical writers, editors, whoever. There are some reliable problems with experts and explanations that are worth reviewing.

The start of the problem is that the way we acquire expertise is to take our conscious thinking and automatize it, basically. We want to make our thinking so automatic that we no longer have to consciously think about it. So, we basically compile it away. Which creates a problem. For one, what we process into memory may not bear a close resemblance to what we have heard and applied. That is, the semantic language we use to guide our practice and internalize may not be what we store as we automate it.

It’s also the case that we lose access to that compiled away expertise. There’s evidence of this, for one from the results of research by the Cognitive Technology group at the University of Southern California showing experts can’t access about 70% of what they do! Another piece of evidence is the widespread failure of so-called ‘expert systems’ in the 80s, resulting in the AI winter. Whether the locus of the problem is in what actually gets stored, or access to it, the result is that  what we were told to do, and say we do, may not actually be close to what we actually do.

Another problem is that experts also lose touch with what they grappled with as novices. What they take for granted isn’t even on the radar of novices. So it’s difficult to get them to provide good support for acquiring skills or understanding. Their attempts at explanations for reference of instruction fail.

All told, this leads to systematic gaps in content. I’ve been seeing this manifest in explanations that may say what to do, but not why or how. There may be a lack of examples, and the thinking behind the examples I  do see isn’t there.  There’s also a lack of visual support. They’re not including diagrams when it’s conceptual relationships that need understanding. They’re also not including images when context is needed. They shouldn’t necessarily be blamed, because they don’t need the support and can’t even imagine that others do!

It’s clear that experts should not be the ones doing the explanations. They’re experts, and they have valuable input, but there needs to be a process to avoid these problems. We need tech writers, IDs, and others to work with experts to get this right. Too often we see experts being tasked with doing the explanations, and we live with the consequences.

What to do? One step is to let experts know that their expertise is in their domain, but the expertise in extracting that expertise and presenting it lies in others. To do so convincingly, you’ll need the science about why. For another, know techniques to unearth that underlying thinking. Also allow time in your schedule for this to happen. Don’t think the SME can just give you information; you’ll have to process what you get to rearrange it into something useful. You may also need some sticks and carrots.

As I wrestle with the outputs of experts, here’s my plea. There are wonderful ways experts and explanations can work out, but don’t take it for granted. Don’t give experts the job of communicating to anyone but other experts, or to experts on working with experts to get explanations. Fair enough?

Examples before practice

by 4 Comments

I’ve been wrong before, and I’ll be wrong again, and that’s ok <warning: link is NSFW>. It’s like with science: if you change your mind, you weren’t lying before, you’ve learned more now. So I’ve been wrong about the emphasis between practice and examples. What I’ve learned is that, in general, practice isn’t the only area of importance, and the benefits of examples before practice.

So, as part of the Learning Development Accelerator‘s YOK (You Oughta Know) series, I got the chance to interview John Sweller. I’ve known John, I’m very honored to say, from my days at UNSW. I was aware of his reputation as a cog sci luminary, but he also turned out to be a really  nice person. He’s the originator of Cognitive Load Theory (CLT), and he was kind enough to agree to talk about it.

As background, he’s tapped into David Geary’s biologically primary and biologically secondary learning. The core idea is that some things we’ve evolved to learn, like speaking. Then there are things we’ve developed intellectually, like reading and writing, that aren’t natural. Instruction is to assist us to acquire the latter.  The latter typically has high ‘element interactivity’, whereby there are complex interrelationships to master. That is, it’s complex.

CLT posits that we have limited cognitive capacity, and overwhelming that capacity interferes with learning. The model talks about two types of load. The first is intrinsic load, that implied by the learning task. The second is extrinsic load, coming from additional factors in the particular situation. The premise is that learning complex things (biologically secondary) has such a high intrinsic load that we really need to focus on managing load so we can gradually acquire the entailed relationships.

There are a number of implications of CLT, but one is about the value of worked examples. An important element is showing the thinking  behind the steps. A second empirical result is that worked examples  are better than practice! At least, initially, for novices. Yet this upends one of my recommendations, which is generally that the most important thing we can do to improve our learning is focus on better practice. I still believe that, but now with the caveat after worked examples.  

Now, he didn’t tell us when that happens, e.g. when you switch from worked examples to practice. However, like the answer to how much spacing needed for the spaced practice effect, I suspect the answer is ‘it depends’. There’s the ‘expertise reversal’ effect that says as you gain experience, the value of worked examples falls and the value of practice raises. That point, I’d suggest, is dependent on the prior knowledge of the learners, the complexity of the material, the scope, and more.

I’m now recommending, particularly for new material, that improving the learning outcomes includes meaningful practice  after  quality worked examples. That’s my new, better, understanding. Make sense?

As an aside, I talked about CLT in my most recent book, on learning science, with a diagram. In it, I only included intrinsic and extrinsic, as those two seemed critical, yet the classic theory also includes  germane intrinsic load. One of the audience members asked him about that, and John opined that he probably needn’t have included germane. Vindication!

Good and bad advice all in one!

by 2 Comments

I was asked to go to read an article and weigh in. First, please don’t do this if you don’t know me. However, that’s not the topic here, instead, I want to comment on the article. Realize that if you ask me to read an article, you’re opening yourself up to my opinion, good  or bad. This one’s interesting, because it’s both. Then the question is how do you deal with good and bad advice all in one.

This article is about microlearning. If you’ve been paying attention (and there’s no reason you should be), I’ve gone off on the term before. I think it’s used loosely, and that’s a problem because there are separate meanings, which require separate designs, and not distinguishing them means it’s not clear you know what you’re talking about. (If someone uses the term, I’m liable to ask which they mean! You might do the same.).

This article starts out saying that 3-5 minute videos are  not  microlearning. I have to agree with that. However, the author then goes on to document 15 points that are important about microlearning. I’ll give credit for the admission that there’s no claim that this a necessary and complete set. Then, unfortunately, I also have to remove credit for providing no data to  support the claims!  Thus, we have to evaluate each on it’s own merits.  Sorry, but I kinda prefer some sort of evidence, rather than a ‘self-evident’ fallback.

For instance, there’s a claim for brevity. I’ve liked the admonition (e.g. by JD Dillon) that microlearning should be no longer, and no shorter, than necessary. However, there’s also a claim here that it should be “3 – 10 minutes of attention span”. Why? What determines this? Human attention is complex, and we can disappear into novels, or films, or games, for hours. Yes, “Time for learning is a critical derailer”, but…it’s a factor of how important, complex, and costly if wrong the topic is. There’s no one magic guideline.

The advice continues in this frame: there’re calls for simplicity, minimalism, etc. Most of these are good principles,  when appropriately constrained. However, arbitrary calls for “one concept at a time is the golden rule”  isn’t necessarily right, and isn’t based on anything other than “our brains need time for processing”. Yes, that’s what automation is about, but to build chunks for short term memory, we have to activate things in juxtaposition. Is that one concept? It’s too vague.

However, it could be tolerated if some of the advice didn’t fall prey to fallacious reasoning. So, for instance, the call for gamification leans into “Millennials and Gen Z workforce” claims. This is a myth. Gamification itself is already dubious, and using a bad basis as an assumed foundation exacerbates the problem.  There are other problems as well. For one, automatically assuming social is useful is a mistake. Tying competition into the need to compete is a facile suggestion. Using terms like ‘horde’ and ‘herd’ actually feels demeaning to the value of community. A bald statement like “Numbers speak louder than words!” similarly seems to suggest that marketing trumps matter. I don’t agree.

Overall, this article is a mixed bag. So then the question arises, how do you rate it? What do you do? Obviously, I had to take it apart. The desire for a comment isn’t sufficient to address a complex suite of decent principles mixed up with bad advice and justified (if at all) on false premises. I have to say that this isn’t worth your time. There’s better advice to be had, including on microlearning. In general, I’ll suggest that if there’s good and bad advice all in one, it’s overall bad. Caveat emptor!

Learning or Performance Strategy

by 1 Comment

Of late, I’m working in a couple of engagements where the issue of learning and performance strategy have come up. It has prompted some thoughts both on my part and the part of my clients. I think it’s worth laying out some of the issues and thinking, and of course I welcome your thoughts. So here are some reflections on whether to use learning or performance strategy as an organizing concept.

In one case, an organization decreed that they needed a learning strategy. Taken with my backwards design diagram  from the learning science book, I was tasked with determining what that means. In this case, the audience can’t be mandated with classes or tutorials. So really, the only options are to support performance in the moment and develop them over time. Thus we focus on job aids and examples. I think of it as a ‘performance strategy’, not a learning one.

In the other case, an organization is executing on a shift from a training philosophy to a performance focus. Which of course I laud, but the powers-that-be expect it to yield less training without much other change. Here I’m pushing for performance support, and the thinking is largely welcome. However, it’s a mindset shift for a group that previous was developing training.

I general, I support thinking that goes beyond the course, and for the optimal execution side of a full ecosystem, you want to look at outcomes and let that drive you. It includes performance consulting, so you’re applying the  right solution to performance gaps, not the convenient one (read: ‘courses’ ;). Thus, I think it makes more sense to talk performance strategy than learning one.

Even then, the question becomes what does such a strategy really entail, whether learning  or performance. Really, it’s about having a plan in place to systematically prioritize needs and address them in effective ways. It’s not  just design processes that reflect evidence-informed principles, though it includes that. It’s also, however, ways to identify and track problems, attach organizational costs and solution costs, and choose where to invest resources. It includes front-end analysis, but also ongoing-monitoring.

It also involves other elements. For one, the technology to hand; what solutions are in use and ensuring a process of ongoing reviews. This includes both formal learning tools including the LMS and LXP, but also informal learning tools such as social media platforms and collaborative documents. Another issue is management: lifecycle monitoring, ownership, and costs.

There’s a lot that goes into it, but being strategic about your approach keeps you from just being tactical and missing the forest for the trees. A lot of L&D is reactive, and I am suggesting that L&D needs to be come proactive. This includes going from courses to performance, as a first step. The next step is to facilitating informal learning and driving innovation in the organization. Associated elements include meaningful measurement  and truly understanding how we learn for a firm basis upon which to ground both formal  and informal learning. Those are my thoughts a learning or performance strategy, what am I missing?

What makes a good book?

by 1 Comment

I was in contact with a person about a potential book, and she followed up with an interesting question: what’s the vision I have for publishing? She was looking for what I thought was a good book. Of course, I hadn’t really articulated it! I responded, but thought I should share my thinking with you as well. In particular, to get your thoughts!  So, what makes a good book? (I’m talking non-fiction here, of course.)

My first response was that I like books that take a sensible approach to a subject. That is, they start where the learner is and get them realizing this is an important topic. Then the book walks them through the thinking with models and examples. Ultimately, a book should leave them equipped to do new things. In a sense, it’s the author leading the reader through a narrative that leaves them with a different and valuable view of the world.

I think these books can take different forms. Some shake up your world view with new perspectives, so for example Don Norman‘s Design of Everyday Things or Todd  Rose‘s The End of Average. Another types are  ones that provide deep coverage of an important topic, such as Patti Shank‘s  Write Better Multiple-Choice Questions.  A third type are ones that lead you through a process, such as Cathy Moore‘s Map It. These are rough characterizations, that may not be mutually exclusive, but each can be done to fit the description above.

To me the necessary elements are that it’s readable, authoritative, and worthwhile. That is, first there’s a narrative flow that makes it easy to process. For instance, Annie Murphy Paul’s The Extended Mind takes a journalistic approach to important phenomena.  Also, a book needs an evidence-base, grounding in documented experience and/or science. It can re-spin topics (I’m thinking here about Lisa Feldman Barrett’s  How Emotions Are Made), but must have a viable reinterpretation. Finally, it has to be something that’s worth covering. That may differ by reader, but it has to be applicable to  a field. You should leave with a new perspective and potentially new capabilities.

That’s what came off the top of my head. What am I missing in what makes a good book?

The Performance Ecosystem and L&D

by 2 Comments

On LinkedIn recently, a survey in a post asked whether L&D should simply become performance consulting (Y/N). In the ensuing discussion, a comment was made that the binary discussion was flawed, and that a richer picture was possible. I was extremely pleased when she referred to my  Revolutionize Learning & Development book, and posted a diagram from it. I backed her comment, but it occurs to me that there’s more here, and of course I have a connection. So here’re some thoughts on the Performance Ecosystem and L&D.

To start, she cited how I wanted to move to Performance and Development. Indeed, I’ve posted about it, and included a diagram. In it, performance consulting  is represented, but as she noticed, there’s more. I think performance consulting is great, but…it’s not everything. To me, it only addresses the ‘optimal execution’ side of the picture, and ignores the ‘continual innovation’ opportunity.

To be fair, suggesting that L&D take responsibility for informal learning could be considered a stretch. My argument is simply that informal learning has practices and policies that can optimize outcomes,  and that it’s a necessary component of success going forward. (I note that problem-solving, design, research, and innovation all start without a known answer, so they’re learning too!) It’s not necessarily L&D’s role,  but who else (should) know more about learning?

So, innovation is an opportunity. A big one, I suggest. It’s a chance to move to the most valuable role in the organization, going forward. Orgs  need to innovate, and facilitating the best innovation is going to be a critical role. Why  not L&D? Yes, we have to get out of our comfort zone, start working with other business units, and most importantly know learning. So? We should anyway!

The infrastructure necessary is what I call the performance ecosystem. It’s about formal learning, but also more. That includes social, and information and learning resources. It includes facilitation as well as performance interventions. It’s about technology, but how to use it in ways that align with our brains.

The interesting issue for me is how to awaken this awareness. I  suggest  mobile is a gateway to the appropriate thinking. I wrote about mobile before writing the Revolution book (as my then-publisher required), but even there I laid out the case how mobile was not (just) about formal learning. Indeed, when you look at the way people use mobile, it’s very different. It’s also a digital platform, which means that it supports multiple outcomes.

Thus, mobile thinking is a way to break through the mindset of courses, and start looking at the bigger picture of technology supporting how we think, work, and learn to the success of our organizations. Which is why I’m happy to say that I’ll again be running the mobile course with Allen Academy, starting next week. Through 18 Jan, they’re offering this as a two-fer, so you get both the mobile and the learning science course for one low price! Together, you’re addressing my silly clip about L&D, both doing courses well  and going beyond them.

If you want to get your mind around the performance ecosystem and L&D, I suggest that mobile learning is a effective vehicle. You get both some deep advice about mobile, but it also generalizes to digital technology overall. The course itself looks at formal learning, performance support, informal learning, and more, as well as strategic issues. Coupled with learning science, this is a real grounding in the most important opportunities and necessities facing L&D today. Whether you call it P&D or L&D, these are core concepts. Hope to see you there!

 

Relevant and anchored

by 3 Comments

In reviewing my forthcoming book on Make It Meaningful, I’m poring over my Education -Engagment Alignment (EEA). I’m rewriting part to revisit it. Which I’ve done, but in doing so I had a revelation. I’ve maintained that they’re independent elements. However, I now see Relevant and Anchored as complementary components.

‘Anchored’, in my terminology, is ensuring the learning outcome meets a real need. It’s about the relationship between the learning objective and the performance gap. If you’re trying to get better at something, for instance, the objective is specifically related. If you’re learning about dealing with customer objections, you’ve got a specific objective to use a particular approach. It is not  ‘understand’ but ‘do’. You’re not anchoring if your root cause of the performance gap isn’t a lack of skill. If the learning covers information that’s ‘nice to know’, you’re not anchored. This is determined, by the way, by a performance consulting process.

‘Relevant’, again the way I term it, is about whether the learner  cares about that learning objective. If learners don’t care about being a repair tech, having an objective about the problem-solving process can’t matter. This is something we should design into the experience. That is, we should be helping learners ‘get’ that the consequences of acquiring this skill matter to them. We can use curiosity, or consequences, or…but we should  not leave it to chance!

Using the usual present/absent two-factor diagram, it looks like this:

That is, if you have neither the effort is worthless. Which is like a lot of what we see! When you’re anchoring, but not being relevant, the solution is likely to be moderately effective (tho’ not as much as it could be). People stay away if possible! If it’s relevant but not anchored, it’ll be engaging, but not effective and not meaningful. This is the typical tarted up stuff, aka well-produced but not well-designed and produced. However, if you get both in there, you’ve truly made it meaningful.

I suggest you want relevant  and anchored. If we’re putting in the effort, we should be aligning both. I suggest LXD means the elegant integration of learning science with engagement. It sticks better!  We know how to do this, reliably and repeatedly. Our learning doesn’t have to be dull nor ineffective, and we owe our learners this.

The case for good timekeeping

by Leave a Comment

It occurs to me that maybe not everyone has the same view of timekeeping that I do. So I thought I’d make the case for good timekeeping. To me, it’s about  coping with change.

To me, it starts with respect. This includes respect for the audience, the speaker, topic, and context. There are times when timekeeping  should be lax, I believe. My first take is that the defining circumstances are when there are no people involved who aren’t already part of it, there’s not a fixed time agenda, speaker times aren’t set, and the outcome is more important than punctuality.. In other words, rare for a public event.

When an event is public, there tend to be some other constraints. Most importantly, there’s liable to be a schedule. People need to know when to arrive. If it’s a more than one event in the schedule, extending beyond 1 hour, and the audience is diverse (e.g. not just one company), time becomes increasingly important.

Why should we care? Back to the starting point, people might be coming in to see someone in particular. If the schedule isn’t adhered to, folks who were counting on a particular time could be disappointed. If there’s a start and end time, and block, speakers further down the agenda could be impacted if someone runs long. Neither is fair.

I have experienced folks who seem to be unaware of time. In my personal experience as a speaker, I was on a joint presentation where the leadoff presenter seemed to forget that there was anyone else on the agenda! He was gracious once it was pointed out, but it was uncomfortable for me to have to break in and remind him. I’ve also seen people unaware that they were running long, and others unable to amend their presentation on the fly when they  did become aware. I’ve literally seen someone have to stop where they are instead of finding a way to wrap up!

Having had early experience  being  a moderator, after being the victim of such sessions, I made a commitment (in line with the event organizer’s intent) to be rigorous. I’ve now no qualms about, after giving fair warning, stopping someone who hasn’t maintained control. To the contrary of a possible position, I think it’s rude  not to! It’s the speaker’s fault, no one else’s. They aren’t being professional and respectful of the audience and the other speakers. That also includes getting out of the way in a timely manner if it’s a scheduled room, leaving time for the next person to set up.

So my guidelines for timekeeping:

  1. Designate someone as the timekeeper; there can’t be a question over who’s job it is.
  2. Warn speakers ahead of time about the rigor and practices. No excuses!
  3. Have a practice for signaling if things are getting close, e.g. signs for # of mins left, colored lights, messages in chat (tho’ some people seem unable to process them; beware), what have you.
  4. As a presenter, if you don’t have a good basis for assessing your likely length (e.g. I’m about a slide a minute, though with some quick builds it can be faster), practice and check your timing! Realize that live it’s likely to go a bit longer than your practice. Trim if necessary.
  5. Also, have enough awareness of your material that you can adapt on the fly. Sometimes other things happen (once the power went out in a presentation), and you have to adjust.
  6. Be firm; interrupt and stop speakers when it’s time.

That’s off the top of my head; I’m sure there are more comprehensive and thorough lists. My point is to be aware, and prepared. As a speaker, I appreciate it. As an audience member as well. That’s my case for good timekeeping.  What do you think?

Coping with change

by Leave a Comment

It’s an  interesting  time for Quinnovation. New things are afoot business wise, and some personal changes are underway as well. All end up being natural and/or for the good, but…seldom are things as easy as you’d hope and expect. So here are some reflections on coping with change.

First, there are some changes afoot for Quinnovation. If all comes together as planned, there’ll be some updates come the new year. Yet these changes, welcomed as they are, will require me to adopt some new work practices. For instance, my attempts at project management will need to become more effective. I’ve been working on it, but I need to instill some new habits. I’m putting in place some steps, but…not everything works as well as you’d hope. Useful watchwords here are concepts like acceptance and agility.

Which is also true for some expected, but significant personal things. Of course, these are more complicated. They entail large bureaucracies, not just leveraging interpersonal relationships with the principals. Thus, you run into all the flaws of complicated systems, and things fall through the cracks. They’re also largely immovable objects, so remedying problems is an exercise in calm. Watchwords here might be patience and persistence.

Of course, some are just new events in familiar guises, such as new clients, e.g. Quinnovation as usual. Here the mechanisms are familiar, but details are different. This requires reliance on updating familiar patterns to new circumstances. Useful watchwords here are adaptation and execution.

I’m reminded of April Rinne’s Flux, which I recently reviewed. It’s a good time for me to review her 8 superpowers for coping with change. I’m positive and committed, but I also have to acknowledge the shift from my comfort zone. However, that’s what learning’s all about, right? It’s partly formal and partly informal learning, yet this is what I  do. So here’s to more learning!