Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Emphasis and Effort

by 1 Comment

For reasons that aren’t quite clear (even to me), I was thinking about where, on a continuum, do L&D elements fit? Where does performance support go? Formal learning? Informal learning? I began to think that it depends on what focus you’re thinking of. So here’re some nascent thoughts on emphasis and effort.

To start with, I generally think of formal learning as the starting point. For instance, in thinking about performance & development (as an alternative to learning & development), I put training first. Similarly, in my strategy work, I likewise suggest the first step is to put learning science more central in training. Here, the order is:

  • Formal Learning
  • Performance Support
  • Informal Learning

I’m looking as much as where we typically start. This may well be because training is always the line of first-response (throw training at it!). Also perhaps because it’s familiar (it looks like school).

However, in another cut at it, I started with performance support. Here, I was thinking more about the utility to achieve goals rather than the way L&D allocates resources. That is, from a performer’s perspective, if the answer can be in the world, it should. I can use a tool to achieve my goal rather than have to take a course. Still, taking a pre-digested course is easier than having to work together to collaborate and solve it. Of course, if someone else has the answer, just asking and getting it is easier than working to create an unknown answer. (So, do I need to separate out communication from collaboration? Hmm…) Thus, the list here might be:

  • Performance Support
  • Formal Learning
  • Informal Learning

However, if I look at it from the effort required from L&D, a new order emerges. Here, formal learning is hardest. That is, if you’re doing it right. To successfully get a persistent change in the way someone does something is harder than even facilitating informal learning, and performance support is easiest. Not saying that any are trivial, mind you, designing good job aids isn’t easy, it’s just not as hard as designing a whole course. Then the list comes out like this:

  • Performance Support
  • Informal Learning
  • Formal Learning

I guess there isn’t one answer. To do this successfully, however, requires an understanding how to do all of the above, and then apply as priorities demands. If you’ve expert performers, you’ll do something different than if you have high turnover. If you’re doing something complex, your design strategies may differ from something important. However, you do need to know the tradeoffs in emphasis and effort to make the right calls. Am I missing something important here?

Working with SMEs

by 1 Comment

In a recent post, I talked about how expertise is compiled away, and the impact on designing learning and documentation. Someone, of course, asked  how do you then work with SMEs to get the necessary information. Connie Malamed, one of our recognized research translators, has recently written about getting tacit knowledge, but I also want to address the more usual process.  I thought I’d written about it somewhere, but I can’t find it. So here are some thoughts on working with SMEs.

First, I’ve heard from several folks experienced in this that any one SME may not have both necessary elements. One element is to have a good model to guide the performance. The second elements is the ability to articulate that model. Their solution is to work with SMEs in groups. Guy Wallace (Eppic), Roger Schank (Socratic Arts), and Norman Bier (CMU) have all mentioned to me that they’ve found utility in getting SMEs together as a group and having that knowledge negotiation unpack the necessary learnings. They’re all folks worth listening to. You have to manage the process right, of course, but if you can do it, it’s useful.

I suggest that you also want several  different types of SMEs. You want not only the top performers, and theoretical experts, you also want just-past novices (also attributable to Guy) and supervisors. Theorists can give you models, while top performers can talk about the practical implications. Novices can let you know what they found hard to understand, and supervisors provide insight into what performers typically do wrong. All are helpful information for different parts of the learning.

Another trick I use is to focus on decisions. I argue that making better decisions will be more important to organizations than the ability to recite knowledge. SMEs  do have access to all the knowledge they learned, and it’s easy for them to focus on that. That’s where you get ‘info dump’ courses and bullet point-laden slides. By using decisions as a focus, you cut through the knowledge. “What decisions will they make differently/better as a result of this knowledge?” is a helpful question.

You can use questionnaires as well. Asking specifically about the elements: models, misconceptions, consequences, can be a good preliminary step before you actually talk to them. Or have a template for content for them to fill out. Any guidance and structure helps keep them focused.

Another preparatory step is to create a draft proposal of the information. You’ll likely be getting a dump of PDFs and PPTs). Process that material, and make your first, best, guess. It’s easier to critique than generate, so if you’re willing to be wrong (and why not), you can have them shoot holes in what you did. You’ll have focused on decisions (right?), and they’ll fix it, but you’ll have biased them for action.

Of course, you want to ensure you test for confirmation. You should circulate what you have learned, and get validation. You’ll need to have clear objectives that operationalize your learnings. You then should prototype and test what you’ve developed and see if it actually changes the behavior in useful ways. Ensure that your focus actually leads to the necessary change.

There are other elements you want from SMEs, such as their personal interest. However, it’s critical that you get them to focus on behavior change. It’s not easy, but it’s part of the job. Working with SMEs, correctly, is key to designing learning experiences that address real needs. These are my thoughts, what are yours?

Experts and Explanations

by 1 Comment

blueprint pencil rulerI’ve been going through several different forms of expert documentation. As a consequence, I’ve been experiencing a lot of the problems with that! Experts have trouble articulating their thinking. This requires some extra work on the part of those who work with them, whether instructional designers, technical writers, editors, whoever. There are some reliable problems with experts and explanations that are worth reviewing.

The start of the problem is that the way we acquire expertise is to take our conscious thinking and automatize it, basically. We want to make our thinking so automatic that we no longer have to consciously think about it. So, we basically compile it away. Which creates a problem. For one, what we process into memory may not bear a close resemblance to what we have heard and applied. That is, the semantic language we use to guide our practice and internalize may not be what we store as we automate it.

It’s also the case that we lose access to that compiled away expertise. There’s evidence of this, for one from the results of research by the Cognitive Technology group at the University of Southern California showing experts can’t access about 70% of what they do! Another piece of evidence is the widespread failure of so-called ‘expert systems’ in the 80s, resulting in the AI winter. Whether the locus of the problem is in what actually gets stored, or access to it, the result is that  what we were told to do, and say we do, may not actually be close to what we actually do.

Another problem is that experts also lose touch with what they grappled with as novices. What they take for granted isn’t even on the radar of novices. So it’s difficult to get them to provide good support for acquiring skills or understanding. Their attempts at explanations for reference of instruction fail.

All told, this leads to systematic gaps in content. I’ve been seeing this manifest in explanations that may say what to do, but not why or how. There may be a lack of examples, and the thinking behind the examples I  do see isn’t there.  There’s also a lack of visual support. They’re not including diagrams when it’s conceptual relationships that need understanding. They’re also not including images when context is needed. They shouldn’t necessarily be blamed, because they don’t need the support and can’t even imagine that others do!

It’s clear that experts should not be the ones doing the explanations. They’re experts, and they have valuable input, but there needs to be a process to avoid these problems. We need tech writers, IDs, and others to work with experts to get this right. Too often we see experts being tasked with doing the explanations, and we live with the consequences.

What to do? One step is to let experts know that their expertise is in their domain, but the expertise in extracting that expertise and presenting it lies in others. To do so convincingly, you’ll need the science about why. For another, know techniques to unearth that underlying thinking. Also allow time in your schedule for this to happen. Don’t think the SME can just give you information; you’ll have to process what you get to rearrange it into something useful. You may also need some sticks and carrots.

As I wrestle with the outputs of experts, here’s my plea. There are wonderful ways experts and explanations can work out, but don’t take it for granted. Don’t give experts the job of communicating to anyone but other experts, or to experts on working with experts to get explanations. Fair enough?

Good and bad advice all in one!

by 2 Comments

I was asked to go to read an article and weigh in. First, please don’t do this if you don’t know me. However, that’s not the topic here, instead, I want to comment on the article. Realize that if you ask me to read an article, you’re opening yourself up to my opinion, good  or bad. This one’s interesting, because it’s both. Then the question is how do you deal with good and bad advice all in one.

This article is about microlearning. If you’ve been paying attention (and there’s no reason you should be), I’ve gone off on the term before. I think it’s used loosely, and that’s a problem because there are separate meanings, which require separate designs, and not distinguishing them means it’s not clear you know what you’re talking about. (If someone uses the term, I’m liable to ask which they mean! You might do the same.).

This article starts out saying that 3-5 minute videos are  not  microlearning. I have to agree with that. However, the author then goes on to document 15 points that are important about microlearning. I’ll give credit for the admission that there’s no claim that this a necessary and complete set. Then, unfortunately, I also have to remove credit for providing no data to  support the claims!  Thus, we have to evaluate each on it’s own merits.  Sorry, but I kinda prefer some sort of evidence, rather than a ‘self-evident’ fallback.

For instance, there’s a claim for brevity. I’ve liked the admonition (e.g. by JD Dillon) that microlearning should be no longer, and no shorter, than necessary. However, there’s also a claim here that it should be “3 – 10 minutes of attention span”. Why? What determines this? Human attention is complex, and we can disappear into novels, or films, or games, for hours. Yes, “Time for learning is a critical derailer”, but…it’s a factor of how important, complex, and costly if wrong the topic is. There’s no one magic guideline.

The advice continues in this frame: there’re calls for simplicity, minimalism, etc. Most of these are good principles,  when appropriately constrained. However, arbitrary calls for “one concept at a time is the golden rule”  isn’t necessarily right, and isn’t based on anything other than “our brains need time for processing”. Yes, that’s what automation is about, but to build chunks for short term memory, we have to activate things in juxtaposition. Is that one concept? It’s too vague.

However, it could be tolerated if some of the advice didn’t fall prey to fallacious reasoning. So, for instance, the call for gamification leans into “Millennials and Gen Z workforce” claims. This is a myth. Gamification itself is already dubious, and using a bad basis as an assumed foundation exacerbates the problem.  There are other problems as well. For one, automatically assuming social is useful is a mistake. Tying competition into the need to compete is a facile suggestion. Using terms like ‘horde’ and ‘herd’ actually feels demeaning to the value of community. A bald statement like “Numbers speak louder than words!” similarly seems to suggest that marketing trumps matter. I don’t agree.

Overall, this article is a mixed bag. So then the question arises, how do you rate it? What do you do? Obviously, I had to take it apart. The desire for a comment isn’t sufficient to address a complex suite of decent principles mixed up with bad advice and justified (if at all) on false premises. I have to say that this isn’t worth your time. There’s better advice to be had, including on microlearning. In general, I’ll suggest that if there’s good and bad advice all in one, it’s overall bad. Caveat emptor!

Generic Thinking Skills?

by 3 Comments

Recently, a colleague asked a few of us about our views on critical thinking skills. This is actually a contentious topic. There are broad claims of the need for them, increasingly, even showing up in job advertisements. On the other hand, researchers and others have weighed in against them, saying that expertise is the only lever. I tend to lump critical thinking skills in with the broader issue of generic thinking skills, so what are the issues?

Upfront, I’ll admit that I like the concept of generic thinking skills. Say, for instance,  learning-to-learn skills. That is, domain independent skills that lead to better approaches. It seems to make sense that, in the absence of specific knowledge, some general approaches are more useful than others. For instance, faced with a new domain, I’d be inclined to expect that systematic experimentation and observation would be better than random trial and error.

On the other hand, prominent psychologists like John Sweller and Paul Kirschner have said that domain-specific skills are the only way to bet. There is significant evidence that expertise matters in successful approaches to problem-solving, and others. While we have some innate skills for domains that are biologically primary, learning in other domains requires expertise.

Is there, then, any evidence for generic skills? Based on Micki Chi’s work on the value of self-explanation, Kate Bielaczyc and others have found that instruction on systematically explaining steps in examples help, across domains. In my own Ph.D. thesis, I trained folks on analogical reasoning skills, and found improvement (for component skills that weren’t a) already ceilinged or b) were perceived to be immutable, across different problem types.

How, then, do we reconcile these conflicting viewpoints? My (self- :) explanation is that it’s a matter of degree, a continuum rather than a dichotomy. The more domain knowledge you possess, the more likely you are to find a good answer. However, what if you’re in a new domain where you don’t have relevant expertise to hand? In that case, I’ll suggest that there are benefits to some approaches over others, and training those general skills is justifiable. That is, general skills are weaker than domain specific skills, but general skills are better than nothing!

We know that there are practices that improve outcomes. For instance, I’ve written about how to, and not to, do brainstorming. Similarly, I believe Harold Jarche’s Seek-Sense-Share model works across domains. Systematic creativity is  not an oxymoron!  That’s the story I’m holding on to about generic thinking skills. What are your thoughts on the topic?

Accreditation?

by 1 Comment

As occasionally happens, I was asked a question on LinkedIn. In this case, it was about my thoughts on accreditation. Also, as occasionally happens, I thought that I’d share my thoughts in this forum, and look for feedback to improve my thinking. So here’re some thoughts on accreditation. I welcome yours!

First, let’s be clear, I am not an expert on accreditation. I haven’t accredited anything, for one ;). I  did look into it, at one point many years ago. I’ve also served on independent board of directors or advisory boards for several entities. In the former case, we have a legal responsibility to provide guidance. In the latter case, we provide the best guidance, but of course the organization isn’t obliged to comply. The former, in particular, serves as a quality check, or a form of accreditation.

When I looked into accreditation for educational institutions, the requirement isn’t about the actual curriculum, but instead that there is a library and that there are processes for review and revision of course offerings. That is, it was about the support for learning and quality processes, not the actual offering. This creates a process support that should ensure quality, yet also the ability to apply this to institutions with a wide variety of offerings.

Institutions can also seek accreditation by organizations in particular areas of curriculum. Offerings in computer science, business, and others for instance, receive review and then can receive approval by bodies that represent the particular field. This depends on the quality of the organization doing the accreditation and their processes, of course.

There’s also accreditation on the quality of the educational process. You can also be reviewed and accredited on the basis of your pedagogy, for instance your online teaching approach. It depends, of course, on what they stipulate as quality, but that’s always going to be the case.

There are, of course, dubious accreditations. It’s not unknown for an organization or collection thereof to establish their own accrediting body that basically rubber stamps the organization(s). Caveat emptor.

In general, I think that having a scrutable external validation is a good check on quality. Whether that makes sense is probably an issue of scope. A small offering of a particular course might benefit from an independent advisory board, which provides some oversight. The larger the organization and the scope of activity, the greater the need for some external validity check.

From the other side, I think a certificate or credentials help the learner signify what they’ve accomplished.  However, without accreditation or at least a scrutable process, how do you know the skill/knowledge is appropriate and accurate? I think accreditation has the potential to be a ‘reality check’ on any offering.

Learning or Performance Strategy

by 1 Comment

Of late, I’m working in a couple of engagements where the issue of learning and performance strategy have come up. It has prompted some thoughts both on my part and the part of my clients. I think it’s worth laying out some of the issues and thinking, and of course I welcome your thoughts. So here are some reflections on whether to use learning or performance strategy as an organizing concept.

In one case, an organization decreed that they needed a learning strategy. Taken with my backwards design diagram  from the learning science book, I was tasked with determining what that means. In this case, the audience can’t be mandated with classes or tutorials. So really, the only options are to support performance in the moment and develop them over time. Thus we focus on job aids and examples. I think of it as a ‘performance strategy’, not a learning one.

In the other case, an organization is executing on a shift from a training philosophy to a performance focus. Which of course I laud, but the powers-that-be expect it to yield less training without much other change. Here I’m pushing for performance support, and the thinking is largely welcome. However, it’s a mindset shift for a group that previous was developing training.

I general, I support thinking that goes beyond the course, and for the optimal execution side of a full ecosystem, you want to look at outcomes and let that drive you. It includes performance consulting, so you’re applying the  right solution to performance gaps, not the convenient one (read: ‘courses’ ;). Thus, I think it makes more sense to talk performance strategy than learning one.

Even then, the question becomes what does such a strategy really entail, whether learning  or performance. Really, it’s about having a plan in place to systematically prioritize needs and address them in effective ways. It’s not  just design processes that reflect evidence-informed principles, though it includes that. It’s also, however, ways to identify and track problems, attach organizational costs and solution costs, and choose where to invest resources. It includes front-end analysis, but also ongoing-monitoring.

It also involves other elements. For one, the technology to hand; what solutions are in use and ensuring a process of ongoing reviews. This includes both formal learning tools including the LMS and LXP, but also informal learning tools such as social media platforms and collaborative documents. Another issue is management: lifecycle monitoring, ownership, and costs.

There’s a lot that goes into it, but being strategic about your approach keeps you from just being tactical and missing the forest for the trees. A lot of L&D is reactive, and I am suggesting that L&D needs to be come proactive. This includes going from courses to performance, as a first step. The next step is to facilitating informal learning and driving innovation in the organization. Associated elements include meaningful measurement  and truly understanding how we learn for a firm basis upon which to ground both formal  and informal learning. Those are my thoughts a learning or performance strategy, what am I missing?

Courtesy, shopping carts, and the organization

by 1 Comment

There’s a popular meme on the internet that I think is kind of apt. It asks whether you’re the type of person who returns your shopping cart. I think this is an important concept, because it illustrates to me one of the facets that make societies, and organizations, work. So let me talk about courtesy, shopping carts, and the organization.

The meme is basically saying do you return the cart, or leave it near your car. You’ve seen the results: parking spaces blocked by an abandoned cart, carts pulled up on curbs into planters, etc. This, to me, is like whether you bring a bag to clean up after your dog (and place it an appropriate receptacle). Or dump your ashtray and car trash on the side of the street. It’s about recognizing a) that there aren’t necessarily folks who have this as a job, and b) it interferes with innocent others, and c) therefore it undermines a pleasant environment. It may have to do with what your cultural expectations are, but I’ll suggest it’s worth the small effort.

Why does this matter? Because it seems to me that societies work better when folks are courteous. When folks respect one another, they find ways to make things work. When they don’t have that respect, they find or stumble into ways to aggravate situations. Now, I get that sometimes being discourteous is a way to get revenge against a real or perceived injustice. Yet, I suggest there are better ways to register your discontent that more accurately target the perpetrators of the injustice. Random acts of discourtesy can lead to perceptions that you’re just a jerk. Because some of it is people thinking that they don’t have to care about other people. I fear it’s getting worse.

The issue I want to address here, rather than a general societal rant, is about what this means in organizations. What does courtesy have to do with working life? I suggest it has to do with creating an environment in which people can work together for organizational success in an optimal way. That is, if we’re helping make the workplace pleasant, we’re making it effective. This means things like offering to help when you’ve useful information to provide. It means paying attention to the organizational norms. I suggest it also means pointing out when those norms aren’t best for positive interaction. Others: Refilling the printer paper tray when you’ve used the last. Taking notes for someone who  has to miss the meeting. Mot missing a meeting if you’ve no real excuse. Keeping meetings on time and on point. Not holding a meeting when there’s a better way.

There are a lot of little courtesies in everyday life. You may have a grudge or grievance, but deal with it appropriately. A lack of courtesy because you’re upset about something else isn’t appropriate or helpful. I reckon it just makes you look like a jerk. That’s my take, what are your thoughts on courtesy, shopping carts, and the organization?

Happy New(s) Year!

by 1 Comment

It’s the new year, and I’ve been hinting for a while that something’s up. Well, now I can announce it. The new year seems like the right time for news, so Happy New(s) Year!

While Quinnovation will continue, I have joined another initiative as well. If you’ve been paying attention, and I hope you have, two summers ago Will Thalheimer & Matt Richter ran a new and different event, the L&D Conference. It had two sessions of things, so it could reach most of the globe, and they were recorded so you could watch after the fact. There were also some asynch workshops. There was an emphasis on evidence-informed speakers and content.

It was successful enough that it convinced Matt and Will that there was the opportunity for a society promoting the same values. At the beginning of 2021, they began this initiative, the Learning Development Accelerator  (LDA). The society has had regular events, articles, also workshops. They also ran the conference again.

However, Will got a full-time job with Tier 1. Thus, trying to keep up with demand was, well, trying. After one unsuccessful attempt, they settled on a second choice to replace Will. I’ve agreed. Thus, I will now be working with Matt (who’s heroically shouldered most of the work) to keep the society going. It’s started, and continued, as all virtual. Which helps support the goal of being quite global. The other emphasis continues, to be evidence-based. I can align with that. I think it’s our obligation as professionals. We need to promote those who are translating research into evidence-informed practices.

There’s a second initiative of LDA as well. Matt thinks LDA should publish, and had offered me the chance to serve as initial publisher (more like Editor-In-Chief). So LDA Press is kicking off. We’re going to have to start slow to grow, but stay tuned for initial publications. The goal is to fill in the gaps of the books we should have, and provide a better relationship with authors. We’re already talking to some folks.

That’s pretty much it. I’ll continue to speak, write, run workshops, and assist clients as Quinnovation as well, but I’ll also be putting energy into what I think is an important contribution. Of course, it’s also about learning, stepping out of my comfort zone. My focus will be on trying to help introduce members to people and ideas they should know. The more we know, the better we can do our job! So that’s my Happy New(s) Year, and wishing you and yours all the best for the coming year.

Time for Reflection

by 1 Comment

My dad used to regale me with this tale of his best friend, who told his new employer when he started: “If you see me with my feet up on my desk and it appears I’m sleeping, I’m not. I’m working. I’ll still do the work of 2 other engineers.” And he did!  I love this story, because it brings out an element that we seem to be losing, the value of taking time for reflection.

Now, he may actually have been sleeping, yet that doesn’t concern me; sleeping is a mechanism for processing, too. What concerns me are folks who can’t be seen to be taking time off from ‘the work’. We’re in a mode where we push people to work harder and faster. We say “work smarter” but don’t tell people what that means!

I’ve spent time in a job early in my career reading (relevant) magazines like Byte, with my feet up on the desk. Yet, I immodestly suggest I cranked out work  at least as fast as my colleagues. I found reading, and now searching for answers, to be a valuable use of my time. Why? Because I’m  learning. I reflect on what I do and how to do it better, learning to do new things that I need to meet my current challenges.

Sure, I do the work. However, I also take walks, put my feet up and ponder, and more. I blog, for instance ;). There are other ways I write as well, and experiment, and look to refine my thinking. Also, I look things up, read books, and generally track my field and answers to specific questions.  My work improves as a consequence. Moreover, we  all benefit from taking time to reflect. It’s documented in the work conducted by Garvin, Edmondson, & Gino as one of the elements of a learning organization.

So, I’ll keep promoting, and practicing, taking time for reflection. I hope you can, too. Moreover, I hope you can help get such time recognized as valuable in your organization. We focus too much on the fast, and as they say: “fast, cheap, or good, pick 2”. I’m not sure fast is always the best solution. Certainly for learning. After all, it is about learning…