Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Impactful decisions

by 1 Comment

I’ve been talking about impact in a variety of ways, and have also posited that decisions are key. I really haven’t put them together, so perhaps it’s time ;). So here’re some thoughts on impactful decisions.

To start with, I’ve suggested that what will make a difference to orgs, going forward (particularly in this age of genAI), is the ability to make better decisions. That is, either ones we’re not making right now, or new ones we need to be able to make.  When we’re moving away from us doing knowledge tasks (e.g. remembering arbitrary bits of information), our value is going to be in pattern-matching and meaning-making. When faced with a customer’s problems, we’ll  need to match it to a solution. We need to look at a market, and discern new products and approaches. As new technologies emerge, we’ll have to discern the possibilities. What makes us special is the ability to apply frameworks or models to situations despite the varying contexts. That’s making decisions.

To do this, there are several steps. What are the situations and decisions that need to be made? We should automate rote decisions. So then we’ll be dealing with recognizing situations, determining models, using them to make predictions of consequences, and choose the right one. We need to figure out what those situations are, the barriers to success, and figuring out what can be in the world, and what needs to be in the head. Or, for that matter, what we can solve in another way!

We also need to determine how we’ll know when we’ve succeeded. That is, what’s the observable measure that says we’re doing it right. It frequently can be triggered by a gap in performance. It’s more than “our sales aren’t up to scratch”, but specifics: time to close? success rate? Similarly for errors, or customer service ratings, etc. It needs to be tangible and concrete.  Or it can be a new performance we need. However, we need some way to know what the level is now and what it should be, so we can work to address it.

I note that it may feel ephemeral: “we need more innovation”, or “we need greater collaboration”, or… Still, these can be broken down. Are people feeling safe? Are they sharing progress? Is constructive feedback being shared? Are they collaborating? There are metrics we can see around these components, and they may not be exhaustive, but they’re indicative.

Then, we need to design to develop those capabilities. We should be designing the complements to our brain, and then developing our learning interventions. Doing it right is important! That means using models (see above) and examples (models in context), and then appropriate practice, with all the nuances: context, challenge, spacing, variation, feedback…  So, first the analysis, then the design. Then…

The final component is evaluation. We first need to see if people are able to make these decisions appropriately, then whether they’re doing so, and whether that’s leading to the needed change. We need to be measuring to see if we’re getting things right after our intervention, it’s translating to the workplace, and leading to the necessary change.

When we put these together, in alignment, we get measurable improvement. That’s what we want, making impactful decisions. Don’t trust to chance, do it by design!

Engineering solutions

by 1 Comment

Every once in a while, I wonder what I’m doing (ok, not so infrequently ;). And it’s easy to think it’s about applying what’s known about learning to the design of solutions. However, it’s more. It is about applying science results to designing improvements, but, it’s broader than learning, and not just individual. Here are some reflections on engineering solutions.

As I’ve probably regaled you with before, I was designing and programming educational computer games, and asking questions like “should we use spacebar and return, or number keys to navigate through menus?” (This was a long time ago.) I came across an article that argued for ‘cognitive engineering’, applying what we knew about how we think to the design of systems. Innately I understood that this also applied to the design of learning. I ended up studying with the author of the article, getting a grounding in what was, effectively, ‘applied cognitive science’.

Now, my focus on games has been on them as learning solutions, and that includes scenarios and simulation-driven experiences. But, when looking for solutions, I realize that learning isn’t always the answer. Many times, for instance, we are better off with ‘distributed‘ cognition. That is, putting the answer in the world instead of in our heads. This is broader than learning, and invokes cognitive science. Also, quite frankly, many problems are just based in bad interface designs!  Thus, we can’t stop at learning. We truly are more about performance than learning.

In a sense, we’re engineers; applying learning and cognitive science to the design of solutions, (just as chemical engineering is about applying chemistry). Interestingly, the term learning engineering has another definition. This one talks about using the benefits of engineering approaches, such as data, and technology-at-scale, to design solutions. For instance, making adaptive systems requires integrating content management, artificial intelligence, learning design, and more.

Historically, our initial efforts in technology-facilitated learning did take teams. The technology wasn’t advanced enough, and it took learning designers, software engineers, interface designers and more to generate solutions like Plato, intelligent tutoring systems, and the like.  I’ve argued that Web 1.0 took the integration of the tech, content design, and more, which usually was more than one person could handle. Now, we’ve created powerful tools that allow anyone to create content. Which may be a problem! The teams used to ensure quality. Hopefully, the shift back comes with a focus on process.

We can apply cognitive science to our own design processes. We’ve evolved many tools to support not making reliable mistakes: design processes, tools like checklists, etc. I’ll suggest that moving to tools that make it easy to produce content haven’t been scaffolded with support to do the right thing. (In fact, good design makes it hard to do bad things, but our authoring tools have been almost the opposite!)  There’s some hope that the additional complexity will focus us back on quality instead of being a tool for quantity. I’m not completely optimistic in the short term, but eventually we may find that tools that let us focus on knowledge aren’t the answer.

I’m thinking we will start looking at how we can use tools to help us do good design. You know the old engineering mantra: good, fast, and cheap, pick 2. Well, I am always on about ‘good’. How do we make that an ongoing factor? Can we put in constraints so it’s hard to do bad design? Hmm… An interesting premise that I’ve just now resurrected for myself. (One more reason to blog!) What’re your thoughts?

 

Getting concrete about mental models

by 1 Comment

Photo by Samuel Cruz on Unsplash of pouring concreteI’ve been on the stump for mental models since, well, I left for grad school. My work colleagues, as a going away present, gave me a copy of Gentner & Stevens’ Mental Models. My Ph.D. thesis also incorporated them. (In it, I looked at how people used such models, and then looked at the effects of training interventions. TL;DR: there were some benefits!)  However, I suspect that the idea is a bit complex, and I tend to err on the side of the conceptual (mea culpa!). So, I thought it was time to be getting concrete about mental models.

For background, mental models are simplified explanations about how some portion of the world works. They’re causal conceptual models that connect the relationships and how effects propagate through systems. Done well, they give us a basis to explain and predict outcomes. For performance, they give us a basis to make decisions; we can run them and see what the various outcomes of choices of actions would be. Then we can choose the best one. And they guide decisions in many different circumstances.

For instance, for something procedural, they tell us why we doing things in a certain order. For instance, when I’m cooking breakfast, I want the onions nicely browned. I know not to put them in before the pan heats, because my mental model of heat transference and vegetable properties lets me know that throwing them into a hot pan will brown them. Instead, if I throw them into a cold pan and raise the heat, they’re likely to just get soft. (There’re nuances around this, of course.) And this can stretch to let me know when to throw in various things, and also why to cook the meat, take it out, and then throw in the veggies. If I forget a step, my mental models can fill in and help me regenerate the necessary step.

Similarly for other decisions. In trouble-shooting, for instance, my knowledge of what makes cars work lets me know that if it’s not running, it’s likely one of two sources. Given that petrol engines require an air/fuel mixture ignited by a spark, I can suspect that either the fuel or electrical system is at fault. Then I can take steps to test each. (Ok, at least I could, back in the days of caburetors and distributors. Electronic ignition and fuel injection have thrown off my game!) Knowing the causal properties let me break down the possible contributors. It’s not going to be the brakes!

And so on. In fact, models guide many types of decisions. Good models, based on empirical research, such as cognitive load theory, give us reasons to do things like precede practice with examples. (Bad models, of course, like learning styles, lead us to waste time and money chasing unobtainable benefits.)

Research on mental models tells us several things. For one, our brains are always building models. It also tells us that once we have a model, we’re reluctant to replace it (no “please sir, may I have another model”), and instead, patch it. Thus, if we want to support the optimal performance, we should provide good models, and build their relevance through examples that show them, and addressing them in feedback.

I hope that the two examples here make the use of models a bit more comprehensible. The goal, of course, is that you start paying more attention to them in your learning designs. They’re not necessarily obvious to elicit, but they are there. Do try to find them, represent them explicitly, and refer to them in examples and feedback. Oh, and do avoid the bad ones (you can look to the research translators for guidance). They’re part of what makes us best at performing optimally, and so should be part of your learning solutions. (In other words, build and use your model of mental models! :)

Nuances of aligning

by Leave a Comment

I apparently talk about alignment a lot. There’re good reasons, of course. First, I am referring to two different alignments. One is aligning organizations with the way our brains work. Otherwise, we won’t get the best out of people. The other is aligning our learning experience designs with what we know about how our brains work. Also critically important. In this case, prompted as always by conversations, I realized that I wanted to explore the nuances of aligning for the latter.

So, I’ve talked before about how  we should make sure we have meaningful objectives, and then align practice to that. I’ve also become enlightened about how important examples are as well. But within both of those, there’s more.

It came up in reviewing a design, looking to refine the approach. In it, there were examples, but they weren’t being used quite systematically enough. Then, the practice also wasn’t quite reflecting what people did. In the course of the conversation, I realized that there were nuances that seemed to be missing.

When you’re focusing on performance, you should be looking at what people will need to be doing. Too often, folks can talk about what they want people to know. However, what matters is what people do. Thus, you really need to dig down into that.

Then, you need to be making sure your examples show people doing whatever it is they need to be able to do. Similarly, you need to be asking people to be doing that, as well. For good examples, they should have a narrative flow, and show the underlying thinking. Good practice should require contextualized decision making like they’ll have to actually perform. Not characterizing the situation, but making decisions based upon those situations. So, not saying “is this an X or a Y situation”, but instead “do you choose action A or B”?

Then, of course, there are the actual choices of situation. The first task should be elementary. It may require scaffolding, so the circumstances might be simple, or some of the task is performed, etc. Then, you systematically add complexity in the task, while also broadening the situations seen. You’re simultaneously supporting both the acquisition of skill, and the ability to transfer to appropriate situations.

Then, of course, you want to make the situations appropriately compelling. That may mean choosing the best stories, some exaggeration, and storytelling. For practice, of course, there’s also the feedback: performance-focused, model-based, and minimal.

Look, I’m not saying this is easy. If it was easy, we’d get AI to do it ;). Yet AI doesn’t, and really can’t, understand the nuances of aligning. We can, and do. Yes, it is somewhat rocket science, done properly. We’re talking about systematically creating change in arguably the most complex thing in the known universe, after all. However, we do have good principles and practices. We just need to make sure we know, and use them.

That’s what makes our field so fascinating and important, after all. The creativity involved is also why it’s fun. Then, we’re also achieving important goals, improving people. We owe it to our stakeholders to do it right. (We are the leaders of the future economy, after all!)  That’s my take, what am I missing?

They nest!

by Leave a Comment

A nestIn a conversation with some colleagues, we were discussing how to handle wrapping up learning. In it, I realized something that I’ve always assumed, but don’t know that I’ve articulated. So, let me explain why I say “they nest!”

For context, we were talking about designing elearning. In this case, we have four modules that make up a full course. We’re using an ongoing story in the first three, and then the fourth includes several scenarios for the learner.

The question that prompted my reflection was an ask about how to wrap up the story. My inference was that the question was whether  we needed to wrap the story at the end of the last module. My answer was that we wrapped up the story at the end of module 3, and kick off module 4 stating that we’re moving on. Then we’d wrap up the whole experience at the end of the fourth module.

What I realized was that I haven’t articulated that each learning experience itself has an opening and a closing. So, when smaller learning experiences are embedded in a larger one (which isn’t all that unusual), we need to close off the embedded ones. We’ll also need to open the transition to the next embedded learning experience from the previous one. Then we finally close off the last one. That is, we open and close each learning experience on its own.

In short, these experiences embed, or to put it another way, they nest. Which is aligned with how we think about things anyways: for instance the ways we can talk about wheels, or talk about cars and their wheels. This way of thinking about it makes sense to me, how about you?

We can be logical

by Leave a Comment

So, I’ve been on a bit of a crusade saying we’re not formal logical reasoning beings. And, I do think it’s important to emphasize this in the face of some legacy beliefs. On the other hand, I think there’s evidence that we can be logical. So, how do we reconcile this?

The reason I push against a belief that we’re logical is that too often we are designing as if that’s the case. We see it in way too many policies, practices, and the like. Yet, as has been documented, that’s not our default.

On the other hand, we can be effective reasoners. We have created complex mathematics, advanced science, and generally improved our situation. Something is going on. But what?

Well, Kahneman talks about how we, effectively, have two systems, fast and slow. The slow one takes cognitive effort, so we tend to avoid it. The fast one, then, is default. It’s based upon instinct. Which can be good in two situations: one, where our instincts are likely to be right (e.g. dealing with biologically primary information) or where we have expertise. It can also be bad, where we use it inappropriately.

On the other hand, we can use the slow route. It’s hard, but it works.  This is where we reason things out. (We have to be careful, because being hard, we can depend on it inappropriately.) We can use cognitive support, and complementary skills, but we can document the situation, explore alternatives, trial solutions, and reason our way to good decisions.

And we should! Frankly, I’d rather have in office a policy wonk building coalitions of expertise than a solitary ‘profile’ claiming solutions across the board. I want evidence-based approaches, not simplistic and wrong answers to complex problems!

So, we can be formal logical reasoning beings. Under the right circumstances, with the right support. We should automate what we can so we build the necessary expertise, and provide the conditions for good decisions. That can sometimes be fast, and sometimes be slow, but better to be right than to be expedient. Not perfect, of course, but I’m suggesting we err on the side of likelihood.

That’s my view, at any rate. We can be logical, and that’s a matter of design. We should evaluate and optimize situations so we get the best decisions. That recognizes when training is helpful, when performance support can be used, and when we should support good innovation (problem-solving, research, design, etc). So let’s take a healthy informed look at how we make decisions, and increase the likelihood of good ones. That’s my decision, at any rate. What’s yours?

For ‘normals’

by 5 Comments

So, I generally advocate for evidence-based practices. And, I realized, I do this with some prejudice. Which isn’t my intent! So, I was reflecting on what affects such decisions, and I realized that perhaps I need a qualification. When I state my prescriptions then, I might have to add “for ‘normals'”.

First, I have to be careful. What do I mean by ‘normal’? I personally believe we’re all on continua on many factors. We may not cross the line to actively qualify as obsessive-compulsive, or attention-deficit, or sensorily-limited. Yet we’re all somewhere on these dimensions. Some of us cross some or more of those lines (if we’re ever even measured; they didn’t have some of these tests when I was growing up). So, for me, ‘normal’ are folks who don’t cross those lines, or cope well enough. Another way to say it is ‘neurotypical’ (thanks, Declan).

What prompted this, amongst other things, is a colleague who insisted that learning styles did matter. In her case, she couldn’t learn unless it was audio, at least at first. Now, the science doesn’t support learning styles. However, if you’re visually-challenged (e.g. legally blind), you really can’t be a visual learner. I had another colleague who insisted she didn’t dream in images, but instead in audio. I do think there are biases to particular media that can be less or more extreme. Of course, I do think you probably can’t learn to ride a bicycle without some kinesthetic elements, just as learning music pretty much requires audio.

Now, Todd Rose, in his book The End of Average, makes the case that no one is average. That is, we all vary. He tells a lovely story about how an airplane cockpit carefully designed to be the exact average actually fit no one! So, making statements about the average may be problematic. While we’ve had it in classrooms, now we also have the ability to work beyond a ‘one-size fits all’ response online. We can adapt based upon the learner.

Still, we need to have a baseline. The more we know about the audience, the better a job we can do. (What they did with cockpits is make them adjustable. Then, some people still won’t fit, at least not without extra accommodation)  That said, we will need to design for the ‘normal’ audience. We should, of course, also do what we can to make the content accessible to all (that covers a wide swath by the way). And, while I assume it’s understood, let me be explicit here that I am talking “for ‘normals'”. We should ensure, however, that we’re accommodating everyone possible.

Facilitating in the dark

by 1 Comment

I recently spoke to the International Association of Facilitators – India, having chosen to focus on transfer. My intent was for them to be thinking about ensuring that the skills they facilitate get applied when useful. My preparation was, apparently, insufficient, leaving me to discover something mid-talk. Which leads me to reflecting on facilitating in the dark.

So, I’m not a trained facilitator (nor designer, nor trainer, nor coach). While I’ve done most of this (with generally good results),I’m guided by the learning science behind whatever. So, in this case, I thought they were facilitating learning by either serving as trainers or coaches. Imagine my surprise when I found out that they largely facilitate without knowing the topic!

In general, to create learning experiences, we need good performance objectives. From there, we design the practice, and then align everything else to succeed on the final practice. We also (should) design the extension of the learning to coaching past any formal instruction, and generally ensuring that the impact isn’t undermined.

How, then, do you get models, examples, and provide feedback on practice if you don’t know the domain? What they said was that they were taking it from the learners themselves. They would get the learners together and facilitate them into helping each other, largely. This included creating an appropriate space.

To me, then, there are some additional things that need to be done. (And I’m not arguing they don’t do this.) You need to get the learners to:

  • articulate the models
  • provide examples
  • ensure that they articulate the underlying thinking
  • think about how to unpack the nuances
  • ensure sufficient coverage of contexts
  • provide feedback on others’ experiences

This is in addition to creating a safe space, opening and closing the experience, etc.

So it caused me to think about when this can happen. I really can’t see this happening for novices. They don’t know the frameworks and don’t have the experience. They need formal instruction. Once learners have had some introduction and practice, however, this sort of facilitation could work. It may be a substitute for a community of practice that might naturally provide this context. You’d just be creating the safe space in the facilitation instead of the community.

The necessary skills to do this well, to be agile enough mentally to balance all these tasks, even with a process, is impressive.  I did ask whether they ended up working in particular verticals, because it does seem like even if you came in facilitating in the dark, you couldn’t help learn while doing the facilitation. There did seem to be some agreement.

Overall, while I prefer people with domain knowledge doing facilitation, I can see this. At least, if the community can’t do it itself. We don’t share enough about learning to learn, and we could. I do think a role for L&D is to spread the abilities to learn, so that more folks can do it more effectively. The late Jay Cross believed this might be the best investment a company could make!

Nonetheless, while facilitating in the dark may not be optimal, it may be useful. And that, of course, is really the litmus test. So it was another learning opportunity for me, and hopefully for them too!

 

Myths are models

by 4 Comments

A recent LinkedIn post talked about how models are good, but myths are bad. Which was a realization for me. I’ve kept myths and models largely separate in my mind, but I realize that’s not the case. Myths are models, just wrong ones. And, I suppose, we need to deal with them as such. (Also, folks hang on to myths and models if they’re tied up with identity, but we should still be able to deal with the logical rationale.)

So, I’m an advocate for mental models. There are a variety of reasons, personal, pragmatic, and principled. Personally, I was gifted a book on mental models by my workmates as I left for graduate school. Pragmatically, they’re useful. On principle, they’re how we reason about the world. Heck, our brains are constantly building them!

The important aspects of models are that they’re predictive (and explanatory). That is, they tell us the outcomes of actions in particular situations. They are models of a small bit of the world, and are used to understand a perturbation of the model. They’re causal, in that they talk about how the world works, and conceptual in that they talk about the elements of the world. They’re incomplete, in that they only need to account for the parts of the world relevant to the particular situation.

Examples include using an analogy of water flowing in pipes for thinking about electric circuits. Or how advertisements use association with valued things or people to induce a positive affect. You can use them to explain what happened, or what will happen. It’s the latter that’s important for the purposes of providing a basis for guiding decisions, and thus their role in learning.  They guide us in deciding how to take actions under different circumstances.

Models can be good or bad. The old ‘planets circling a sun’ model of electrons in orbit around a nucleus of protons and neutrons turned out to be inaccurate as our understanding increased. We then moved to probability clouds as a better model. Many of our mistakes come from using the wrong model, for a variety of reasons. We can mistake the situation, or think a model is accurate and useful.

We should avoid models that aren’t appropriate for the situation. Myths are models that aren’t appropriate for any situation. So, for instance, learning styles, generations, ‘attention span of a goldfish’, and ‘images are processed 60K faster than prose’ are examples of myths. They lead people to make decisions that are erroneous, such as providing different learning prescriptions. They are models, because they do categorize the world and lead to prescriptions about what to do. They’re myths, because their implications will lead to decisions that waste time and money.

As the saying goes, “all models are wrong, but some are useful”. They’re wrong because they’re only part of the world. The good ones give us useful predictions, The bad ones lead us to make bad decisions. The useful ones are to be lauded, shared, and used. Myths, however, should be debunked and avoided. Myths are models, but not all models are good. It’s important that I remember that!

Quality or Quantity?

by 4 Comments

Recently, there’s been a lot of excitement about Generative Artificial Intelligence (Generative AI). Which is somewhat justified, in that this technology brings in two major new capabilities. Generative AI is built upon a large knowledge base, and then the ability to generate plausible versions of output. Output can in whatever media: text, visuals, or audio. However, there are two directions we can go. We can use this tool to produce more of the same more efficiently, or do what we’re doing more effectively. The question is what do we want as outcomes: quality or quantity?

There are a lot of pressures to be more efficient. When our competitors are producing X at cost Y, there’s pressure to do it for less cost, or produce more X’s per unit time. Doing more with less drives productivity increases, which shareholders generally think are good. There’re are always pushes for doing things with less cost or time. Which makes sense, under one constraint: that what we’re doing is good enough.

If we’re doing bad things faster, or cheaper, is that good? Should we be increasing our ability to produce planet-threatening outputs? Should we be decreasing the costs on things that are actually bad for us? In general, we tend to write policies to support things that we believe in, and reduce the likelihood of undesirable things occurring (see: tax policy). Thus, it would seem that if things are good, go for efficiency. If things aren’t good, go for quality, right?

So, what’s the state of L&D? I don’t know about you, but after literally decades talking about good design, I still see way too many bad practices: knowledge dump masquerading as learning, tarted up drill-and-kill instead of skill practice, high production values instead of meaningful design, etc. I argue that window-dressing on bad design is still bad design. You can use the latest shiny technology, compelling graphics, stunning video, and all, but still be wasting money because there’s no learning design underneath it.  To put it another way, get the learning design right first, then worry about how technology can advance what you’re doing.

Which isn’t what I’m seeing with Generative AI (as only the latest in the ‘shiny object’ syndrome. We’ve seen it before with AR/VR, mobile, virtual worlds, etc. I am hearing people saying “how can I use this to work faster”,  put out more content per unit time”, etc, instead of “how can we use this to make our learning more impactful”. Right now, we’re not designing to ensure meaningful changes, nor measuring enough of whether our interventions are having an impact. I’ll suggest, our practices aren’t yet worth accelerating, they still need improving! More bad learning faster isn’t my idea of where we should be.

The flaws in the technology provide plenty of fodder for worrying. They don’t know the truth, and will confidently spout nonsense. Generative AIs don’t ‘understand’ anything, let alone learning design. They are also knowledge engines, and can’t create impactful practice that truly embeds the core decisions in compelling and relevant settings. They can aid this, but only with knowledgeable use. There are ways to use such technology, but it comes from starting with the point of actually achieving an outcome besides having met schedule and budget.

I think we need to push much harder for effectiveness in our industry before we push for efficiency.  We can do both, but it takes a deeper understanding of what matters. My answer to the question of quality or quantity is that we have to do quality first, before we address quantity. When we do, we can improve our organizations and their bottom lines. Otherwise, we can be having a negative impact on both. Where do you sit?