design Archives - Page 3 of 79

An outside perspective

14 May 2024 by Clark 1 Comment

Someone reached out to me for a case study on addressing a workplace problem. I was willing, but there’s a small problem; I’ve never had to address a workplace learning problem. At least, in the way most people expect. Instead, I provide an outside perspective. What’s that mean?

So, first of all, I don’t come from an instructional design (ID) background. I did get some exposure to educational approaches when I designed my own undergraduate degree in Computer-Based Education. Yet, there weren’t any ID courses where I was a student. As a graduate student, I took psychology courses on learning. I also read Reigeluth’s survey of ID design approaches. Further, I got a chance to interview the gracious and wise David Merrill. But, again, no formal ID courses were on tap.

On the flip side, I was in a vibrant program that was developing a cognitive science degree, and read everything on learning I could find: behavioral, cognitive, social, neural, even machine learning! I was in my post-doc as they were forming the learning science approach, too, and I was at a relevant institution. Still, no ID. So, I do have deep learning roots, just not ID.

Then, after the post-doc, I taught. That is, practiced learning design, and continued reading and talking ID, and attending relevant conferences. Just not a formal ID course. Then I joined a small startup to design an adaptive learning platform, and then started consulting, but never a workplace learning role inwardly faced.

What that means is that I bring an ‘outside’ perspective to L&D. Which, I think, isn’t a bad thing. I’ve helped firms meet realistic goals in innovative ways, courtesy of not having my thinking pre-constrained. I’ve been able to interpret learning science in practical terms, and infer what ID says (also, I’ve read it and reflected in context on it). So, I’ve talked L&D design, and ID improvements, but from the view of an outsider.

Many times outsiders can bring new perspectives. And, they can be ignorant of all the contextual details. Thus, it’s really important to ask and establish those constraints, and then to be sensitive to the ones that they didn’t mention. (One of the benefits of the court jester was to reframe things in ways that showed the humor in the hidden assumptions.) Still, I’m not apologizing. I think the background I’ve acquired is useful to people who need to meet real goals, and have a decent track record in doing so. I welcome your thoughts on whether an outside perspective is of benefit.

AI as a System

7 May 2024 by Clark Leave a Comment

At the recent Learning Guild‘s Learning & HR Tech Solutions conference, the hot topic was AI. To the point where I was thinking we need a ‘contrarian’ AI event! Not that I’m against AI (I mean, I’ve been an AI groupie for literal decades), I think it’s got well-documented upsides. (And downsides.) Just right now, however, I feel that there’s too much hype, and I’m waiting for the trough of disillusionment to hit, ala the Gartner hype cycle. In the meantime, though, I really liked what Markus Bernhardt was saying in his session. It’s about viewing AI as a system, though of course I had a pedantic update to it ;).

So, Markus’ point was that we should separate out data from the processing method. Markus presented a simple model to think about AI that I liked. In it, he proposed three pieces that I paraphrase:

the information you use as your basis
the process you use with the information
and the output you achieve

Of course, I had a quibble, and ended up diagramming my own way of thinking about it. Really, it only adds one thing to his model, an input! Why?

So I have the AI system containing the process and data it operates on. I like that separation, because you can use the same process on other data, or vice versa. As the opening keynote speaker, Maurice Conti, pointed out, the AI’s not biased, the data is. Having good data is important. As is having the right process to achieve the results you want (that is, a good match between problem and process; the results are the results ;). Are you generating or discriminating, for instance? Then Markus said you get the output, which could be prose, and image, a decision, …

However, I felt that it’s important to also talk about the input. What you input determines the output. With different queries, for instance, you can get different results. That’s what prompt engineering is all about! Moreover, your output can then be part of the input in an iterative step (particularly if your system retains some history). Thus, thinking about the input separately is, to me, a useful conceptual extension.

It may seem a trivial addition, but I think it helps to think about how to design inputs. Just as we align process with data for task, we need to make sure that the input matches to the process to get the best output. So, maybe I’m overcomplicating thinking about AI as a system. What say you?

Engaged and/or Effective

30 April 2024 by Clark Leave a Comment

I’ve regularly talked about how learning can, and should, be ‘hard fun’. Yet, I haven’t really talked about each, effectiveness and engagement, independently. Of course, there’s a quadrant map that separately talks about engaged and/or effective. Let me remedy the lack!

The lack of either engagement or effectiveness is relatively rare, thankfully. You do see it, when under-skilled and under-resourced folks are making a course. For instance giving SMEs authoring tools or dumping a bunch of PPTs and PDFs on an inexperienced instructional designer. Or, when folks won’t spend enough to even get production values, let alone actual effectiveness. What you get is information dump (because experts don’t have access to what they actually do, research tells us), but not with professional polish. It’s ‘content’ without distinction. More importantly, if there is practice, it’s on knowledge retrieval rather than knowledge application. Which leads to what in cognitive science is called ‘inert knowledge’. It may be remembered, but it won’t be used when relevant.

We also see a lot of ‘tarted up’ information dump. Here, there are good production values. It looks nice, because it’s well-produced. However, it’s still information (usually with a quiz). Here, folks know a bit about visual design, and use tools and templates that make it look good. They may even have experienced designers on staff, but…time and cost expectations keep folks from doing the right thing. It could also be a lack of understanding of the importance of challenging contextual practice. That’s all too common, too! It’s still a trivial pursuit.

Quite simply, learning needs to be effective. If it’s not, we’re wasting money. Now, that’s been shown to be the case in many ways. Over the years, we’ve heard estimates from 10-15% of our training efforts are working. Which means we’re wasting 85-90% of our investment. Yet we know what leads to good learning (e.g. the Serious eLearning Manifesto). Learning science gives us good guidelines, but we still see too much information dump. Yet, if it’s not engaging, learners aren’t likely to commit appropriately, and we’re not optimizing the outcomes. It just seems like work.

When we understand the necessary alignment between engagement and effectiveness, however, we truly can deliver ‘hard fun’. That alignment is what my research and design efforts yielded. It was also the core of my book on serious game design and my most recent tome on making learning meaningful. (The latter is really a complement to my learning science book, and an attempt to bring both together to do learning experience design.)

It’s not necessarily easy to generate ‘hard fun’, nor is it the cheapest option. However, it gets easier with practice (like most things), and it’s the most cost-effective option. That is, if you truly want results. But if you don’t, why are you bothering? There are requirements, like making sure you have a real learning need, but that should be true, regardless. You shouldn’t be asking about engaged and/or effective, you should be shooting for both. Right?

Daymond John #LHRCon Keynote Mindmap

24 April 2024 by Clark Leave a Comment

Daymond John opened the second day of the Learning Guild’s Learning & Human Resources Technology conference. He spoke on being an entrepreneur.

In an entertaining presentation, he wove his life story with some take-home lessons. (I missed the last one, but my neighbor caught it.)

Support retention and transfer

23 April 2024 by Clark Leave a Comment

In a discussion we were having with David Ganulin on marketing, my colleague Matt Richter ended up talking about how many ‘team building’ activities don’t work. The typical model is an event where folks get together off-campus and face challenges together. They have to work together to overcome the challenges. Yet, Matt’s claim was that the empirical evidence was that the results didn’t transfer back to the workplace. What does it take? How can we support transfer to achieve persistent results?

The classic model is the ‘ropes course’. Folks have to work together to get everyone safely across some challenge. By working together to achieve success, you should build team cohesion and respect the different capabilities of your colleagues. Yet, investigations suggest that what’s learned doesn’t carry back to the workplace. People who got along, when they get back to the workplace, can be surprised and disappointed that the same conflicts exist.

What’s happening, of course, is context-specificity. The resulting benefits worked in the context of the team-building, but it’s not the same context as work. Just like the ‘brain training’ exercises didn’t transfer to other tasks, so to any learning is likely to dissipate quickly and still not transfer to another context. What do we need to do, then, to generate retention over time and support transfer to the workplace as well?

For one, we need more than one practice. I just read the results of interesting research suggesting two stages of memory. The first stage says initial memories can last briefly, but for sustained retention, you need a second stage of retrieval practice. Yes, we should know that, but too often we don’t practice it! (Which also suggests that a test at the end of a learning event may not be a good indicator!) Also, I’ll suggest, if we want appropriate transfer, we have to engineer it.

How do you engineer transfer? I’ll posit two steps. For one, you need experience across several different contexts. So, do task A together, then B which is widely different, then C, which is different again. You could do a task that requires different physical attributes (tall, small, strong, heavy), and then one that requires different creative approaches (art, music, prose). Along the way, you reinforce a particular team approach that works across contexts. You facilitate reflection, as well, on what’s common.

Matt went further, suggesting that then you need to take that facilitation back to the workplace, and I’ll agree that it’d be ideal. If you then brought the models back to the workplace and facilitated their application to situations at work, you could extend the internalization and appropriate re-contextualization of the learning.

One-shot events are unlikely to generate the sustained transfer you need, at least not without specific design and support. If you’re not trying to achieve retention (over time after the event until needed) and transfer (to all appropriate and no inappropriate) situations, why bother? If you do want retention and transfer (and you should), design for it. Specifically engineer to support retention and transfer. Use spaced repetition with increased challenge to achieve the former. Use contextual variance and reflection facilitation to support the latter. When you do, you’ll have outcomes worth the investment.

Conti #LHRCon Keynote Mindmap

23 April 2024 by Clark Leave a Comment

Maurice Conti opened the Learning Guild’s Learning Solutions & Human Resources Technology conference (April ’24 Orlando). He proceeded to talk about the implications of artificial intelligence (AI) for Human Resources (HR).

He started with the history of AI, and proceeded to focus on Generative AI. His core message was that you needed three things: your toolset, your skillset, and your mindset. He evangelized for people be optimistic, not stressed.

More on coaching

16 April 2024 by Clark Leave a Comment

Recently, the LDA had a debate about coaching, following on the podcast interview. The wise Emma Weber represented the pro argument, while the LDA’s own Matt Richter was con. (Note that these are false divides, we explore the topic for the sake of unpacking issues.) Superb moderating from Kat Koppett was a bonus! As the discussion went, it uncovered more on coaching, without yielding any finality (for reasons we’ll explore).

So, one of the problems emerged immediately, getting into definitions. Matt pushed a bit on the ‘like sport’ notion, where coaching has lots of specific knowledge, while Emma was more on the domain-independent side of coaching. What emerged was that different people have different definitions. Some folks (like me) put coaching further on the domain-dependent side, with mentoring being the more abstract. However, it’s clear others view coaching as the more advanced and deeper side.

This divide isn’t new, but it does provide some barriers, not least to research! As that issue came up, Kat pointed us to a study that began by saying “However, the coaching research suggests a large variety of processes and outcomes, lacking clarity on the primary psychological dimensions most impacted.” Their meta-analysis suggested that “executive coaching is a powerful instrument for organizations to support positive change and personal development.” Which is a good thing, for sure. Their definition does seem to err more on the general side, which is interesting. And, to my own understanding, an important lesson.

One issue that stuck with me was thinking through the range of development. After the formal learning experience, I think there’re times when folks need to be observed, and provided some feedback as they perform. It became clear that the domain-independent model wants the learner to recognize for themselves when they’re not doing well and need to ask for assistance. Yet, a crucial inflection point is making that transition, and I believe that folks aren’t there right away. Similarly, we may not have the resources to add in all the complexities to a particular model for this task initially. So, we expect coaches (read: supervisors and managers) to help develop understanding. Maybe that’s not coaching, by definition, but it’s a task.

I’ll agree at some point you can start guiding folks to their own improvements, but I suspect that only comes when some base level of understanding is reached. We should be clear about this type of interaction as well as the one advocated for coaching! Similarly, we need clarity on labelling! We didn’t end up coming to any finality on that, sadly.

An issue I hadn’t thought about, but became important in the discussion is the issue of appropriate coaching. Clearly, some approaches to coaching don’t work . Knowing when you can expect the coachee to be capable of domain-independent coaching would be one important criteria. Knowing how to ask questions appropriately is another. My concern here is that there are a fair few models about coaching, and with the terminological and empirical barriers, how do you determine the best methods? If we’re to be evidence-based, how can we be?

I can’t say we came to any conclusions, but I do feel we unpacked more of the issues, and did give ourselves some guidance as to what to do when, even if we don’t have agreed upon names for it all yet. Coaching is important, of both types. The data from that study shows coaching can help. We know also that extending the learning experience through feedback on performance helps. We just need to figure out how best to combine them so we know more about coaching. Those are my thoughts , at least, I look forward to yours.

Impactful decisions

2 April 2024 by Clark 1 Comment

I’ve been talking about impact in a variety of ways, and have also posited that decisions are key. I really haven’t put them together, so perhaps it’s time ;). So here’re some thoughts on impactful decisions.

To start with, I’ve suggested that what will make a difference to orgs, going forward (particularly in this age of genAI), is the ability to make better decisions. That is, either ones we’re not making right now, or new ones we need to be able to make. When we’re moving away from us doing knowledge tasks (e.g. remembering arbitrary bits of information), our value is going to be in pattern-matching and meaning-making. When faced with a customer’s problems, we’ll need to match it to a solution. We need to look at a market, and discern new products and approaches. As new technologies emerge, we’ll have to discern the possibilities. What makes us special is the ability to apply frameworks or models to situations despite the varying contexts. That’s making decisions.

To do this, there are several steps. What are the situations and decisions that need to be made? We should automate rote decisions. So then we’ll be dealing with recognizing situations, determining models, using them to make predictions of consequences, and choose the right one. We need to figure out what those situations are, the barriers to success, and figuring out what can be in the world, and what needs to be in the head. Or, for that matter, what we can solve in another way!

We also need to determine how we’ll know when we’ve succeeded. That is, what’s the observable measure that says we’re doing it right. It frequently can be triggered by a gap in performance. It’s more than “our sales aren’t up to scratch”, but specifics: time to close? success rate? Similarly for errors, or customer service ratings, etc. It needs to be tangible and concrete. Or it can be a new performance we need. However, we need some way to know what the level is now and what it should be, so we can work to address it.

I note that it may feel ephemeral: “we need more innovation”, or “we need greater collaboration”, or… Still, these can be broken down. Are people feeling safe? Are they sharing progress? Is constructive feedback being shared? Are they collaborating? There are metrics we can see around these components, and they may not be exhaustive, but they’re indicative.

Then, we need to design to develop those capabilities. We should be designing the complements to our brain, and then developing our learning interventions. Doing it right is important! That means using models (see above) and examples (models in context), and then appropriate practice, with all the nuances: context, challenge, spacing, variation, feedback… So, first the analysis, then the design. Then…

The final component is evaluation. We first need to see if people are able to make these decisions appropriately, then whether they’re doing so, and whether that’s leading to the needed change. We need to be measuring to see if we’re getting things right after our intervention, it’s translating to the workplace, and leading to the necessary change.

When we put these together, in alignment, we get measurable improvement. That’s what we want, making impactful decisions. Don’t trust to chance, do it by design!

Engineering solutions

19 March 2024 by Clark 1 Comment

Every once in a while, I wonder what I’m doing (ok, not so infrequently ;). And it’s easy to think it’s about applying what’s known about learning to the design of solutions. However, it’s more. It is about applying science results to designing improvements, but, it’s broader than learning, and not just individual. Here are some reflections on engineering solutions.

As I’ve probably regaled you with before, I was designing and programming educational computer games, and asking questions like “should we use spacebar and return, or number keys to navigate through menus?” (This was a long time ago.) I came across an article that argued for ‘cognitive engineering’, applying what we knew about how we think to the design of systems. Innately I understood that this also applied to the design of learning. I ended up studying with the author of the article, getting a grounding in what was, effectively, ‘applied cognitive science’.

Now, my focus on games has been on them as learning solutions, and that includes scenarios and simulation-driven experiences. But, when looking for solutions, I realize that learning isn’t always the answer. Many times, for instance, we are better off with ‘distributed‘ cognition. That is, putting the answer in the world instead of in our heads. This is broader than learning, and invokes cognitive science. Also, quite frankly, many problems are just based in bad interface designs! Thus, we can’t stop at learning. We truly are more about performance than learning.

In a sense, we’re engineers; applying learning and cognitive science to the design of solutions, (just as chemical engineering is about applying chemistry). Interestingly, the term learning engineering has another definition. This one talks about using the benefits of engineering approaches, such as data, and technology-at-scale, to design solutions. For instance, making adaptive systems requires integrating content management, artificial intelligence, learning design, and more.

Historically, our initial efforts in technology-facilitated learning did take teams. The technology wasn’t advanced enough, and it took learning designers, software engineers, interface designers and more to generate solutions like Plato, intelligent tutoring systems, and the like. I’ve argued that Web 1.0 took the integration of the tech, content design, and more, which usually was more than one person could handle. Now, we’ve created powerful tools that allow anyone to create content. Which may be a problem! The teams used to ensure quality. Hopefully, the shift back comes with a focus on process.

We can apply cognitive science to our own design processes. We’ve evolved many tools to support not making reliable mistakes: design processes, tools like checklists, etc. I’ll suggest that moving to tools that make it easy to produce content haven’t been scaffolded with support to do the right thing. (In fact, good design makes it hard to do bad things, but our authoring tools have been almost the opposite!) There’s some hope that the additional complexity will focus us back on quality instead of being a tool for quantity. I’m not completely optimistic in the short term, but eventually we may find that tools that let us focus on knowledge aren’t the answer.

I’m thinking we will start looking at how we can use tools to help us do good design. You know the old engineering mantra: good, fast, and cheap, pick 2. Well, I am always on about ‘good’. How do we make that an ongoing factor? Can we put in constraints so it’s hard to do bad design? Hmm… An interesting premise that I’ve just now resurrected for myself. (One more reason to blog!) What’re your thoughts?

Getting concrete about mental models

27 February 2024 by Clark 1 Comment

I’ve been on the stump for mental models since, well, I left for grad school. My work colleagues, as a going away present, gave me a copy of Gentner & Stevens’ Mental Models. My Ph.D. thesis also incorporated them. (In it, I looked at how people used such models, and then looked at the effects of training interventions. TL;DR: there were some benefits!) However, I suspect that the idea is a bit complex, and I tend to err on the side of the conceptual (mea culpa!). So, I thought it was time to be getting concrete about mental models.

For background, mental models are simplified explanations about how some portion of the world works. They’re causal conceptual models that connect the relationships and how effects propagate through systems. Done well, they give us a basis to explain and predict outcomes. For performance, they give us a basis to make decisions; we can run them and see what the various outcomes of choices of actions would be. Then we can choose the best one. And they guide decisions in many different circumstances.

For instance, for something procedural, they tell us why we doing things in a certain order. For instance, when I’m cooking breakfast, I want the onions nicely browned. I know not to put them in before the pan heats, because my mental model of heat transference and vegetable properties lets me know that throwing them into a hot pan will brown them. Instead, if I throw them into a cold pan and raise the heat, they’re likely to just get soft. (There’re nuances around this, of course.) And this can stretch to let me know when to throw in various things, and also why to cook the meat, take it out, and then throw in the veggies. If I forget a step, my mental models can fill in and help me regenerate the necessary step.

Similarly for other decisions. In trouble-shooting, for instance, my knowledge of what makes cars work lets me know that if it’s not running, it’s likely one of two sources. Given that petrol engines require an air/fuel mixture ignited by a spark, I can suspect that either the fuel or electrical system is at fault. Then I can take steps to test each. (Ok, at least I could, back in the days of caburetors and distributors. Electronic ignition and fuel injection have thrown off my game!) Knowing the causal properties let me break down the possible contributors. It’s not going to be the brakes!

And so on. In fact, models guide many types of decisions. Good models, based on empirical research, such as cognitive load theory, give us reasons to do things like precede practice with examples. (Bad models, of course, like learning styles, lead us to waste time and money chasing unobtainable benefits.)

Research on mental models tells us several things. For one, our brains are always building models. It also tells us that once we have a model, we’re reluctant to replace it (no “please sir, may I have another model”), and instead, patch it. Thus, if we want to support the optimal performance, we should provide good models, and build their relevance through examples that show them, and addressing them in feedback.

I hope that the two examples here make the use of models a bit more comprehensible. The goal, of course, is that you start paying more attention to them in your learning designs. They’re not necessarily obvious to elicit, but they are there. Do try to find them, represent them explicitly, and refer to them in examples and feedback. Oh, and do avoid the bad ones (you can look to the research translators for guidance). They’re part of what makes us best at performing optimally, and so should be part of your learning solutions. (In other words, build and use your model of mental models! :)

« Previous Page