Quinnovating

Uniqueness

17 December 2024 by Clark Leave a Comment

In a conversation yesterday, we were talking about what works in presenting yourself (in this case, for a job). I mentioned that in the US you have to perhaps overpromise, whereas my experience in Oz (coloured, as it is, by its Brit origins ;), was that you underpromise. The latter worked well for me, because I believe I tend to err on the side of quiet; I don’t like boasts. I was suggesting, in this case, that you needed what made one unique to a particular situation. Thinking further, I think I do value what is uniqueness. What do I mean?

So, to get a (proper?) Ph.D., you are expected make a unique contribution to understanding. Consider our knowledge as a giant ball, and what a thesis does is push out one tiny bump. The goal is something no one else has done. For instance, for my Ph.D., I broke analogy up into a different set of steps, and measured performance. My specification of steps was unique, but that wasn’t the contribution (in my mind, at least). What I also did was try training to improve those processes (four of the six, for reasons), and it did impact a couple, with good reasons not to have impacted the others. It wasn’t earth-shattering, by any means (I suspect no one cites my thesis!), but it was a contribution. (And, of course, it grounded me in the literature and practices.)

When I think of folks I respect, in many cases it’s because they have made a unique contribution. By the way, I suppose I should be clear: unique isn’t enough, it has to be a positive contribution (which can include ruling out things). It’s like innovation: not just an idea, but a good one! So, for instance, Will Thalheimer’s been a proponent of evidence-informed practices, but his unique contribution is LTEM. So too with Patti Shank and multiple choice questions, Michael Allen with SAM, Harold Jarche with PKM, etc. I’m kind of thinking right now that Julie Dirksen’s new book is what’s really new! I am inclined to think that new syntheses are also valuable.

For instance, my own books on myths and learning science are really syntheses, not new ideas. (Maybe my mobile books too?) Reflecting, I think that the three books that I wanted to publish, my first on games, my fourth on L&D strategy, and my most recent on engagement (channeling the core from the first book), are more unique contributions. Though I will self-servingly and possibly wrongly suggest my way of thinking about contexts, models, and more are innovations. Like Allen’s CCAF (Context – Challenge – Activity – Feedback), perhaps.

Which isn’t to say syntheses that organize things into new and more comprehensible ways isn’t also a contribution. In addition to (immodestly) my afore-mentioned books in that category, I think of folks like Connie Malamed, Christy Tucker, Matthew Richter, Ruth Clark, Jane Bozarth, etc. These folks do a great job of taking received wisdom and collating and organizing it so as to be comprehensible. And I could be providing too short a shrift in some cases.

My stance is that I don’t see enough ‘uniqueness’. Original ideas are few and far between. Which may be expected, but we have to be careful. There are a lot more touted ideas than there are good ones. What really is different? What’s worth paying attention to? It’s not an easy question, and I may be too harsh. There is a role for providing different perspectives on existing things, to increase the likelihood that people hear of it. But those should be new perspectives. I’m not interested in hearing the same ideas from different folks. So, does this make sense, or am I being too harsh?

By the way, I suspect that there are more ideas than we actually hear about. I know people can be hesitant about sharing them for a variety of reasons. If you’ve got an idea, share it with someone! If they get excited, it may well be new and worthwhile. Take a chance, we may all benefit.

The enemy of the good

10 December 2024 by Clark Leave a Comment

We frequently hear that ‘perfection is the enemy of the good’. And that may well be true. However, I want to suggest that there’s another enemy that plagues us as learning experience designers. We may be trying to do good, but there are barriers. These are worthy of explicit discussion.

You also hear about the holy trinity of engineering: cheap, fast, or good; pick two. We have real world pressures that want us to do things efficiently. For instance, we have lots of claims that generative AI will allow us to generate more learning faster. Thus, we can do more with less. Which isn’t a bad thing…if what we produce is good enough. If we’re doing good, I’ll suggest, then we can worry about fast and cheap. But doing bad faster and cheaper isn’t a good thing! Which brings us to the second issue.

What is our definition of ‘good’? It appears that, too often, good is if people ‘like’ it. Which isn’t a bad thing, it’s even the first level in the Kirkpatrick-Katzell model: asking what people think of the experience. One small problem: the correlation between what people think of an experience, and it’s actual impact, is .09 (Salas, et al, 2012). That’s zero with a rounding error! What it means is that people’s evaluation of what they think of it, and the actual impact, isn’t correlated at all. It could be highly rated and not be effective, or highly rated and be effective. Etc. At core, you can’t tell by the rating.

What should be ‘good’? The general intent of a learning intervention (or any intervention, really) is to have an impact! If we’re providing learning, it should yield a new ability to ‘do’. There are a multitude of problems here. For one, we don’t evaluate performance, so how would we know if our intervention is having an impact? Have learners acquired new abilities that are persisting in the workplace and leading to the necessary organizational change? Who knows? For another, folks don’t have realistic expectations about what it takes to have an impact. We’ve devolved to a state where if we build it, it must be good. Which isn’t a sound basis for determining outcomes.

There is, of course, a perfectly good reason to evaluate people’s affective experience of the learning. If we’re designing experiences, having it be ‘hard fun’ means we’ve optimized the engagement. This is fine, but only after, we’ve established efficacy. If we’re not having a learning impact in terms of new abilities to perform, what people think about it isn’t of use.

Look, I’d prefer us to be in the situation where perfection to be the enemy of the good! That’d mean we’re actually doing good. Yet, in our industry, too often we don’t have any idea whether we are or not. We’re not measuring ‘good’, so we’re not designing for it. If we measured impact first, then experience, we could get overly focused on perfection. That’d be a good problem to have, I reckon. Right now, however, we’re only focused on fast and cheap. We won’t get ‘good’ until we insist upon it from and for ourselves. So, let’s shall we?

Convincing stakeholders

3 December 2024 by Clark Leave a Comment

As could be expected (in retrospect ;), a recurrent theme in the discussions from our recent Learning Science Conference was how to deal with objections. For instance, folks who believe myths, or don’t understand learning. Of course, we don’t measure, amongst other things. However, we also have mistaken expectations about our endeavors. That’s worth addressing. So, here I’m talking about convincing stakeholders.

To be clear, I’m not talking about myths. Already addressed that. But is there something to be taken away? I suggested (and practiced in my book on myths in our industry), that we need to treat people with respect. I suggest that we need to:

Acknowledge the appeal
Also address what could be the downsides
Then, look to the research
Finally, and importantly, provide an alternative

The open question is whether this also applies to talking learning.

In general, when talking about trying to convince folks about why we need to shift our expectations about learning, I suggest that we need to be prepared with a suite of stories. I recognize that different approaches will work in different circumstances. So, I’ve suggested we should have to hand:

The theory
The data/research
A personal illustrative anecdote
Solicit and use one of their personal anecdotes
A case study
A case study of what competitors are doing

Then, we use the one we think works best with this stakeholder in this situation.

Can we put these together? I think we can, and perhaps should. We can acknowledge the appeal of the current approach. E.g., it’s not costing too much, and we have faith it’s working. We should also reveal the potential flaws if we don’t remedy the situation: we’re not actually moving any particular needle. Then we can examine the situation: here we draw upon one of the second list about approaches. Finally, we offer an alternative: that if we do good learning design, we can actually influence the organization in positive ways!

This, I suggest, is how we might approach convincing stakeholders. And, let me strongly urge, we need to! Currently there are far too many who believe that learning is the outcome of an event. That is, if we send people off to a training event, they’ll come back with new skills. Yet, learning science (and data, when we bother) tells us this isn’t what happens. People may like it, but there’s no persistent change. Instead, learning requires a plan and a journey that develops learners over time. We know how to do good learning design, we just have to do it. Further, we have to have the resources and understanding to do so. We can work on the former, but we should work on the latter, too.

Across Contexts

26 November 2024 by Clark Leave a Comment

(Have I talked about looking across contexts for learning before? I looked and couldn’t find it. Though I’m pretty good about sharing diagrams?!? So, here it is; if again, please bear with me).

In our recent learning science conference, one topic that came up was about contexts. That is, I suggest the contexts we see across examples and practice define the space of transfer. We know that contextual performance is better than abstract (c.f. Bransford’s work at Vanderbilt with the Cognitive Technology Group). The natural question is how to choose contexts. The answer, I suggest, is ad hoc: choose the minimal set of contexts that spans the space of transfer. What we’re talking about is looking for a set chosen across contexts that support the best learning.

So, in talks I’ve used the diagram to say that if you choose the set of contexts represented by the ‘o’s, you’ll be unlikely to transfer to A, whereas if you choose the ‘x’s, you’re much more likely. Let me make that concrete: let’s talk negotiation (something we’re all likely to experience). If all your contexts are about vendors (e.g. ‘o’s,) you may not apply the principles to negotiating with a customer, A. If, however, you have contexts negotiating with vendors, customers, maybe even employers (‘x’s), you’re more likely to transfer to other situations. (Though your employer might not like it! ;)

The point that was asked was how to choose the set. You can be algorithmic about it. If you could measure all dimensions of transfer, and ensure you’re progressing from simple to complex along those, you’d be doing the scientific best. It might lead you to choose too many, however. It may be that you can choose a suite based upon a more heuristic approach to coverage. Here I mean picking ones that provide some substantive coverage based upon expertise (say, from your SME or supervisors of performance). I suspect that you’ll have to make your best first guess and then test to see if you’re getting appropriate transfer, regardless.

It’s important to ensure that the set is minimal. You don’t want too many contexts to make the experience onerous. So pick a set that spans the space, but also is slim. The right set will illuminating the ways in which things can vary without being too large. Another criteria is to have interesting contexts. You are, I’ll suggest, free to exaggerate them a little to make them interesting if they’re not inherently so.

You may also need some times when the context says not to use the focus here. What I mean is that while it could seem appropriate to extend whatever’s being learned to this situation, you shouldn’t. Some ideas support over-generalization, and you’ll need to help people learn where those limits are.

Note that the contexts are those across both examples and practice. So, learners will see some contexts in examples, then others in practice. It may be (if it’s complex, or infrequent, or costly) that you need to have lots of practice, and this isn’t a worry. Still, making sure you’re covering the right swatch across contexts will support achieving the impact in all appropriate situations.

I’m less aware of research on the spread of contexts for transfer (PhD topic, anyone?), and welcome pointers. Still, cognitive theory suggests that this all makes sense. It does to me, how about you?

Beyond Learning Science?

19 November 2024 by Clark Leave a Comment

The good news is, the Learning Science Conference has gone well. The content we (the Learning Development Accelerator, aka LDA) hosted from our stellar faculty was a win. We’ve had lively discussions in the forum. And the face to face sessions were great! The conference continues, as the content will be there (including recordings of the live sessions). The open question is: what next? My short answer is going beyond learning science.

So, the conference was about what’s known in learning science. We had topics about the foundations, limitations, media, myths, informal/social, desirable difficulty, applications, and assessment/evaluation. What, however, comes next? Where do you go from a foundation in learning science?

My answer is to figure out what it means! There are lots of practices in L&D that are grounded in learning science, but go from there to application. My initial list looks like this:

Instructional design. Knowing the science is good, but how do you put it into a process?
Modalities. When you’re doing formal learning, you can still do it face to face, virtually, online, or blended. What are the tradeoffs, and when does each make sense?
Performance consulting. We know there are things where formal learning doesn’t make sense. We want gaps and root causes to determine the right intervention.
Performance support. If you determine job aids are the answer, how do you design, develop, and evaluate them? How do they interact with formal learning?
Innovation. This could (and should; editorial soapbox) be an area for L&D to contribute. What’s involved?
Diversity. While this is tied to innovation, it’s a worthy topic on its own. And I don’t just mean compliance.
Technology. There are lots of technologies, what are their learning affordances? XR, AI, the list goes on.
Ecosystem. How do you put the approaches together into a coherent solution for performance? If you don’t have an ‘all singing, all dancing’ solution, what’s the alternative?
Strategy. There’s a pretty clear vision of where you want to be. Then, there’s where you are now. How do you get from here to there?

I’m not saying this is the curriculum for a followup, I’m saying these are my first thoughts. This is what I think follows beyond learning science. There are obviously other ways we could and should go. These are my ideas, and I don’t assume they’re right. What do you think should be the followon? (Hint: this is likely what next year’s conference will be about. ;)

Taking a higher perspective

12 November 2024 by Clark Leave a Comment

A number of years ago, I did some consulting to a training organization. The issue was that they didn’t seem to have a sustained relationship with their folks. And, this has seemed to me like an obvious and solvable problem. However, I may be missing something, so perhaps you can help me in taking a higher perspective.

In the particular instance, they provided training in particular areas. That is, folks would attend their courses and then, at least theoretically, be able to perform in new ways. Yet, they felt that folks didn’t necessarily sustain allegiance to them nor their offerings.

I asked what else they offered. From the perspective of a performer, I’m not there to learn! Instead, I’m there to acquire new skills so I can perform better. And, if we take to heart what performance consulting has to say, there’re also resources such as job aids. These lead to success where learning isn’t even necessary. There’s more, too.

We can go further, of course. What about community? If you’re focused on a particular area of performance, would it make sense to be connected to others in the same endeavor? I’ll suggest that it’s likely. As folks develop in ability, they need to start interacting with others.

This organization wasn’t alone, of course. I’ve engaged with a number of organizations over the years that faced the same issue. (Whether they knew it or not.) In fact, I suspect it’s more prevalent that we agree. Particularly in this era of information available online, how do you generate a sustained relationship?

It seems to me that if we’re taking a higher perspective, we’ll realize that courses are just a component of a full development ecosystem. Of course, there are lots of issues involved: finding ways to curate or create all the elements, content management, platform choice and integration, and more. Still, this seems to me to be at least part of the answer. So, what am I missing?

Doing the right thing

4 November 2024 by Clark 2 Comments

I have made commitments, and I almost always honor them. The few times I haven’t have been due to circumstances beyond my control, and those who’ve been affected have understood. This time, for a change, it’s been harder to make the right decision. What goes into doing the right thing?

I dropped out of Learning 24 (in Dec) when the source of the commitment abandoned their support. I hadn’t intended to go, but they asked. Then owing to a management change, they reneged. And the conference understood.

I was committed to DevLearn, however, despite the same change in support circumstances. Not only did I have a session accepted, but I was going to do a book signing too. Then I agreed to host a panel, and then to co-facilitate a workshop. I was excited and eager. It is my favorite face-to-face conference.

So, no worries. Until…I got Covid. The first time. I’ve had the vaccinations, and wore a mask on my travels. But this was bad: fever, sore throat, more. My voice sounds more like a croaking frog than human voice. And my voice is bad enough as it is!

Now, the CDC says that 24 hours after the fever’s passed, you’re ok as long as you take precautions: masking, distancing. However, I can’t really facilitate or moderate a panel without talking to folks. I can’t really be around folks without a mask when I’m eating. I’m going to a crowded place that doesn’t want you to stay in your room (no water boiler, microwave, or refrigerator in the room).

Not an easy decision. I really wanted to meet my commitments to the Guild, I love the event, and I could use the exposure (see management changes, above ;). On the other hand, I wouldn’t want anyone else to suffer like this hit me, and traveling where I’m around lots of folks just isn’t smart. I’m risking lots of other folks’ health. That’s not a good choice.

It might also preclude me from getting better, what with travel and being active. I’m being quite sessile, and isolating from my family. Not fun, but it’s right for them and me.

In the long run, I decided to not attend. It’s a sacrifice for me and the Guild, but at this time it’s the right thing to do. I may find out tomorrow or midweek that I could’ve been ok or at least attended the later things, but hindsight’s 20:20, as they say. At this point when I need to make decisions on travel and accommodation, the proper thing to do is to not expose the rest of you to this.

I’m not happy, but I am convinced I’m doing the right thing. And that’s better than the alternative. I won’t see you in Las Vegas, but you’ll survive, and most likely better than if I did. Safe travels, and if you’re there, have a great conference! At least I should be online the week after for the Learning Science conference, Stay curious, my friends.

What L&D resources do we use?

29 October 2024 by Clark 1 Comment

This isn’t a rhetorical question. I truly do want to hear your thoughts on the necessary resources needed to successfully execute our L&D responsibilities. Note that by resources in this particular case, I’m not talking: courses, e.g. skill development, nor community. I’m specifically asking about the information resources, such as overviews, and in particular tools, we use to do our job. So I’m asking: what L&D resources do we need?

I’m not going to ask this cold, of course. I’ve thought about it a bit myself, creating an initial framework (click on the image to see it larger). Ironically, considering my stance, it’s based around ADDIE. That’s because I believe the elements are right, just that it’s not a good basis for a design process. However, I do think we may need different tools for the stages of analysis, design, development, implementation, and evaluation, even if don’t invoke them in a waterfall process. I also have categories for overarching strategy, and for specific learning topics. These are spaces in which resources can reside.

There are also several different types of resources I’ve created categories for. One is an overview of the particular spaces I indicate above. Another are for information resources, that drill into a particular approach or more. These can be in any format: text or video typically. Because I’m weird for diagrams, I have them separately, but they’d likely be a type of info resource. Importantly, one is tools. Here I’m thinking performance support tools we use: templates, checklists, decision trees, lookup tables. These are the things I’m a bit focused on.

Of course, this is for evidence-based practices. There are plenty of extant frameworks that are convenient, and cited, but not well-grounded. I am looking for those tools you use to accomplish meaningful solutions to real problems that you trust. I’m looking for the ones you use. The ones that provide support for excellent execution. In addition to the things listed above, how about processes? Frameworks? Models? What enables you to be successful?

Obviously, but importantly, this isn”t done! That is, I put my first best thoughts out there, but I know that there’s much more. More will come to me (already has, I’ve already revised the diagram a couple of times), but I’m hoping more will come from you too. That includes the types of resources, spaces, as well as particular instances.

The goal is to think about the resources we have and use. I welcome you putting in, via comments on the blog or wherever you see this post, and let me know which ones you find to be essential to successful execution. I’d really like to know what L&D resources do we use. Please take a minute or two and weigh in with your top and essential tools. Thanks!

A busy few weeks

22 October 2024 by Clark 1 Comment

Things always seem to come in fits and spurts. It may be relatively quiet (that is, lots to do but can schedule as suits) and then boom. What’s coming up are a busy a few weeks, and I thought I’d share. Because, of course, some may be relevant to you.

Next week isn’t. Relevant to you, that is. I’ll be off for a couple of days guiding a client strategy. I was just supposed to do a keynote, but…when I heard it was a strategy session I offered to help facilitate it. That said, I do think we’ve created a good plan. Fingers crossed.

The week after that is DevLearn, arguably my favorite F2F L&D conference. I’ll be speaking at 3PM on Thursday, 7 November on achieving impact with your interventions. Then I’ll be signing books at 9:30 AM on Friday the 8th near the conference bookstore. I’m coming in for the full thing, arriving Tuesday and leaving Saturday, but it won’t be my usual visit. I’ll be around, saying hi to old friends and meeting new, of course. I’ll also be introducing a colleague new to L&D around.

Then, and this is exciting, I’ll be spending the subsequent week (11-15 November) either participating in or presenting in sessions for our Learning Science conference. I’m doing a couple (informal/social learning, and making learning ‘stick’) of our curated sessions on my own. Then I’m doing one on myths with my LDA co-director, Matt Richter. The rest of the conference, as mentioned is great folks and important topics. Content’s up front, and no conflicting sessions when we discuss the topics live.

I’ll have a week after that to recover, and then of course Thanksgiving week. I hope to see you live around LV, or online the subsequent week. I’ll try to keep posting here once a week, but things may be a wee bit more random what with a busy few weeks until mid-November. By December, somewhat back to normal except of course the holidays. In the meantime, as I say to my family: be good, stay safe, and have fun!

Learning Science Conference 2024

15 October 2024 by Clark Leave a Comment

I believe, quite strongly, that the most important foundation anyone in L&D can have is understanding how learning really works. If you’re going to intervene to improve people’s ability to perform, you ought to know how learning actually happens! Which is why we’ve created the Learning Science Conference 2024.

We have some of the most respected translators of learning science research to practice. Presenters are Ruth Clark, Paul Kirschner, Will Thalheimer, Patti Shank, Nidhi Sachdeva, as well as Matt Richter and myself. They’ll be providing a curated curriculum of sessions. These are admittedly some of our advisors to the Learning Development Accelerator, but that’s because they’ve reliably demonstrated the ability to do the research, and then to communicate the results of theirs and others’ work in terms of the implications for practice. They know what’s right and real, and make that clear.

The conference is a hybrid model; we present the necessary concepts asynchronously, starting later this month. Then from 11- 15 November, we’ll have live online sessions led by the presenters. These are at two different times to accommodate as much of the globe as we can! In these live sessions we’ll discuss the implications and workshop issues raised by attendees. We will record the sessions in case you can’t make it. I’ll note, however, that participating is a chance to get your particular questions answered! Of course, we’ll have discussion forums too.

We’ve worked hard to make this the most valuable grounding you can get, as we’ve deliberately chosen the topics that we think everyone needs to comprehend. I suggest there’s something there for everyone, regardless of level. We’re covering the research and implications around the foundations of learning, practices for design and evaluation, issues of emotion and motivation, barriers and myths, even informal and social learning. It’s the content you need to do right by your stakeholders.

Our intent is that you’ll leave equipped to be the evidence-based L&D practitioner our industry needs. I hope you’ll take advantage of this opportunity, and hope to see you at the Learning Science Conference 2024.