Quinnovating

Hyping the news

31 January 2023 by Clark Leave a Comment

I just saw another of these ‘n things you must…if you…’ headlines, and as usual it had the opposite effect they intended. I guess I’m a contrarian, because such headlines to me are an immediate warning. It happened to be in an area I know about, and I hadn’t done any of the necessary things. Yet, I have done the thing they were saying needed the prerequisites. Arguably well (do awards count?). It made me reflect on how we’re hyping the news. Some thoughts…

Yes, I know that such headlines are clickbait. ‘n‘ should be small. Yet when I tried to boil down Upside’s ‘deeper learning’ list for an infographic, it came to 14 items. Inconvenient for hype, I’m afraid, but what I’d put in the white paper. Of course there’s more, but I’m trying to be comprehensive, not ‘attractive’. Similarly, when I created my EEA alignment, I had nine elements. Not because they were convenient for marketing, but because that’s what emerged from the work.

I similarly see lists for ‘the five things’, or the ‘8 things’ (somehow 8 seems to be a maximum, at least for marketing ;). What worries me about these lists is if they’re comprehensive. Is that really all? Have you ensured that they’re necessary and sufficient? Did you even have a process? It took four of us working through months to come up with the eight elements of the Serious eLearning Manifesto. None of the above lists (Manifesto, EEA, deeper learning) are definitive, but they are the result of substantial work and thinking. Not just pulled together for a marketing push.

There are good lists, don’t get me wrong. Ones where people have worked to try to identify critical elements, or good choices based upon principled grounds. Typically, if it’s the case, there are pointers to the basis for these claims. Either there’s someone who’s known for work in the area, or they’re transparent about process. However, there are also lists where it’s clear someone’s just pulled together some random bits. Look for inconsistency, mismatches of types, etc.

In the broader picture, it’s clear that generating fear and outrage and sensationalism sell. I just want to demonstrate a resistance, and prefer a clear argument over a rant. (Here I’m trying to do the former, not the latter. ;) This goes with probably my broader prescription: I do want policy wonks making decisions. I really don’t want simple wrong answers to complicated questions no matter how appealing.

So, my short take is if you know the area, read with a critical eye. If you don’t, look for warning signs, and see what those who do know have to say about it. Caveat emptor. That’s my take on trying to stay immune to the hyping of news.

Learners as learning evaluators

24 January 2023 by Clark 7 Comments

Many years ago, I led the learning design of an online course on speaking to the media. It was way ahead of the times in a business sense; people weren’t paying for online learning. Still, there were some clever design factors in it. I’ve lifted one to new purposes, but also have a thought about how it could be improved. So here are some thoughts on learners as learning evaluators.

The challenge is the result of two conflicting challenges. For one, we want to support free answers on the part of learners. This is for situations where there’s more than one way to respond. For example a code solution, or a proposed social response. The other is the desire for auto-marking, that is independent asynchronous learning. While it’s ideal to have an instructor in the loop to provide feedback, the asynchronous part means that’s hard to arrange. We could try to have an intelligent programmed response (c.f. artificial intelligence), but those can be difficult to develop and costly. Is there another solution?

One alternative, occasionally seen, is to have the learner evaluate their response. There are positive benefits to this, as it gets learners to become self-evaluators. One of the mechanisms to support this is to provide a model answer to compare to the learners’ own response. We did this in that long-ago project, where learners could speak their response to a question, then listen to theirs and a model response.

There are some constraints on doing this; learners have to be able to see (or hear) their response in conjunction with the model response. I’ve seen circumstances where learners respond to complex questions and get the answer, but they don’t have a basis to compare. That is, they don’t get to see their own response, and the response was complex enough not to be completely remembered. One particular instance of this is in multiple response choices where you pick a collection out.

I want to go further, however. I don’t assume that learners will be able to effectively compare their response to the model response. At least, initially. As they gain expertise, they should, but early on they may not have the requisite support. You can annotate the model answer with the underlying thinking, but there’s another option.

I’m considering the value of having an extra rubric that states what you should notice about the model answer and prompts you to see if you have all the elements. I’m suggesting that this extra support, while it might add some cognitive load to the process, also reduces the load by supporting attention to the important aspects. Also, this is scaffolding that can be gradually removed, allowing learners to internalize the thinking.

I think we can have learners as learning evaluators, if we support the process appropriately. We shouldn’t assume that ability, at least initially, but we can support it. I’m not aware of research on this, though I certainly don’t doubt it. If you do know of some, please do point me to it! If you don’t, please conduct it! :D Seriously, I welcome your thoughts, comments, issues, etc.

Don’t make me learn!

10 January 2023 by Clark 1 Comment

In a conversation with a client, the book Don’t Make Me Think was mentioned. Though I haven’t read it, I’m aware of its topic: usability. The underlying premise also is familiar: make interfaces that use pre-existing knowledge and satisficing solutions. (NB: I used to teach interface design, having studied under one of the gurus.) However, in the context of the conversation, it made me also ponder a related topic: “don’t make me learn”. Which, of course, prompted some reflection.

There are times, I’ll posit, when we don’t want employees to be learning. There are times when learning doesn’t make sense. For instance, if the performance opportunities are infrequent, it may not make sense to try to have it in people’s heads. If there’s a resource people can use to solve the problem rather, than learning, that is probably a better answer. That is, in almost any instance, if the information can be in the world, perhaps it should.

One reason for this is learning, done properly, is hard. If a solution must be ‘in the head’ – available when needed and transferring to appropriate situations – there’ll likely be a fair bit of practice required. If it’s complex, much more so. Van Merriënboer’s Four Component Instructional Design is necessarily rigorous! Thus, we shouldn’t be training unless it absolutely, positively, has to be in the head when needed (such as in life-threatening situations such as aviation and medicine).

I’m gently pushing the idea that we should avoid learning as much as possible! Make the situation solvable in some other way. When people talk about ‘workflow learning’, they say that if it takes you out of the workflow, it’s not workflow. I’ll suggest that if it doesn’t, it’s not learning. Ok, so I’m being a bit provocative, but too often we err on the side of throwing training at it, even when it’s not the best solution. Let’s aim for the reverse, finding other solutions first. Turn to job aids or community (learning can be facilitated around either, as well), but stop developing learning as a default.

So, don’t make me learn, unless I have to. Fair enough?

Looking ahead

3 January 2023 by Clark Leave a Comment

A number of people are indicating that 2022 is another year to move on from. And, of course, we do need to move on (as if there were an alternative ;). Still, 2022 was a good year for Quinnovation, and here’s hoping that continues. Here’re some random thoughts looking ahead.

For one, I saw an interesting piece leveraging the financial adage (really: caution) that “past performance is not indicative of future results”. That comes with various investment opportunities; just because they’ve done well in the past doesn’t meant that will continue. The nice twist in the article was to apply it to yourself: if the past year wasn’t a great one, that doesn’t mean you’re going to continue to suffer. Things can get better despite what happened in the past (or worse), though of course taking your own proactive steps is recommended. Indeed, given that for me, 2020 and 2021 were slow years didn’t mean 2022 had to be. Fortunately!

In the broader sense, I think that despite some hiccups, we’re seeing positive trends. For instance, I increasingly see calls for greater attention to evidence-based practices. While that doesn’t mean it’s happening yet, but the notice is hopefully precedes implementation!

We’ve still some legacies slowing us down, of course. I do think that the belief in us as formal reasoning beings will continue to be a barrier. Still, the above clarion call should help us move (however slowly) to right that wrong.

I’m optimistic, by nature (despite being skeptical). Thus, I think we are working our way forward. I reckon I’ll keep working on that, at least. I am continuing with the Learning Development Accelerator, and Upside Learning, as well of course continuing to do Quinnovative things. I’m looking ahead to us having an impact, together!

Happy Holidays and the New Year!

27 December 2022 by Clark Leave a Comment

This year, my traditional Tuesday post means this is the last post of this year. The next will be in 2023! Which means it’s time for reflection, heartfelt thanks, and so on. So here’s some thoughts and wishes for happy holidays and the new year.

First, it’s been a really good year, overall. After two too-quiet years (2020 and 2021), the year has been joyously busy. Almost too, but that beats the alternative! I’ve been fortunate to be working not only with great clients, but also with Matt Richter and team for the Learning Development Accelerator (LDA), and with Amit Garg and the Upside Learning team. Both have been very fulfilling.

I’ve been serving as the co-director of the LDA, and as such helped drive a few of the initiatives. For one, the You Oughta Know weekly webinar series was a blessing! I got to interview some of my heroes in learning such as John Sweller and Rich Mayer, as well as many eminent friends. We also ran the Learning & Development Conference in a new format this year. I think it went well. We’re moving on to new ideas for this coming year (stay tuned).

Serving as Upside Learning’s Chief Learning Strategist has also been a great experience. These are folks who’ve made a welcome serious commitment to learning science. I’m helping them find the balance between rigor and commercial viability. I’ve always recognized the need to strike a pragmatic balance between principle and practicality. Thus, it’s truly ‘hard fun’ to help figure it out. More mischief is afoot (so again, stay tuned).

I’ve had the chance to realize a couple of things. For one, I’ve been fortunate to have the bandwidth to do things like publish books (my most recent also came out this year). I likely wouldn’t have had that if I had a full-time job. It was an enormous source of stress (and not a few bad decisions) to not have the security of such work, particularly when the kids were young and I was the sole bread-winner. Yet, things have turned out for the best.

Another realization is that I love working with folks to find the balance between what theory would suggest and what fits in practice. I like working through these exercises, because I learn, and I think this is where I add unique value. I also like sharing the underlying thinking, because I think we need more of it and it’s hard to scale as an individual contributor. I’m grateful I’ve had the chance for the books and to speak at various venues around the world. Also this blog!

So, thanks to my clients, my partners, and all those who strive to pay attention to what research says and do the right thing. I wish you all the best for happy holidays and the new year. May we continue to learn and grow. Stay curious, my friends!

Meta-reflections

20 December 2022 by Clark Leave a Comment

I was recently pinged about a new virtual world, a ‘metaverse‘ inspired new place for L&D. It looked like a lot of previous efforts! I admit I was underwhelmed, and I think sharing why might be worthwhile. So here are some meta-reflections.

I’ve written before on virtual worlds. In short, I think that when you need to be social and 3D, they make sense. At other times, there’s a lot of overhead for them to be useful that can be met in other ways. Further, to me, the metaverse really is just another virtual world. Your mileage may vary, of course.

This new virtual world had, like many others, the means to navigate in 3D, and to put information around. The demo they had was a virtual museum. Which, I presume, is a nice alternative to trying to get to a particular location. On the other hand, if it’s all digital, is this the best way to do it? Why navigate around in 3D? Why not treat it as an infographic, and work in 2D, leading people through the story? What did 3D add? Not much, that I could see.

My take has, and continues to be, as they say, “horses for courses”. That is, use the right tool for the job. I complained about watching a powerpoint presentation in Second Life (rightly so). Sure, I get that we tend to use new technologies in old ways first until we get on top of the new capabilities. However, I also argue that we can short-circuit this process if we look at core affordances.

The followup message was that this was the future of L&D, and we’d get away from slide decks and Zoom calls, and do it all in this virtual world. I deeply desire this not to be true! My take is that slide decks, Zoom, virtual worlds, and more all have a place. It’s a further instance of get the design right first, then figure out how to implement it. I want an ecosystem of resources.

Sure, I get that such a meta verse could be an integrating environment. However, do you really want to do all your work in a virtual world? Some things you can’t, I reckon, machining materials, for instance. Moreover, we have benefits from being out in the world. There are other issues as well. You might be better able to deal with diversity, etc, in a virtual world, but it’ll disadvantage some folks. Better, maybe, to address the structural problems rather than try to cover them over?

As always, my takeaway is use technology to implement better approaches, don’t meld your approaches to your tech. Those are, at least, my meta-reflections. What are yours?

Information to Miniscenarios

13 December 2022 by Clark 2 Comments

I’ve talked in the past about miniscenarios. By this, I mean rewriting multiple choice questions (MCQs) to actually be situations requiring decisions and choices thereto. I evangelize this, regularly. I’ve also talked about what you need from subject matter experts (SMEs). What I haven’t really done is talk about how how you map information to miniscenarios. So it’s time to remedy that.

So, first, let’s talk about the structure of a mini-scenario. I’ve suggested that it’s an initial context or story, in which a situation precipitates the need for a decision. There’s the right one, and then alternatives. Not random or silly ones, but ones that represent ways in which learners reliably go wrong. There’s also feedback, which is best as story-based consequences first, then actual conceptual feedback.

So what’s the mapping? One of the things we (should) get from SMEs are the contexts in which these decisions come to play. Thus, the setting for the mini-scenario is one of these contexts. It may be made fantastic in story, but the necessary contextual elements have to exist. (“Pat had been recently promoted to line supervisor…”)

Then, we have the decisions the learners need to be able to make. These often come in the form of performance objectives. This forms the basis for choosing a situation that precipitates the decision, and the decision itself. (“The errors in manufacturing were higher than the production agreement stipulated. Pat:”) Also, at least, the correct answer. (* worked backward through the process.)

The wrong answers come from some other information we need from SMEs: misconceptions. These are the ways that individuals go wrong when performing. I’ve advocated before that you may want different types of SMEs. It may be that supervisors of the performers have more insight here than content experts. Regardless, you want to make these alternatives available as possible responses. You’ll want to address the difficultly of discrimination between alternatives as a way to manipulate the challenge of the task; it should be appropriate to the learners’ level. (*asked team members what they thought the problem was; *exhorted the team to pay more attention to quality).

The feedback starts with the consequences, which you should also get from SMEs. What happens when you get it right? What happens with each wrong answer? These may come from stories about wins and losses that you also want to collect. (“Pat’s team did not like the implicit claim that they weren’t working hard enough.”)

Finally, there’re the models that are the basis for good performance, and consequently also the basis for the feedback. These you should also collect, because you use them to explain why a choice is good or bad. You don’t want to just say right or wrong, learners need to understand the underlying reason to reinforce their understanding. (Which may also mean they also need to see their answer with the feedback, so they remember what they chose.) Importantly, they need specific feedback for each wrong answer, btw, so your implementation tool needs to support that! (When investigating errors, don’t start with the team. We always look at the process first, as system flaws need to be eliminated first.)

Pretty much everything you need from SMEs plays a role in providing practice. Miniscenarios aren’t necessarily the best practice, but they’re typically available in your authoring environment. Writing them isn’t necessarily as easy as generating typical recognition questions, but they more closely mimic the actual task, and therefore lead to better transfer. Plus, you’ll get better as you practice. So know the mapping of information to miniscenarios, practice your miniscenario writing, and put it into play!

(Social) Media Moves

6 December 2022 by Clark 1 Comment

Time change, and so must we. Recent changes in the social media market mean that I’ve had to adapt. Somewhat. So here’s an update on my social media moves.

Hopefully, you’re aware of the changes that have affected Twitter. After acquisition, the new owner made a number of moves both internally and to the business model. Most of them seem counter to what I advocate: aligning with how people think, work, and learn. I’ve been on Twitter for 14+ years, and was one of the first team recruited to run #lrnchat. I have first met people who are now friends through Twitter. It’s been great. ’til now.

Now, I’m not leaving it, yet. I don’t want to be forced to make decisions by others. Still, while the decision to cut the Covid misinformation containment bothered me, the one to cut the child abuse team has forced my hand. I am no longer posting on Twitter. I’ve stopped auto-posting these blog screeds.

Of course, auto-posting is problematic. I can’t seem to get it working for LinkedIn. IFTTT works for Facebook, and for Twitter, but…not LinkedIn (testing again), and the plugin I was using doesn’t seem to any longer either. Not that I expect you to see me on Facebook (that’s just for friends & family, sorry), but I do engage a lot on LinkedIn. That’s my most active arena right now.

I’m experimenting with alternatives to the little bird. I’ve set up an account on Mastodon (sfba.social) as many have, and likewise just got in to post.news. On both, as on LinkedIn and Twitter, I’m @quinnovator (surprise, eh?). Hey, I want to make it easy to connect! If you’re on one of those, please do. Besides, I’ve got a brand to maintain, right? Not sure how to autopost to either, though.

I have yet to really get a handle on Mastodon and post.news, so I’m still experimenting. No insights yet (update: I can’t find myself on Post.news, nor can I see/edit my own profile). I’ll probably maintain the same criteria on them that I’ve used on LinkedIn: I’ll connect to most anyone in the L&D field. We’ll see. So those are my current social media moves. I welcome feedback.

Exposing myself

29 November 2022 by Clark Leave a Comment

This blog is where I “learn out loud”. I try to show my thinking. Yet, I realize I haven’t been doing that as much as I should, or perhaps not as effectively and mindfully as I should So here’re some reflections on sharing my thinking, or ‘exposing myself’.

Worked examples, as we know from John Sweller, are most effective when they come before practice. Similarly, in Cognitive Apprenticeship, instructors model the desired performance. And I do try to share my thinking. However, I may not have been doing it properly. It needs to unpack the expertise; it can’t be just the knowledge, but what the thinking behind it is. The expertise that experts don’t have access to!

For instance, in my workshops, I often show the outcome of what I (with a team) did, after I have learners do it. Other times, like in the recent L&D conference, I shared my deeper thinking on analyzing technology through affordance, but then didn’t challenge learners to apply that themselves (to be fair, I only had an hour; on the other hand, I didn’t properly prepare).

Increasingly, I think one of the ways to scale my impact is to share my thinking, in context. When people are practitioners, that may be enough, but at some times it may also help to provide ‘challenges’ and provide feedback as well. That, however, doesn’t scale as well. I can only review so many projects…

I guess my biggest take away is to be more conscious about making the underlying thinking clear. I think it’s helpful to learn out loud, but also to ensure that I’m showing the thinking. So, I’ll keep exposing myself…or, at least, my thinking. I’ll try to do it more explicitly and clearly, as well.

« Previous Page