Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Hyping the news

31 January 2023 by Clark Leave a Comment

I just saw another of these ‘n things you must…if you…’ headlines, and as usual it had the opposite effect they intended. I guess I’m a contrarian, because such headlines to me are an immediate warning. It happened to be in an area I know about, and I hadn’t done any of the necessary things. Yet, I have done the thing they were saying needed the prerequisites. Arguably well (do awards count?). It made me reflect on how we’re hyping the news. Some thoughts…

Yes, I know that such headlines are clickbait. ‘n‘ should be small. Yet when I tried to boil down Upside’s ‘deeper learning’ list for an infographic, it came to 14 items.  Inconvenient for hype,  I’m afraid, but what I’d put in the white paper. Of course there’s more, but I’m trying to be comprehensive, not ‘attractive’.  Similarly, when I created my EEA alignment, I had nine elements. Not because they were convenient for marketing, but because that’s what emerged from the work.

I similarly see lists for ‘the five things’, or the ‘8 things’ (somehow 8 seems to be a maximum, at least for marketing ;). What worries me about these lists is if they’re comprehensive. Is that really all? Have you ensured that they’re necessary and sufficient? Did you even have a process? It took four of us working through months to come up with the eight elements of the Serious eLearning Manifesto.  None of the above lists (Manifesto, EEA, deeper learning) are definitive, but they are the result of substantial work and thinking. Not just pulled together for a marketing push.

There are good lists, don’t get me wrong. Ones where people have worked to try to identify critical elements, or good choices based upon principled grounds. Typically, if it’s the case, there are pointers to the basis for these claims. Either there’s someone who’s known for work in the area, or they’re transparent about process. However, there are also lists where it’s clear someone’s just pulled together some random bits. Look for inconsistency, mismatches of types, etc.

In the broader picture, it’s clear that generating fear and outrage and sensationalism sell. I just want to demonstrate a resistance, and prefer a clear argument over a rant. (Here I’m trying to do the former, not the latter. ;) This goes with probably my broader prescription: I do want policy wonks making decisions. I really don’t want simple wrong answers to complicated questions no matter how appealing.

So, my short take is if you know the area, read with a critical eye. If you don’t, look for warning signs, and see what those who do know have to say about it. Caveat emptor. That’s my take on trying to stay immune to the hyping of news.

Learners as learning evaluators

24 January 2023 by Clark 7 Comments

Many years ago, I led the learning design of an online course on speaking to the media. It was way ahead of the times in a business sense; people weren’t paying for online learning. Still, there were some clever design factors in it. I’ve lifted one to new purposes, but also have a thought about how it could be improved. So here are some thoughts on learners as learning evaluators.

The challenge is the result of two conflicting challenges. For one, we want to support free answers on the part of learners. This is for situations where there’s more than one way to respond. For example a code solution, or a proposed social response. The other is the desire for auto-marking, that is independent asynchronous learning. While it’s ideal to have an instructor in the loop to provide feedback, the asynchronous part means that’s hard to arrange. We could try to have an intelligent programmed response (c.f. artificial intelligence), but those can be difficult to develop and costly. Is there another solution?

One alternative, occasionally seen, is to have the learner evaluate their response. There are positive benefits to this, as it gets learners to become self-evaluators. One of the mechanisms to support this is to provide a model answer to compare to the learners’ own response. We did this in that long-ago project, where learners could speak their response to a question, then listen to theirs and a model response.

There are some constraints on doing this; learners have to be able to see (or hear) their response in conjunction with the model response. I’ve seen circumstances where learners respond to complex questions and get the answer, but they don’t have a basis to compare. That is, they don’t get to see their own response, and the response was complex enough not to be completely remembered. One particular instance of this is in multiple response choices where you pick a collection out.

I want to go further, however. I don’t assume that learners will be able to effectively compare their response to the model response. At least, initially. As they gain expertise, they should, but early on they may not have the requisite support. You can annotate the model answer with the underlying thinking, but there’s another option.

I’m considering the value of having an extra rubric that states what you should notice about the model answer and prompts you to see if you have all the elements. I’m suggesting that this extra support, while it might add some cognitive load to the process, also reduces the load by supporting attention to the important aspects. Also, this is scaffolding that can be gradually removed, allowing learners to internalize the thinking.

I think we can have learners as learning evaluators, if we support the process appropriately. We shouldn’t assume that ability, at least initially, but we can support it. I’m not aware of research on this, though I certainly don’t doubt it. If you do know of some, please do point me to it! If you don’t, please conduct it! :D Seriously, I welcome your thoughts, comments, issues, etc.

Debating debates

17 January 2023 by Clark Leave a Comment

This is the year, at the LDA, of unpacking thinking (the broader view of my previous ‘exposure‘). The idea is to find ways to dig a bit into the underlying rationale for decisions, to show the issues and choices that underly design decisions. How to do that? Last year we had the You Oughta Know series of interviews with folks who represent some important ideas. This year we’re trying something new, using debates to show tradeoffs. Is this a good idea? Here’s the case, debating debates.

First, showing underlying thinking is helpful. For one, you can look at Alan Schoenfeld’s work on showing his thinking as portrayed in Collins & Brown’s Cognitive Apprenticeship. Similarly, the benefits are clear in the worked examples research of John Sweller. While it’s fine to see the results, if you’re trying to internalize the thinking, having it made explicit is helpful.

Debates are a tried and tested approach to issues. They require folks to explore both sides. Even if there’s already a reconciliation, I feel, it’s worth it to have the debate to unpack the thinking behind the positions. Then, the resolution comes from an informed position.

Moreover, they can be fun! As I recalled here, in an earlier debate, we agreed to that end. Similarly, in some of the debates I had with Will Thalheimer (e.g. here), we deliberately were a bit over-the-top in our discussions. The intent is to continue to pursue the fun as well as exposing thinking. It is part of the brand, after all ;).

As always, we can end up being wrong. However, we believe it’s better to err on the side of principled steps. We’ll find out. So that’s the result of debating debates. What positions would you put up?

Don’t make me learn!

10 January 2023 by Clark 1 Comment

In a conversation with a client, the book Don’t Make Me Think was mentioned. Though I haven’t read it, I’m aware of its topic: usability. The underlying premise also is familiar: make interfaces that use pre-existing knowledge and satisficing solutions. (NB: I used to teach interface design, having studied under one of the gurus.) However, in the context of the conversation, it made me also ponder a related topic: “don’t make me learn”. Which, of course, prompted some reflection.

There are times, I’ll posit, when we don’t want employees to be learning. There are times when learning doesn’t make sense. For instance, if the performance opportunities are infrequent, it may not make sense to try to have it in people’s heads. If there’s a resource people can use to solve the problem rather, than learning, that is probably a better answer. That is, in almost any instance, if the information can be in the world, perhaps it should.

One reason for this is learning, done properly, is hard. If a solution must be ‘in the head’ – available when needed and transferring to appropriate situations – there’ll likely be a fair bit of practice required. If it’s complex, much more so. Van Merriënboer’s Four Component Instructional Design is necessarily rigorous! Thus, we shouldn’t be training unless it absolutely, positively, has to be in the head when needed (such as in life-threatening situations such as aviation and medicine).

I’m gently pushing the idea that we should avoid learning as much as possible! Make the situation solvable in some other way. When people talk about ‘workflow learning’, they say that if it takes you out of the workflow, it’s not workflow. I’ll suggest that if it doesn’t, it’s not learning. Ok, so I’m being a bit provocative, but too often we err on the side of throwing training at it, even when it’s not the best solution. Let’s aim for the reverse, finding other solutions first. Turn to job aids or community (learning can be facilitated around either, as well), but stop developing learning as a default.

So, don’t make me learn, unless I have to. Fair enough?

Looking ahead

3 January 2023 by Clark Leave a Comment

A number of people are indicating that 2022 is another year to move on from. And, of course, we do need to move on (as if there were an alternative ;). Still, 2022 was a good year for Quinnovation, and here’s hoping that continues.  Here’re some random thoughts looking ahead.

For one, I saw an interesting piece leveraging the financial adage (really: caution) that “past performance is not indicative of future results”. That comes with various investment opportunities; just because they’ve done well in the past doesn’t meant that will continue. The nice twist in the article was to apply it to yourself: if the past year wasn’t a great one, that doesn’t mean you’re going to continue to suffer. Things can get better despite what happened in the past (or worse), though of course taking your own proactive steps is recommended. Indeed, given that for me, 2020 and 2021 were slow years didn’t mean 2022 had to be. Fortunately!

In the broader sense, I think that despite some hiccups, we’re seeing positive trends. For instance, I increasingly see calls for greater attention to evidence-based practices. While that doesn’t mean it’s happening yet, but the notice is hopefully precedes implementation!

We’ve still some legacies slowing us down, of course. I do think that the belief in us as formal reasoning beings will continue to be a barrier. Still, the above clarion call should help us move (however slowly) to right that wrong.

I’m optimistic, by nature (despite being skeptical). Thus, I think we are working our way forward. I reckon I’ll keep working on that, at least. I am continuing with the Learning Development Accelerator, and Upside Learning, as well of course continuing to do Quinnovative things. I’m looking ahead to us having an impact, together!

Happy Holidays and the New Year!

27 December 2022 by Clark Leave a Comment

This year, my traditional Tuesday post means this is the last post of this year. The next will be in 2023! Which means it’s time for reflection, heartfelt thanks, and so on. So here’s some thoughts and wishes for happy holidays and the new year.

First, it’s been a really good year, overall. After two too-quiet years (2020 and 2021), the year has been joyously busy. Almost too, but that beats the alternative! I’ve been fortunate to be working not only with great clients, but also with Matt Richter and team for the Learning Development Accelerator (LDA), and with Amit Garg and the Upside Learning team. Both have been very fulfilling.

I’ve been serving as the co-director of the LDA, and as such helped drive a few of the initiatives. For one, the You Oughta Know weekly webinar series was a blessing! I got to interview some of my heroes in learning such as John Sweller and Rich Mayer, as well as many eminent friends. We also ran the Learning & Development Conference in a new format this year. I think it went well. We’re moving on to new ideas for this coming year (stay tuned).

Serving as Upside Learning’s Chief Learning Strategist has also been a great experience. These are folks who’ve made a welcome serious commitment to learning science. I’m helping them find the balance between rigor and commercial viability. I’ve always recognized the need to strike a pragmatic balance between principle and practicality. Thus, it’s truly ‘hard fun’ to help figure it out. More mischief is afoot (so again, stay tuned).

I’ve had the chance to realize a couple of things. For one, I’ve been fortunate to have the bandwidth to do things like publish books (my most recent also came out this year). I likely wouldn’t have had that if I had a full-time job. It was an enormous source of stress (and not a few bad decisions) to not have the security of such work, particularly when the kids were young and I was the sole bread-winner. Yet, things have turned out for the best.

Another realization is that I love working with folks to find the balance between what theory would suggest and what fits in practice. I like working through these exercises, because I  learn, and I think this is where I add unique value.  I also like sharing the underlying thinking, because I think we need more of it and it’s hard to scale as an individual contributor. I’m grateful I’ve had the chance for the books and to speak at various venues around the world. Also this blog!

So, thanks to my clients, my partners, and all those who strive to pay attention to what research says and do the right thing. I wish you all the best for happy holidays and the new year. May we continue to learn and grow. Stay curious, my friends!

Meta-reflections

20 December 2022 by Clark Leave a Comment

Lake reflectionI was recently pinged about a new virtual world, a ‘metaverse‘ inspired new place for L&D. It looked like a lot of previous efforts! I admit I was underwhelmed, and I think sharing why might be worthwhile. So here are some meta-reflections.

I’ve written before on virtual worlds. In short, I think that when you need to be social and 3D, they make sense. At other times, there’s a lot of overhead for them to be useful that can be met in other ways. Further, to me, the metaverse really is just another virtual world. Your mileage may vary, of course.

This new virtual world had, like many others, the means to navigate in 3D, and to put information around. The demo they had was a virtual museum. Which, I presume, is a nice alternative to trying to get to a particular location. On the other hand, if it’s all digital, is this the best way to do it? Why navigate around in 3D? Why not treat it as an infographic, and work in 2D, leading people through the story? What did 3D add? Not much, that I could see.

My take has, and continues to be, as they say, “horses for courses”. That is, use the right tool for the job. I complained about watching a powerpoint presentation in Second Life (rightly so). Sure, I get that we tend to use new technologies in old ways first until we get on top of the new capabilities. However, I also argue that we can short-circuit this process if we look at core affordances.

The followup message was that this was the future of L&D, and we’d get away from slide decks and Zoom calls, and do it all in this virtual world. I deeply desire this not to be true! My take is that slide decks, Zoom, virtual worlds, and more all have a place. It’s a further instance of get the design right first, then figure out how to implement it. I want an ecosystem of resources.

Sure, I get that such a meta verse could be an integrating environment. However, do you really want to do all your work in a virtual world? Some things you can’t, I reckon, machining materials, for instance. Moreover, we have benefits from being out in the world. There are other issues as well. You might be better able to deal with diversity, etc, in a virtual world, but it’ll disadvantage some folks. Better, maybe, to address the structural problems rather than try to cover them over?

As always, my takeaway is use technology to implement better approaches, don’t meld your approaches to your tech. Those are, at least, my meta-reflections. What are yours?

Information to Miniscenarios

13 December 2022 by Clark 2 Comments

I’ve talked in the past about miniscenarios. By this, I mean rewriting multiple choice questions (MCQs) to actually be situations requiring decisions and choices thereto. I evangelize this, regularly. I’ve also talked about what you need from subject matter experts (SMEs). What I haven’t really done is talk about how how you map information to miniscenarios. So it’s time to remedy that.

So, first, let’s talk about the structure of a mini-scenario. I’ve suggested that it’s an initial context or story, in which a situation precipitates the need for a decision. There’s the right one, and then alternatives. Not random or silly ones, but ones that represent ways in which learners reliably go wrong. There’s also feedback, which is best as story-based consequences first, then actual conceptual feedback.

So what’s the mapping? One of the things we (should) get from SMEs are the contexts in which these decisions come to play.  Thus, the setting for the mini-scenario is one of these contexts. It may be made fantastic in story, but the necessary contextual elements have to exist. (“Pat had been recently promoted to line supervisor…”)

Then, we have the decisions the learners need to be able to make. These often come in the form of performance objectives. This forms the basis for choosing a situation that precipitates the decision, and the decision itself. (“The errors in manufacturing were higher than the production agreement stipulated. Pat:”) Also, at least, the correct answer. (* worked backward through the process.)

The wrong answers come from some other information we need from SMEs: misconceptions. These are the ways that individuals go wrong when performing. I’ve advocated before that you may want different types of SMEs. It may be that supervisors of the performers have more insight here than content experts. Regardless, you want to make these alternatives available as possible responses. You’ll want to address the difficultly of discrimination between alternatives as a way to manipulate the challenge of the task; it should be appropriate to the learners’ level. (*asked team members what they thought the problem was; *exhorted the team to pay more attention to quality).

The feedback starts with the consequences, which you should also get from SMEs. What happens when you get it right? What happens with each wrong answer? These may come from stories about wins and losses that you also want to collect. (“Pat’s team did not like the implicit claim that they weren’t working hard enough.”)

Finally, there’re the models that are the basis for good performance, and consequently also the basis for the feedback. These you should also collect, because you use them to explain why a choice is good or bad. You don’t want to just say right or wrong, learners need to understand the underlying reason to reinforce their understanding. (Which may also mean they also need to see their answer with the feedback, so they remember what they chose.) Importantly, they need specific feedback for each wrong answer, btw, so your implementation tool needs to support that!  (When investigating errors, don’t start with the team. We always look at the process first, as system flaws need to be eliminated first.)

Pretty much everything you need from SMEs plays a role in providing practice. Miniscenarios aren’t necessarily the best practice, but they’re typically available in your authoring environment.  Writing them isn’t necessarily as easy as generating typical recognition questions, but they more closely mimic the actual task, and therefore lead to better transfer. Plus, you’ll get better as you practice. So  know the mapping of information to miniscenarios, practice your miniscenario writing, and put it into play!

(Social) Media Moves

6 December 2022 by Clark 1 Comment

Time change, and so must we. Recent changes in the social media market mean that I’ve had to adapt. Somewhat. So here’s an update on my social media moves.

Hopefully, you’re aware of the changes that have affected Twitter. After acquisition, the new owner made a number of moves both internally and to the business model. Most of them seem counter to what I advocate: aligning with how people think, work, and learn. I’ve been on Twitter for 14+ years, and was one of the first team recruited to run #lrnchat. I have first met people who are now friends through Twitter. It’s been great. ’til now.

Now, I’m not leaving it, yet. I don’t want to be forced to make decisions by others. Still, while the decision to cut the Covid misinformation containment bothered me, the one to cut the child abuse team has forced my hand. I am no longer posting on Twitter. I’ve stopped auto-posting these blog screeds.

Of course, auto-posting is problematic. I can’t seem to get it working for LinkedIn. IFTTT works for Facebook, and for Twitter, but…not LinkedIn (testing again), and the plugin I was using doesn’t seem to any longer either. Not that I expect you to see me on Facebook (that’s just for friends & family, sorry), but I do engage a lot on LinkedIn. That’s my most active arena right now.

I’m experimenting with alternatives to the little bird. I’ve set up an account on Mastodon (sfba.social) as many have, and likewise just got in to post.news. On both, as on LinkedIn and Twitter, I’m @quinnovator (surprise, eh?).  Hey, I want to make it easy to connect! If you’re on one of those, please do. Besides, I’ve got a brand to maintain, right?  Not sure how to autopost to either, though.

I have yet to really get a handle on Mastodon and post.news, so I’m still experimenting. No insights yet (update: I can’t find myself on Post.news, nor can I see/edit my own profile). I’ll probably maintain the same criteria on them that I’ve used on LinkedIn: I’ll connect to most anyone in the L&D field. We’ll see. So those are my current social media moves. I welcome feedback.

Exposing myself

29 November 2022 by Clark Leave a Comment

This blog is where I “learn out loud”. I try to show my thinking. Yet, I realize I haven’t been doing that as much as I should, or perhaps not as effectively and mindfully as I should  So here’re some reflections on sharing my thinking, or ‘exposing myself’.

Worked examples, as we know from John Sweller, are most effective when they come before practice. Similarly, in Cognitive Apprenticeship, instructors model the desired performance. And I do try to share my thinking. However, I may not have been doing it properly. It needs to unpack the expertise; it can’t be just the knowledge, but what the thinking behind it is. The expertise that experts don’t have access to!

For instance, in my workshops, I often show the outcome of what I (with a team) did, after I have learners do it. Other times, like in the recent L&D conference, I shared my deeper thinking on analyzing technology through affordance, but then didn’t challenge learners to apply that themselves (to be fair, I only had an hour; on the other hand, I didn’t properly prepare).

Increasingly, I think one of the ways to scale my impact is to share my thinking, in context. When people are practitioners, that may be enough, but at some times it may also help to provide ‘challenges’ and provide feedback as well. That, however, doesn’t scale as well. I can only review so many projects…

I guess my biggest take away is to be more conscious about making the underlying thinking clear. I think it’s helpful to learn out loud, but also to ensure that I’m showing the thinking. So, I’ll keep exposing myself…or, at least, my thinking. I’ll try to do it more explicitly and clearly, as well.

« Previous Page
Next Page »

Clark Quinn

The Company

Search

Feedblitz (email) signup

Never miss a post
Your email address:*
Please wait...
Please enter all required fields Click to hide
Correct invalid entries Click to hide

Pages

  • About Learnlets and Quinnovation

The Serious eLearning Manifesto

Manifesto badge

Categories

  • design
  • games
  • meta-learning
  • mindmap
  • mobile
  • social
  • strategy
  • technology
  • Uncategorized
  • virtual worlds

Blogroll

  • Charles Jennings
  • Christy Tucker
  • Connie Malamed
  • Dave's Whiteboard
  • Donald Clark's Plan B
  • Donald Taylor
  • Harold Jarche
  • Julie Dirksen
  • Kevin Thorn
  • Mark Britz
  • Mirjam Neelen & Paul Kirschner
  • Stephen Downes' Half an Hour

License

Previous Posts

  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2008
  • September 2008
  • August 2008
  • July 2008
  • June 2008
  • May 2008
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • May 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006

Amazon Affiliate

Required to announce that, as an Amazon Associate, I earn from qualifying purchases. Mostly book links. Full disclosure.