Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Where are we at?

by 1 Comment

Signs pointing multiple directions with distances. I was talking with a colleague, and he was opining about where he sees our industry. On the other hand,  had some different, and some similar thoughts. I know there are regular reports on L&D trends, with greater or lesser accuracy. However, he was, and I similarly am looking slightly larger than just “ok, we’re now enthused about generative AI“. Yes, and, what’s that a signal of? What’s the context? Where are we at?

When I’m optimistic, I think I see signs of an awakening awareness. There are more books on learning science, for instance. (That may be more publishers and people looking for exposure, but I remain hopeful.)  I see a higher level of interest in ‘evidence-based’. This is all to the good (if true). That is, we could and should be beginning to look at how and why to use technology to facilitate learning appropriately.

On the cynical side, of course, is other evidence. For example, the interest in generative AI seems to be about ways to reduce costs. That’s not really what we should be looking at. We should be freeing up time to focus on the more important things, instead of just being able to produce more ‘content’ with even less investment. The ‘cargo cult’ enthusiasm about: VR, AR, AI, etc still seems to be about chasing the latest shiny object.

As an aside, I’ll still argue that investing in understanding learning and better design will have a better payoff than any tech without that foundation. No matter what the vendors will tell you!  You can have an impact, though of course you risk having a previous lack of impact exposed…

So, his point was that he thought that more and more leaders of L&D are realizing they need that foundation. I’d welcome this (see optimism, above ;).  Similarly, when I argue that if Pine & Gilmore are right (in The Experience Economy) as to what’s the next step, we should be the ones to drive the Transformation Economy (experiences that transform you).  Still,  is this a reliable move in the field? I still see folks who come in from other areas of the biz to lead learning, but don’t understand it. I’ll also cite the phenomena that when folks come into a new role they need to be seen to be doing something. While them getting their mind around learning would be a good step, I fear that too many see it as just management & leadership, not domain knowledge. Which, reliably, doesn’t work. Ahem.

Explaining the present, let alone predicting the future, is challenging. (“Never predict anything, particularly the future!”) Yet, it would help to sort out whether there is (finally) the necessary awakening. In general, I’ll remain optimistic, and continue to push for learning science, evidence, and more. That’s my take. What’s yours? Where are we at?

Lazy thinking?

by 1 Comment

Now, I’m the last who should throw stones. I can be quite guilty of lazy thinking, particularly when there’re commercial decisions to be made. (Providers have done a fabulous job of making sure you can’t compare apples to apples, and when there’re so many such situations…) Yet, there’s one place where I struggle with the consequences. That’s in our professional field, and it seems like there’re too many opportunities to yield.

A trigger was a recent conversation where an individual was talking about generations. Grouping folks by when they’re born is problematic at best. For one, the boundaries used seem to vary by who’s doing the categorizing. Not a solid basis. Moreover, the research suggests that there really aren’t meaningful differences. What do exist are explainable by age differences (which isn’t the same thing, for one it’s a continuum, not discrete chunks). Really, it’s a mild form of age discrimination, differentiating people by when they’re born, not who they are or how they behave. (Also problematic is the notion that events affect certain segments of the population, but that’s a longer conversation).  It’s one of the myths in my book on same.

Other examples include learning styles, hemispheres, gender differences, and more. First, they’re categorizations on things that people can’t control. Second, they don’t get backing from data. I just read that medical science has been excluding women from research based upon an assumption about temperature variability that was exposed as being irrelevant!

Sure, it’s much easier if we can reliably group people into segments that mean we have a reliable basis to do different things. Marketers do this with psychographics, for instance. Demographics can also matter. The problem here is that we’re using unreliable metrics. First, there are assumptions that turn out to be flawed. They frequently use self-report, also problematic. Some also have a flawed theoretical foundation.

Yes, it’s hard to keep on top of all of this. Ideally, you’d have time to investigate them all. In practice, there are other things to do. We all need ways to simplify our lives. Plus, vendors are telling you that they, at least, are immune to the complaints (with self-interest at stake).  On the other hand, there are good sources of insight from reliable translators of research. There are also practices we can follow to make it manageable. More help is on the way (at the LDA we’re working on it; stay tuned).

While lazy thinking is understandable, it’s not acceptable, at least not in our professional field. While we may not be sued for malpractice, we certainly should be responsible. So let’s avoid taking the easy path, at least when it matters. In our professional capacity, it matters when we’re designing for our learners. Let’s do so on evidence, not assumptions.

The Pivotal Point

by 3 Comments

We (the Learning Development Accelerator) just released Guy Wallace’s latest tome, The L&D Pivot Point. Then, we had an interview with him to explain what it’s about. Despite having a ring-side seat (I served as editor, caveat emptor), it was eye-opening to hear him talk about what it’s about! It really is about the pivotal point in L&D, when you move from just offering courses to looking at performance. It’s such an important point that it’s worth reiterating.

So, the official blurb for the book talks about his tried and tested processes. In the interview, he talks about how he’s synthesized the work of the leaders of the performance improvement movement, people like Joe Harless, Geary Rummler, Thomas Gilbert, Robert Mager, Thiagi, and more. While the models they used differed, Guy’s created a synthesis that makes sense, and more importantly, works. He talked about how he refined his work to balance effectiveness with efficiency. Moreover, his approach avoids any redundant work.

Interestingly, he also recounted how his approach achieved buy-in from the stakeholders to the extent that he had to fight to not keep them all on the team through all the stages! That’s a great outcome, and it comes from demonstrating value. He focuses on where performance needs are critical, and thus it has a natural interest, but too many of the approaches can stifle that interest. Instead, his intent focus on meaningful outcomes truly engages everyone from the performers to the executives.

Guy also is quite open about the problems facing our industry. Despite the necessity of starting as order takers (essentially, “you can’t say ‘no'”), he estimates that only 20% of the time is the problem a learning or skills problem. Which resonates with other data I’ve seen about the value of training interventions! Instead, there can be many drivers for problems in performance.  His approach includes detailed analyses that identify the root cause of the problem, and when to determine that it’s worth trying an intervention. He’s quite open about how that can lead to a shift in intervention focus. At other times, it might lead to a hiatus while problems get attention.

One other thing I found interesting in the interview was how he talked about potential barriers to success up front. While it might seem like a deterrent, he pointed out how it led to making sense later. That is, folks would soon see that, for instance, supervisor support was critical to success. He includes a rigorous analysis of potential barriers as part of the book.

Quite simply, L&D has a problem of going from go-to-whoa without considering whether a course is the right solution. Guy’s book is a way to avoid doing that, and systematically evaluating what the pivotal point should be for determining whether we can successfully intervene or not, and how. There’s much more: how to manage the process, deal with stakeholders, and test your assumptions. It’s in his own inimitable style (lessons learned on editing ;), but there’s deep wisdom there. That’s my take, at least, I welcome yours.

Getting Engagement Right

by Leave a Comment

I’m on record stating that I think learning experience design (LXD) is the elegant integration of learning science and engagement. In addition, I’ve looked at both. Amongst the things that stand out for me are that there are an increasing suite of resources for learning science. For one, I have my own book on the topic! There are other good ones too. However, on the flip side, for engagement, I didn’t find much. I had intended to write an LXD book, but then ATD asked for the learning science one. Once it was done, however, I quickly realized that I wanted to write the complement. Thus, Make It Meaningful was born. It’s available, but I’m also running a workshop on the topic, starting this coming week! Four weekly 2 hour meetings, at the convenient time of noon ET. It’s all about getting engagement right. So, what’s covered?

For the first week, there’s an overview of the importance of engagement, and how to set the ‘hook’. We’ll briefly review the reasons why to consider the engagement side (and trust me, this is something you want to do). Then we’ll talk about the first step, getting learners to the a motivated state to begin the learning. We’ll look at barriers to success as well, and what to do.

In the second week, we’ll talk about ‘landing’ the experience. Once the hook is set, it doesn’t mean you’ve got them through the experience. Instead, there’s much to do to maintain that motivation. In addition, you want to ensure that anxiety doesn’t overwhelm the learning, and you want to build confidence. We’ll talk about principles as well as heuristics.

In the third week, we dig into what this means in practical terms. What is an engaging introduction? What about the models and examples? The critical element is the practice that learners perform. We’ll talk about how aligning the practice with the desired objectives while making a compelling context is necessary, but doable.

In the last week, we’ll talk about making a design process that can reliably deliver on learning experience. We’ll take a generic design process and go through the changes that ensure both an effective learning design and an engaging experience. We’ll work from analysis, through specification, and on to evaluation (we won’t talk much about implementation, because of my quip that getting the design right leaves lots of ways to create the solution, and not doing so renders everything else extraneous.

Sure, you can just buy the book, and that’s ok. I’m all about getting the word out, and getting better learning happening for our learners. However, in the workshop, not only do you get the book, but we’ll work through the ideas systematically, put them into practice, and address the individual questions you may have. Look, getting engagement right is an advanced topic, but it’s increasingly what will differentiate our solutions from the knowledge ones that come from typical approaches, no matter how technologically augmented. This stuff matters! So, I hope to see you there.

A brief AI overview?

by 2 Comments

At the recent and always worthwhile DevLearn conference, I was part of the panel on Artificial Intelligence (AI). Now, I’m not an AI practitioner, but I have been an AI groupie for, well, decades. So I’ve seen a lot of the history, and (probably mistakenly) think I have some perspective. So I figured I’d share my thoughts, giving a brief AI overview.

Just as background, I took an AI course as an undergrad, to start. Given the focus on thinking and tech (two passions), it’s a natural. I regularly met my friend for lunch after college to chat about what was happening. When I went to grad school, while I was with a different advisor, I was in the same lab as David Rumelhart. That happened to be just at the time he was leading his grad students on the work that precipitated the revolution to neural nets. There was a lot of discussion of different ways to represent thinking. I also got to attend an AI retreat, sponsored by MIT, and met folks like John McCarthy, Ed Feigenbaum, Marvin Minsky, Dan Dennet, and more! Then, as a faculty member in computer science, I had a fair affiliation with the AI group. So, some exposure.

So, first, AI is about using computer technology to model intelligence. Usually, human intelligence, as a cognitive science tool, but occasionally just to do smart things in any means possible. Further, I feel reasonably safe to say that there are two major divisions in AI: symbolic and sub-symbolic. The former dominated AI for several decades, and this is where a system does formal reasoning through rules. Such systems do generate productive results (e.g. chatbots, expert systems), but eventually don’t do a good job of reflecting how people really think. (We’re not formal logical reasoners!)

As a consequence, sub-symbolic approaches emerged, that tried architectures to do smart things in new ways. Neural nets end up showing good results. They find use in a couple of different ways. One is to set them loose on some data, and see what they detect. Such systems can detect patterns we don’t, and that’s proven useful (what’s known as unsupervised learning).

The other is to give them a ‘training set’ (also known as supervised learning), a body of data about inputs and decisions. You provide the inputs, and give feedback on the decisions until they make them in the same way.Then they generalize to decisions that they haven’t had training on. It’s also the basis of what’s now called generative AI, programs that are trained on a large body of prose or images, and can generate plausible outputs of same. Which is what we’re now seeing with ChatGPT, DALL-E, etc. Which has proven quite exciting.

There are issues of concern with each. Symbolic systems work well in well-defined realms, but are brittle at the edges. In supervised learning, the legacy databases unfortunately frequently have biases, and thus the resulting systems also have these biases! (For instance, housing loan data have shown bias.) They also don’t understand what they’re saying. So generative AI systems can happily tout learning styles from the corpus of data they’ve ingested, despite scientific evidence to the contrary.

There are issues in intellectual property, when the data sources don’t receive acknowledgement nor recompense.  (For instance, this blog has been used for training a sold product, yet I haven’t received a scintilla of return.) People may lose jobs if they’re currently doing something that AI can replace. While that’s not bad (that is, don’t have people do boring rote stuff), it needs to be done in a way that doesn’t leave those folks destitute. There should be re-skilling support. There are also climate costs from the massive power requirements of such systems. Finally, such systems are being put to use in bad ways (e.g. fakes). It’s not surprising, but we really should develop the guardrails before these tools reach release.

To be fair, there are some great opportunities out there. Generative AI can produce some ideas you might not have thought of. The only problem is that some of them may be bad. Which brings me to my final point. I’m more a fan of Augmenting Intellect (ala Engelbart) than I am of Artificial Intelligence. Such systems can serve as a great thinking partner! That is, they support thinking, but they also need scrutiny. Note that there can be combinations, such as hybrids of unsupervised and supervised, and symbolic with sub-symbolic.
With the right policies, AI can be such a partner. Without same, however, we open the doors to substantial risks. (And, a few days after first drafting this, the US Gov announced an approach!) I think having a brief AI overview provides a basis for thinking usefully about how to use them successfully. We need to be aware to avoid the potential problems. I hope this helps, and welcome your corrections, concerns, and questions.

DnD n LnD

by 2 Comments

multi-sided diceLast Friday, I joined in on a Dungeons & Dragons (DnD) campaign. This wasn’t just gratuitous fun, however, but was explicitly run to connect to Learning & Development folks (LnD). Organized by the Training, Learning, and Development Community (a competitor to LDA? I have bias. ;), there was both some preliminary guidance, and outcomes. I was privileged to play a role, and while not an official part of the followup (happening this week), I thought I’d share my reflections.

So, first, my DnD history. I played a few times while in college, but… I gave it up when a favorite character of mine was killed by an evil trap (that was really too advanced for our party). I’ve played a lot of RPGs since then, with a lot of similarities to the formal DnD games (tho’ the actual ones are too complex). Recently, with guidance from offspring two, our family is getting back into it (with a prompt from a Shakespeare and DnD skit at the local Renaissance Faire).

Then, I’ve been into games for learning since my first job out of college, programming educational computer games. It also became the catalyst for my ongoing exploration of engagement to accompany my interest in cognition/learning, design, and technology. The intersection of which is where I’ve pretty much stayed (in a variety of roles), since then! (And, led to my first book on how to do same.)

Also, about DnD. It’s a game where you create a character. There are lots of details. For one, your characteristics: strength, dexterity, wisdom, intelligence, and more. Those combine with lots of attributes (such race & role). Then, there’s lots of elaboration: backstory, equipment, and more. This can alter during the game, where your abilities also rise. This adds complexity to support ongoing engagement. (I heard one team has been going for over 40 years!)

Characters created by the players are then set loose in a campaign (a setting, precipitating story, and potential details). A Dungeon Master runs the game, Keegan Long-Wheeler in our case, writing it and managing the details. Outcomes happen probabilistically by rolling dice. Computers can play a role. For one, through apps that handle details like rolling the dice. Then folks create games that reflect pre-written campaigns.

One important thing, to me, is that the players organize and make decisions together. We were a group who didn’t necessarily know each other, and we were playing under time constraints. This meant we didn’t have the dialog and choices that might typically emerge in such playing. Yet, we managed a successful engagement in the hour+ we were playing. And had fun!

I was an early advocate of games for learning. To be clear, not the tarted up drill and kill we were mostly doing, but inspired by adventure games. John Carroll had written about this back in the day, I found out. However, I’d already seen adventure games having the potential to be a basis for learning. Adventure games naturally require exploring. In them, you’re putting clues together to choose actions to overcome obstacles. Which, really, is good learning practice! That is, making decisions in context in games is good practice for making decisions in performance situations. Okay, with the caveat that you should design the game so that decisions have a natural embed.

The complexity of DnD is a bit much, in my mind, for LnD, but…the design!  The underlying principles of designing campaigns bears some relation to designing learning experiences. I believe designing engaging learning may be harder than designing learning or games, but we do have good principles. I do believe learning can, and should, be ‘hard fun‘.  Heck, it’s the topic of my most recent tome! (I believe learning should be the elegant integration of learning science with engagement.)

This has been an opportunity to reflect a bit on the underlying structure of games, and what makes them work. That’s always a happy time for me. So, I’m curious what you see about the links between games and learning!

What does it take to leave?

by 4 Comments

I did it, I finally left. I’m not happy about it, but it had to happen. (Actually, it happened some weeks ago.) So, what does it take to leave?

I’m talking about Twitter (oh, yeah, ‘X’ as in what’s been done to it ), by the way. I’d been on there a fair bit. Having tossed my account, I can’t see when my first tweet was, but at least since 2009. How do I know? Because that’s when I was recruited to help start #lrnchat, an early tweetchat that has still been going as recently as this past summer! I became an enthusiast and active participant.

And, let me be clear, it’s sad. I built friendships there with folks with long before I met them in person. And I learned so much, and got so much help. I like to tell the story about when I posted a query about some software, and got a response…from the guy who wrote it! For many years, it was a great resource, both personal and professional!

So, what happened? Make no mistake, it was the takeover by Elon Musk. Twitter went downhill from there, with hiccups but overall steadily. The removal of support, the politics, the stupid approaches to monetization, the bad actors, it all added up. Finally, I couldn’t take it any more. Vote with your feet. (And yes, I’m mindful of Jane Bozarth’s admonition: “worth every cent it cost you”. Yep, it was free, and that was unexpected and perhaps couldn’t be expected to last. However, I tolerated the ads, so there was a biz basis!)

Perhaps it’s like being an ex-smoker, but it riles me to see media still citing X posts in their articles. I want to yell “it’s dead, what you hear are no longer valid opinions”. I get that it’s hard, and lots of folks are still there, but… It had become, and I hear that it continues to be, an increasing swamp of bad information. Not a good source!

So where am I now? There isn’t yet an obvious solution. I’m trying out Mastodon and Bluesky. If you’re there, connect! I find the former to be more intimate. The latter is closer to twitter, but I’m not yet really seeing my ‘tribe’ there. I am posting these to both (I think). I’m finding LinkedIn to be more of an interaction location lately, as well, though it’s also become a bit spammy. #sideeffects? I keep Facebook for personal things, not biz, and I’m not on Instagram. I also won’t go on Threads or TikTok.

So, what does it take to leave? I guess when the focus turns from facilitating enlightening conversation at a reasonable exchange, to monetization and ego. When there’s interference in clean discourse, and opposition to benign facilitation. And, yes, I’m not naive enough to believe in total philanthropy (tho’ it happens), but there are levels that are tolerable and then there’s going to a ridiculous extreme. Wish I had $44B to lose! I know I’m not the only one wishing those who’ve earned riches would focus on libraries and other benevolent activities instead of ego-shots into space, but this is the world we’ve built. Here’s to positive change in alignment with how people really think, work, and learn.

Bad research

by 1 Comment

How do you know what’s dubious research? There are lots of signals, more than I can cover in one post. However, a recent discovery serves as an example to illustrate some useful signals. I was trying to recall a paper I recently read, which suggested that reading is better than video for comprehending issues. Whether that’s true or not isn’t the issue. What is the issue is that in my search, I came across an article that really violated a number of principles. As I am wont to do, let’s briefly talk about bad research.

The title of the article (paraphrasing) was “Research confirms that video is superior to text”. Sure, that could be the case! (Actually the results say, not surprisingly, that one media’s better for some things, and another’s better at other; BTW, one of our great translators of research to practice, Patti Shank, has a series of articles on video that’s worth paying attention to.) Still, this article claimed to have a definitive statement about at least one study. However, when I looked at it, there were several problems.

First, the study was a survey asking instructors what they thought of video. That’s not the same as an experimental study! A good study would choose some appropriate content, and then have equivalent versions in text and video, and then have a comprehension test. (BTW, these experiments have been done.) Asking opinions, even of experts, isn’t quite as good. And these weren’t experts, they were just a collection of instructors. They might have valid opinions, but their expertise wasn’t a basis for deciding.

Worse, the folks conducting the study had. a. video. platform.  Sorry, that’s not an unbiased observer. They have a vested interest in the outcome. What we want is an impartial evaluation. This simply couldn’t be it. Not least, the author was the CEO of the platform.

It got worse. There was also a citation of the unjustified claim that images are processed 60K times better than text, yet the source of that claim hasn’t been found! They also cited learning styles! Citing unjustified claims isn’t a good practice in sound research. (For instance, when reviewing articles, I used to recommend rejecting them if they talked learning styles.) Yes, it wasn’t a research article on it’s own, but…I think misleading folks isn’t justified in any article (unless it’s illustrative and you then correct the situation).

Look, you can find valuable insights in lots of unexpected places, and in lots of unexpected ways. (I talk about ‘business significance’ can be as useful as statistical significance.) However, an author with a vested interest, using an inappropriate method, to make claims that are supported by debunked data, isn’t it. Please, be careful out there!

Get the basics right first!

by Leave a Comment

I’m currently advising several organizations on their approaches to the use of technology to support learning. Moreover, I’ve been doing so for more than two decades, and see a lot more such situations as well. One of the things that I struggle with is seeing folks getting all agog over new technology, yet without getting the design right beforehand.  Thus, let me make a simple suggestion: get the basics right first!

So, we know what leads to good learning. Heck, I’ve written a book summarizing what’s known about it, and I’m not the only one. Despite that fact that humans are complex, and increasingly so are our learning goals, there exist robust principles. We know that we should provide a sufficient quantity of appropriately challenging contextualized practice with aligned feedback, for instance. That is, if we actually want to achieve an outcome.

Yet, too often, we don’t see this. We see, instead, information presentation. Sometimes even with a knowledge test! Yet, such an effort is unlikely to lead to any meaningful change. That is, the investment’s wasted!

Worse, too often we see this being done with fancy new tools. Sure, I get as attracted to shiny new objects as anyone. However, I want to understand their core affordances for learning. Anyone had the dubious pleasure of attending a slide presentation in a virtual world? Or maybe being presented with animated presentations of lots of facts? The new tools may have a short-term effect of novelty, but that’s it. The fundamental aspects of how our brains learn are what’s going to make, or break, a learning investment.

On the other hand, if we start with getting the learning right, first, then there may be additional value coming from the tech. Adaptivity, on top of quality learning design, can accelerate the outcomes.  Immersion, at the right time and place, is better than not. Language models, properly used, can have big impacts. However, it comes from knowing the specific capabilities, and matching them to the need.

While I haven’t done the ‘back of the envelope’ calculation (I’m not a financial whiz), I can state with a fair degree of comfort that you’re better off doing simple learning with good design. Bad design with shiny tech is still bad design! You’ll more likely have an impact putting your investment into learning quality than using fancy tech to deliver dreck. Of course, once you’ve done that, the investment in tech can do a lot more!

I’m not against new tech, heck I’ve written on games, mobile, and more! What I’m against is new tech in lieu of good design. And I’m even more enamored of good tech on top of good design.  So, get the basics right first, then add in the shiny objects. That way you’re going to have a good return on your $$, and that’s a good thing. Right?

PS, speaking of basics, we’ll be running a debate tomorrow (11 Oct) discussing the Learning Experience Design (LXD) label. I’m sure we’ll unpack critical issues. Check it out. 

Talking ‘transfer’

by Leave a Comment

I was looking to wrap up my slide deck for DevLearn (will be talking learning science), and wanted to see if there was anything missing. In particular, I knew I had a fair bit on retention, but I was checking on what I call ‘transfer’. In doing so, I realized that colleagues and others use the term differently! Which means it’s time to talk ‘transfer’.

So one of our good research translators is Will Thalheimer. I found his report (PDF) on transfer, and its usage was as many others I’d found. Will describes transfer as moving learning from the instructional environment to the workplace. No argument this is important! Yet, it’s not how I use the word.

In talking learning science, I’ve seen and said that the two gold standards are retention over time until needed, and transfer to all appropriate (and no inappropriate) situations. (That’s what I’ll be talking about in my session: what leads to those.) My notions here are both part of his transfer.

I get my usage of transfer essentially from folks like John Anderson at Carnegie Mellon Uni, who talks about what facilitates transfer. He’s the one I think of, for instance, when thinking of identical elements transfer, which stipulates that the more elements are the same, the higher the transfer. There’s near and far transfer, as well. That indicates the breadth of area you’re transferring to, really. E.g. negotiation would be far transfer, as there are many situations, whereas training to specifically run projector MX3600D4 is very much near transfer.

In a review of transfer I found (PDF), a distinction is made between ‘low-‘ and ‘high-road’ transfer. The former ‘happens’, while the latter is engineered. Given that, to me, instruction is about engineering success, maximizing likelihoods, I’m all for the latter. So, choosing contexts and supporting reflection.

To memory, even the academic literature about workplace learning seems to support the other interpretation. Yet, my term seems more in line with what the learning science community. In the longer term, we’ll need to reconcile, but for me, the distinction between the elements of retention and transfer are important. So, when I’m talking ‘transfer’, what I mean is the breadth of application. That’s my take, what are your thoughts?