Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Age or experience?

by Leave a Comment

One of the things that has been a recurring theme across things I’ve been looking at lately is experience. Too often we confound age with experience. And, of course, sometimes it’s that we should be talking about it. So, a brief rant on age or experience.

First, I’ll bring up the ‘generations’ myth. It’s appealing, as our brains like buckets for things. We’re kinda wired that way. The only problem is that generations as a concept has been looked at and debunked. Heck, in Ancient Greek days they were complaining that ‘kids just have no respect”! And if you think about it, thinking that someone in Los Angeles CA of a certain age has more in common with someone in Nepal of the same age versus another Angelenõ of a different age is kinda ridiculous.

And, those ‘defining’ events? They affect every conscious person! And it’s so context dependent. A local event may not mean much to you, unless it affects you somehow, and then you share more with everyone else so affected. There’s actually a simpler explanation. Say, for instance, that “young folks want classes while old folks don’t”. That’s explainable by stage of life: when you’re young you need credentials, but later on you can point to your experience.

People share values, and gain motivation by the same underlying factors (differently across culture and personality), and more. Just look at the research on self-determination theory! Attributing to age rather than explaining by experience is a mistake. So, for instance, my kids, who arguably fit the label ‘digital natives’, still come to me (decreasingly, I’ll admit) for tech problems.

Then, there are many things that change as you develop in a domain. For instance, in our Learning Science Conference, my colleague Matt Richter was talking about feedback, and very clearly pointed out how what useful feedback is changes as you gain experience. This holds true for examples, too, the type of useful example changes. Also for practice: with more experience, you need more challenge.

Which, as we further see, is how we go wrong. We do the ‘one size fits all’, not recognizing that things need to change. To be fair, we also do the wrong practice (knowledge test rather than application to problems), give the wrong feedback to begin with (right/wrong), the list goes on. But even when we’re trying to do it right, we forget things like adapting for initial and developing experience. Yet, it’s a factor for instance in how much practice you need, how much spacing, etc.

This problem does go more broadly. We hear it in hiring (age discrimination). Of course, that’s only one problem. For example, gender, race, physical and neurological differences, and more are also present. Sadly. Okay, soapbox: DEI, done right, leads to better outcomes! Actually, that’s got an evidence-base, so probably more than soapbox. Still. So, consider experience as one of the factors distinguishing individuals. Folks can’t control their age, but they can determine their experience. So use it!

Key notes

by Leave a Comment

I’ve seen a lot of keynotes over the years. I’ve even given them! It’s time to reconcile my thoughts. So here are some key notes on keynotes.

One of the things that I’ve seen is flawless performances. Now, one of the things you’re told is that the focus is on you, and slides only should be used as an augment, not on all the time. I confess I’m not good about that (I am not very comfortable in the spotlight; imposter syndrome I suppose). Another is that you should have pauses, and jokes, and such. I do a pretty good dramatic reading (I won the dirty limerick reading contest at work once!), and occasionally even manage to raise a smile or two. The best, however, have their patter completely down. I can’t do that, because my thoughts are continually evolving, but I admire it when it happens. And I’m pretty good at tailoring a talk to the audience (having learned a few times the hard way!).

Inspiration is good, too. Letting people know there’s a way to surpass this barrier works! I try to do that too, though I confess I talk about learning design, not achieving things like climbing Mt Everest (really heard a keynote about that, and it was cool!). And I probably am a bit too conceptual, though I am learning to do better about grounding my principles in practice. However, I do recoil from too much ‘enthusiasm’! Somehow it comes across as artificial. But then, I can be a bit of a curmudgeon (apparently)…

However, what really matters to me is accuracy. I don’t mind if folks are a bit enthusiastic or polished, but I really get wound around the axle a bit when folks state stuff that’s just wrong. For instance, I heard a well-regarded personage opine about games, something I know a wee bit about (my first job, back when dinosaurs strode the earth, was on games, and it’s been a recurrent them in practice, research, and writing for literally decades). And that individual said something just dead wrong. As you may surmise, it really ground my gears. Similarly with learning science, or mobile.  In general, when people have beautifully symmetric ‘n part models’ without grounding, I want to know if those are convenient, or a necessary and sufficient list. (Too often the former.)

I also like when people tout doing things I believe in, but when it’s an area I know about, you better agree with the science, and present it accurately. If you don’t, well, I won’t be quiet about it. (I guess it’s a flaw in my character!) Still, when you say these are the five things to X, and they’re a) not completely separable, b) incomplete, c) wrong, etc, I’m going to be turned off.

Look, I love a good keynote. Many times, they get people who aren’t from the field where the keynote’s presented, and they make connections from adjacent fields. That’s acceptable, even desirable! I like a well-presented talk as well as the next person. I like ideas, and even inspiration. But I will complain about bad information. Always. Those are my key notes on keynotes, what are yours? (And I’m available, if you want L&D advice ahead of the curve but grounded in evidence. Particularly contrary takes… ;)

Beyond LLMs

by Leave a Comment

So, I was recently at the DevLearn conference, and it was, as always, fun. Though, as you might expect, there as non-stop discussion of AI. Of course, even on the panel I was on (with about 130 other Guild Masters; hyperbole, me?) it was termed AI tho’ everyone was talking Large Language Models (LLMs). In preparation, I started thinking about LLMs and their architecture. What I have realized (and argued) is that people are misusing LLMs. What became clear to me, though, is why. And I realize there’s another, and probably better, approach. So let’s talk going beyond LLMs.

As background, you tune LLMs (and the architecture, whether applied to video, audio, or language) to a particular task. Using text/language as an example, their goal is to create good sounding language. And they’ve become very good at it. As has been said, they create what sounds like good answers. (They’re not, as hype has it, revolutionary, just evolutionary, but they appear to be new.)

I made that point on the panel, asking the audience how many thought LLMs made good answers, and there was a reasonable response. Then I asked how many thought it made what sounded like good answers. My point was that they’re not the same. So, they don’t necessarily make good design! (Diane Elkins pointed out that they trained on average, so they create average. If you’re below average, they’re good, if you’re above average, they’ll do worse than what you’d do.) I ranted that tech-enabled bad design is still bad design!

However, I’ve been a fan of predictive coding, as it poses a plausible model of cognition. Then I heard about active inference. And, in a quick search, found out that together, they’re much closer to actual thinking. In particular, combined, they approach artificial general intelligence (AGI for short, and wrongly attributed to our current capability). I admit that I don’t go fully into the math, but conceptually, they build a model of the world (as we do). Moreover, they learn, and keep learning. That is, they’re not training on a set of language statements to learn language, but they’re building explanations of how the world works.

I think that when we want really good systems to know about a domain (say business strategy) and provide good guidance, this is the type of architecture. What I said there, and will say again here, is that this is where we should be applying our efforts. We’re not there yet, and I’m not sure how far the models have evolved. On the other hand, if we were applying the resources going to LLMs… Look, I’m not saying there aren’t roles for LLMs, but too often they’re being used inappropriately I think we can do better when we go beyond LLMs. You heard it here first ;).

Rethinking AI and education

by 1 Comment

I read a post today about Artificial Intelligence and education, and it prompted some thoughts. Including causing me to rethink a post I made earlier! It’s all about using AI to assist with instruction, and it’s probably worth sharing my thinking, to see what you all think. So here are some thoughts on rethinking AI and education.

The post Carl Hendrick wrote talked (at length ;) about how AI is being used to support education. We know that using AI to create answers keeps you from doing the hard work to learn. So, for instance, the trend to use AI by students to write assignments, etc, undermines learning. However, it becomes an ‘us against them’ battle to try to create assignments that require students to think. So far, pretty much the best approach I’ve heard means that you ultimately have to spend some time talking to students one on one. Which doesn’t scale well.

Carl was getting philosophical, about trends and what teaching vs learning means. And it’s an important point: learning is a process, teaching is an intervention, ideally to facilitate that process. If we take Geary’s evolutionary learning model, we pretty much need instruction for certain topics. But it led to me to question the whole premise.

The conversation largely rode on ChatGPT (and, implicitly, other Large Language Models). These models are created to generate plausible language. Not correct answers, note. And, they do it so well that there’s been a revolution in the hype (not the substance, mind you). And what concerns me is that LLMs aren’t able to really ‘know’ anything. In my previous post, I posited that we could perhaps combine ‘agents’ (in some way that’d be secure) to create a tutoring model. But I wonder if that’s the right way.

I am thinking about efforts to generate models that instead of generating plausible language, do ‘knowing’. That is, modeling the predictive coding models of the brain. It might be hard to get them to the right level, but at least they’d understand. If you think back to the Intelligent Tutoring Systems of the past, they built deep models of expertise in the domain. Could systems learn this, instead of interpreting language ‘about’ this? Coupling such a system with a teaching engine (maybe learning what instruction really is) might be a real tutor.

Carl’s point about the nature of teaching is that it’s much more than providing answers. In the experiment he was citing, they carefully built a tutor that they had to tune to do real teaching, fighting its natural predilections of such systems to provide answers that sound like correct ones. That, ultimately, sounds wrong.

(I’m not going into the curriculum and assessment, by the way, I don’t know that what they’re teaching is actually useful in today’s day and age; was it knowledge about physics, or actual ability to use it? There’re robust results that students who learn formal physics still make bad predictions, such as after a semester, still thinking the a ball dropped from a plane still lands directly under where it was dropped!)

My point is that trying to make LLMs be teachers may be using AI in the wrong way. Sure, ITSs don’t scale well, but could we build an engine that learns a domain (rather than handcrafting) and one that learns to teach, and then scale that? It’s not language fluency that matters, it’s pedagogical fluency is my argument. That’s how I’m rethinking AI and education. Your thoughts?

Thinking about motivation

by Leave a Comment

So, I haven’t been a big Self-Determination Theory (SDT) person, simply because I hadn’t really known about it. I learned about it enough to mention in my most recent tome (highlighting the importance of motivation in learning), but that’s about it. However, it’s been popping up more and more (not least with the Motivation Summit the LDA’s putting on; the live sessions will be past, but you can still register to watch the presentations and recordings thereof). And, I am increasingly thinking that there’s some real ‘there’ there, so here’s some thinking about motivation.

SDT posits, quite simply, that there are three consistent elements that contribute to motivation. They are:

  • Competence: the ability to do something, maybe with support, but also development to get better
  • Relatedness: other people who are there believing similarly and supporting you
  • Autonomy: the ability to be who you are

(These are my definitions, by the way, not the official ones, which are no doubt better.) Importantly, they’ve been verified by research across cultures, age ranges, and every other demographic difference. It’s pretty much a human universal. Also importantly, they have practical implications for how you do things.

Quite simply, motivation plays a role in pretty much everything we do! Our motivations include exercising, eating, and sleep; work tasks and job environment; and for us, intent to learn. It’s that latter I was tapping into for designing games, lo those many decades ago. I didn’t at the time know about the SDT framework (in fact, it hadn’t really emerged yet ;), but I was tapping into the elements when I was talking about goals, appropriate challenge, relevance, and more. So when I write about the education/engagement alignment, SDT is a higher-level framework. Simple alignment would have challenge = competence, goals = autonomy, relevance = relatedness. Social learning, too.

Increasingly, as ‘leadership’ becomes a topic, it plays a role as well. I’ve been interested in culture and social for many years as a function for L&D. Leadership is how you create a culture, by how you are socially. How well do you support those elements? I’m not a leadership expert, but I’m increasingly seeing those factors, crossed by types of situations being faced. For instance, in a crisis you have to make a decision, which reduces autonomy for others, but then you should rebuild it. Importantly, I’m finding out that efforts are yielding valuable outcomes, which alone is a reason to pay attention!

So, I’m looking more into it, as an aspect of making a good environment, organizationally and societally. If we’re thinking about motivation, we can start being wise about it. That is, not just for me, and you, but for others. Not just short term, but also long-term. And, explicitly considering our values. Which may be the most important part of it! I think the values that empirically lead to the best for all is a good basis. What say you?

Still the myths

by Leave a Comment

Over on LinkedIn, about the last site worth visiting (I do use Bluesky and Mastodon, but they’re not really ‘sites’ so much as channels), I am still seeing quotes about people believing in misinformation. Learning styles, generations, attentions spans dropping, etc, all these things that aren’t valid are still being touted. Despite our debunking efforts, it’s still the myths!

To be fair, we do seem to be seeing a bit of an ‘anti-science’ movement. Which would be not only silly, but sad! Sure, there are problems with science, but it still beats every other process we have. Anecdotes don’t surpass real evidence, and personal opinion isn’t superior to what proper research tells us.

For one, as Naomi Oreskes makes clear in her book Why Trust Science, what makes science work isn’t just the process. So, yes, scientists conduct experiments, and others review them, and it’s a collective decision to publish them. And, yes, bad papers are still submitted (I used to serve on editorial boards, and my rejection rate was about 95%; but it was a good journal ;). Also, it’s hard to bring in new viewpoints. What Oreskes points out, however, and aptly, is that over time, these processes advance our understanding. We may have fits and spurts, but for the long game we win. For instance, how are you able to read this offering of mine, over miles and minutes? Because science.

So, science denial is counter-productive, but it exists. Gale Sinatra and colleague Barbara Hofer, in their book Science Denial, outline the reasons how this happens. Based upon research into the situation, they document our minds have biases, and how we can be swayed. We can also have our own beliefs, and our tendency to confirmation bias means we only look for evidence that supports our views. Fortunately, they discuss ways to address these problems, but we need to put some of these into place (as with Brian Klaas’ recommendations for fighting corruption).

We have good data that there are things we should avoid. There really aren’t any psychometrically valid instrument for learning styles, and no evidence that should use them if we did. Categorizing people by generations is, basically, a form of stereotyping. Our attentions spans can engage for hours even, as we play games, read novels, watch movies, etc. And so on!

Sometimes, it feels hopeless. But I look and see that we’re getting more attention to learning science. It really is about communication, and it seems we’re (slowly) making headway. So, I’ll keep keeping on (heck, I wrote the book!), despite ‘still the myths’. Hopefully, fewer and less over time. Fingers crossed!

Conference season

by Leave a Comment

Conference season has commenced. Two are already in the books. Three I know about are coming up, and I’m playing a role in two. So, what’s up, and when? Here’s what I know.

So, first, the Learning Development Accelerator (LDA) is running the Creating a Motivating Work Environment Summit. This is in conjunction with the Center for Self-Determination Theory, so it’s scrutable. I’m not part of this, except as a participant. It follows the usual LDA format: access to videos created by the presenters, followed by live sessions at two different times. The videos are already up, and the live sessions are coming soon, Nov 3 – 7!  It’s all online, which makes it easy to attend, and the live sessions will be recorded. There’s a stellar lineup of speakers, naturally! I’m increasingly finding the value in the theory, so I look forward to the session. Caveat: I’m a Co-Director of the LDA, so I have a vested interest in the success, but I still think it’s of interest (at least to me).

Then, the Learning Guild is holding the next DevLearn conference, and I’ll be doing several things. My Wed is pretty full, as I’m starting by hosting a Morning Buzz on building a learning culture. Hosting isn’t the same as presenting, but instead just facilitating the conversation.  Then I’m presenting on the spacing of learning. I’ve been actively engaged in developing a spaced learning strategy, and will be sharing the key principles from learning science research. As well as what’s not (yet) known! I’ll be signing books right after that at the event bookstore. On Thursday, I’m part of a panel on AI (which will probably be interpreted as Generative AI), and will be my usual critical self ;). Of course, I’ll also be wandering the halls and expo. If you’re there, say hello!

Finally, the LDA is also running our second Learning Science Conference. (If you attended last year, you get a big discount!). It uses the same format asa the Motivation Summit, above, that is with specifically curated content in presentations, and live sessions Dec 8 – 12. It starts 3 Nov, and I’m active in this one too. I’ll again be presenting two sections. The first will be on getting information into, and out of, long-term memory, specifically generative and retrieval practice. I talked about the latter, last year, so I’m refining that (my own understanding evolves as does the field), and adding more on generative. Similarly with social and informal learning, which I’ll be presenting again. That is, I’ll be rehashing the old, and adding a bit new.

There’re new speakers, too. We’ve the honor of having Gale Sinatra and Jim Hewitt, and Rich Mayer will be doing a special session with Ruth Clark. Other presenters include my fellow co-Director, Matt Richter, along with Stella Lee and Nidhi Sachdeva. There’ll be special sessions, such as with Will Thalheimer.  Of course, we’ll have a debate, here with me going head to head with authors Bianca Baumann and Mike Taylor on marketing and motivation.  We also will have a panel with greats Julie Dirksen, Jane Bozarth, and Koreen Pagano. And more.

Sure, there’re lots of ways to get on top of learning science and good design. There’s the Serious eLearning Manifesto, books (e.g. my recommended reading list), blogs (like this one), magazines likeTraining, eLearn, journals, and more. However, getting together with your fellow practitioners, live or online, is a real boon, and so conference season is a great opportunity. Hope to see you somewhere soon!

Generative visualized?

by Leave a Comment

Ok, so I shared that I was trying to visualize the generative process. Which is, to be fair, aka elaboration. And, I did take a stab. So I thought I’d share what I’d come up with for generative visualized. Two reasons: 1) for you to quibble with what I came up with, and 2) learning out loud. I reckon if you can see my process, you may learn, you may improve it, all the good things that happen from sharing. So…

First, I was looking to do a higher build than the neural level. Yes, at the neural level we’re strengthening patterns across neural networks. We’re taking two patterns and putting them into conjunction, basically, activating both. That activating in conjunction strengthens the links between them, associating them more. (I’ve a diagram on that, actually, emphasizing that they need time to recover before the next strengthening. If you’re interested, let me know and I’ll share that, too.) But the way we activate patterns is by using words or images, semantically. So, I was looking for words.

They also should be ideas that are familiar enough that I’m not requiring knowledge you don’t know. Further, I need to be making associations that you’d get. So, something familiar. In this case, animals and pets. (Perhaps that’s salient to me, for some reason.) And I needed a relationship between them. I chose, rightly or wrongly, the notion of how wild animals don’t make good pets. So, what’s this look like?

Now, I’m showing the final build, not the whole process. You start with something you’re talking about, like how there are good and bad pets (“do raccoons make good pets?”). And you ask the learners to find some personal or conceptual basis for that distinction. They can find examples, like that good pets are domesticated animals, and bad pets are wild, drawing on their previous knowledge. Or, they might talk about someone who had a bad pet (“my friend’s parents had this pet monkey, but it was always doing bad things”). Of course, pets are a subset of animals, so this becomes confusing at the top level, but that’s okay, as the structures are similar but not exactly the same.

The point is that they’re generating this information, actively connecting their own pre-existing knowledge to the information you’d like them to acquire. It’s causing them to elaborate, and so strengthening the links in their mind. Which is what we want. So, that’s my attempt to capture generative visualized. Does it work, or do you have ideas to improve or replace?

FWIW, this is part of what I’ll be presenting at the LDA’s Learning Science Conference, starting Nov 3 to review the presentations and discuss, with live sessions with the presenters Dec 8-12. Same format as last year (with recognition of that in pricing, for those that attended), but with some new content and presenters. It’s still the things we think you need to be an informed learning designer. 

How to visualize generative?

by Leave a Comment

I’ve been thinking a fair bit about generative learning of late. Not least, because it’s one of the things I’ll be talking about at our LDA Learning Science Conference, coming up (Nov for the background vids and discussion, early Dec for the week of live with presenters). It’s a refresh of the successful event we held last year, but keep what worked, and adding what’s new! And, I’m still wrestling with it, particularly how to represent it!

First, generative learning is the complement to retrieval practice. Research by luminaries such as Robert Bjork, Henry Roedigger, Ericsson, etc, have told us of the value of retrieval practice for memory. Which is all about strengthening our ability to get information out of long-term memory to solve problems. However, we first need to get our models and examples into long-term memory first. I’ve previously termed this elaboration, but generative activities may be a more proactive way of thinking about it.

It’s really about connecting new information to old. We know that’s valuable! And we can present it, but generative activities have learners actively processing information. So, for instance, what Craik & Lockhart talked about as elaborative processing. We can connect it to personal experience, such as asking what this explains that we previously had observed. Or, we can (and must) connect it to previous information, so we can integrate it more tightly into our understanding. So, we can diagram it via a mindmap or otherwise, draw it, write it in different terms, etc. . I suggest that many of Thiagi and Matt Richter’s activities do just this!

Then, as you have probably know, I’m fond of diagrams. And so, I want to find a way to represent the value of generative learning, in ways we can remember and apply. What I’m struggling with is representing the process of elaboration. It could be a sequence of networks, but I’ve been somewhat loath simply because it seems like too much trouble! Which, of course, isn’t a good reason. Another reason (as he reaches for justification), is that networks are a level of abstraction away, and not really cognitive. So maybe I need to show concept relationships being formed?

As you can tell, I’m not really there yet. But, in the spirit of ‘learning out loud’, I thought I’d share where I am, not least to prompt some responses with ideas!  So, what ideas or pointers can you share?

The full story?

by Leave a Comment

As I continue exploring learning, I’m beginning to realize that the picture’s incomplete. Which shouldn’t be a surprise, but it’s a bit of a concern. Of course, I don’t have the answers, but if I point out some of the elements, maybe we’ll identify the gaps. And, I’m sure, there are likely initiatives and results I’m unaware of, and hopefully folks will help fill in the full story.

So, our endeavor is to start with people, with whatever experience and abilities they bring to the table. Then, ultimately (after our ministrations) have them capable of dealing with the situations we’ve identified are important. That includes what needs to be in long-term memory, and having responses aligned to triggers that activate the appropriate actions. Along the way, we have the analysis, the learning science, the coaching, and …?

I talked earlier about how most of our focus has been on getting folks up to some level of initial speed. But we do want folks to ultimately acquire the full range. And, I think, we too often abandon our efforts before getting the full way.

That makes sense for organizational learning, where we have to balance the costs with the value. When lives are on the line (aviation, medicine, military), we probably need to go a very long way. When it’s just inventory, we may allow some checks along the way to catch mistakes, and hope that people will eventually internalize the elements. (There are practices we spend on that are entirely worthless, of course.) For education, when we have kids in schools for years, it is clear that our pedagogy, driven by a wrong curriculum and values, falls far short of actually applying knowledge. But that’s another rant ;).

And we have pretty good guidance for many elements. We know about retrieval practice, with spacing, deliberate choices of next steps, desirable difficulty. We also know about generative activities. I’m not sure we’ve reconciled, however, how much of each. We also have cognitive load theory, which guides us in many ways, but doesn’t necessarily talk about acquisition versus application. The power law of learning suggests that we go from conscious to unconscious, but it assumes doing the right practice. The list goes on: multimedia learning, four component ID, cognitive apprenticeship, …; we have lots of pieces.

And yet, that’s still getting people up to a certain beginning level. How do we go beyond? The aforementioned is all about formal learning, and not about moving on. So we kind of have a hiccup between learning and coaching. How do they segue? We also have the issue of assessment. I have suggested that maybe we need to consider our commitment to apply as well as our ability. And that’s still before actually starting, yet we’re not done then.

Folks are talking about dispositions, I note, but when I look at it, it’s about the broader picture of persistence, etc, not specific to the particular outcome. Happy to be wrong here. But we really want to understand what we do initially, for instruction, for reactivation and continuation, to the final picture. My intuition is that there’s a more systematic structure that we’re not applying.

I don’t have an answer. Certainly not yet! I just think it’s time that we start thinking about it. Well, I guess that’s an open question: do we need an overarching approach? If so, what do we have. Can we extend something like Cognitive Apprenticeship to coaching on the job, and link to community to continue and ultimately own the final development? I welcome feedback!