Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Archives for December 2025

Aphasia analogy for LLMs?

by Leave a Comment

Something triggered, for me, an analogy. I was thinking of aphasias, and thought one might be a good analogy for LLMs. Both to understand, and to use as guidance for using. So here are some initial thoughts on an aphasia analogy for LLMs.

First, while aphasia is complex, two types reliably correlate with damage to specific areas of the brain: Broca’s, and Wernicke’s. Interestingly, they’re partners, as both deal with knowledge and language. That is: what we know, and how we’re able to communicate. Both come from damage to a specific part of the brain, but have opposite effects.

Broca’s aphasia is reasonably clear. The evidence suggests that folks retain their knowledge, but struggle to communicate. That is, what comes out is broken and ungrammatical, but meaningful. People generally have no trouble thinking, just talking about it. There of course is a region of the brain for Broca.

The companion is Wernicke’s aphasia. Here, the language is eloquent, but essentially nonsensical. People may have thoughts, but what they say has no internal cohesion. There can even be made-up words! There’s also a region of the brain known for Wernicke.

You can probably figure out where this is going: LLMs learn from large corpora of text to produce accurate language. Not accurate answers, but accurate language, an important distinction! They’re not damaged, so if most of the training set is accurate, what they say will be accurate. However, if not, they may say other things. And, of course, they can just say things that sound right that aren’t, such as making up books, court cases, and more. They’re essentially Wernicke systems!

What does this mean? It means you probably can have a good idea generation sessions with an LLM, or give it a language task like summarizing or generating language. What it also means is that there’s no way that you should trust what comes out to be accurate, and you need an expert in the loop. Given that they’ve demonstrably been corruptible, besides not always being correct, however, they shouldn’t be trusted to act on your behalf. I worry a wee bit about them being good enough that most of the time what comes out is ok. This could, of course. lull you into a sense of complacency! Hopefully folks will always feel the professional obligation to ensure that what comes out is correct.

So, understand what LLMs do, if not how they do it. Then, act accordingly. (Feel free to also investigate issues like IP, environment, biz model, and more.) Caveat emptor!

Age or experience?

by Leave a Comment

One of the things that has been a recurring theme across things I’ve been looking at lately is experience. Too often we confound age with experience. And, of course, sometimes it’s that we should be talking about it. So, a brief rant on age or experience.

First, I’ll bring up the ‘generations’ myth. It’s appealing, as our brains like buckets for things. We’re kinda wired that way. The only problem is that generations as a concept has been looked at and debunked. Heck, in Ancient Greek days they were complaining that ‘kids just have no respect”! And if you think about it, thinking that someone in Los Angeles CA of a certain age has more in common with someone in Nepal of the same age versus another Angelenõ of a different age is kinda ridiculous.

And, those ‘defining’ events? They affect every conscious person! And it’s so context dependent. A local event may not mean much to you, unless it affects you somehow, and then you share more with everyone else so affected. There’s actually a simpler explanation. Say, for instance, that “young folks want classes while old folks don’t”. That’s explainable by stage of life: when you’re young you need credentials, but later on you can point to your experience.

People share values, and gain motivation by the same underlying factors (differently across culture and personality), and more. Just look at the research on self-determination theory! Attributing to age rather than explaining by experience is a mistake. So, for instance, my kids, who arguably fit the label ‘digital natives’, still come to me (decreasingly, I’ll admit) for tech problems.

Then, there are many things that change as you develop in a domain. For instance, in our Learning Science Conference, my colleague Matt Richter was talking about feedback, and very clearly pointed out how what useful feedback is changes as you gain experience. This holds true for examples, too, the type of useful example changes. Also for practice: with more experience, you need more challenge.

Which, as we further see, is how we go wrong. We do the ‘one size fits all’, not recognizing that things need to change. To be fair, we also do the wrong practice (knowledge test rather than application to problems), give the wrong feedback to begin with (right/wrong), the list goes on. But even when we’re trying to do it right, we forget things like adapting for initial and developing experience. Yet, it’s a factor for instance in how much practice you need, how much spacing, etc.

This problem does go more broadly. We hear it in hiring (age discrimination). Of course, that’s only one problem. For example, gender, race, physical and neurological differences, and more are also present. Sadly. Okay, soapbox: DEI, done right, leads to better outcomes! Actually, that’s got an evidence-base, so probably more than soapbox. Still. So, consider experience as one of the factors distinguishing individuals. Folks can’t control their age, but they can determine their experience. So use it!

Key notes

by Leave a Comment

I’ve seen a lot of keynotes over the years. I’ve even given them! It’s time to reconcile my thoughts. So here are some key notes on keynotes.

One of the things that I’ve seen is flawless performances. Now, one of the things you’re told is that the focus is on you, and slides only should be used as an augment, not on all the time. I confess I’m not good about that (I am not very comfortable in the spotlight; imposter syndrome I suppose). Another is that you should have pauses, and jokes, and such. I do a pretty good dramatic reading (I won the dirty limerick reading contest at work once!), and occasionally even manage to raise a smile or two. The best, however, have their patter completely down. I can’t do that, because my thoughts are continually evolving, but I admire it when it happens. And I’m pretty good at tailoring a talk to the audience (having learned a few times the hard way!).

Inspiration is good, too. Letting people know there’s a way to surpass this barrier works! I try to do that too, though I confess I talk about learning design, not achieving things like climbing Mt Everest (really heard a keynote about that, and it was cool!). And I probably am a bit too conceptual, though I am learning to do better about grounding my principles in practice. However, I do recoil from too much ‘enthusiasm’! Somehow it comes across as artificial. But then, I can be a bit of a curmudgeon (apparently)…

However, what really matters to me is accuracy. I don’t mind if folks are a bit enthusiastic or polished, but I really get wound around the axle a bit when folks state stuff that’s just wrong. For instance, I heard a well-regarded personage opine about games, something I know a wee bit about (my first job, back when dinosaurs strode the earth, was on games, and it’s been a recurrent them in practice, research, and writing for literally decades). And that individual said something just dead wrong. As you may surmise, it really ground my gears. Similarly with learning science, or mobile.  In general, when people have beautifully symmetric ‘n part models’ without grounding, I want to know if those are convenient, or a necessary and sufficient list. (Too often the former.)

I also like when people tout doing things I believe in, but when it’s an area I know about, you better agree with the science, and present it accurately. If you don’t, well, I won’t be quiet about it. (I guess it’s a flaw in my character!) Still, when you say these are the five things to X, and they’re a) not completely separable, b) incomplete, c) wrong, etc, I’m going to be turned off.

Look, I love a good keynote. Many times, they get people who aren’t from the field where the keynote’s presented, and they make connections from adjacent fields. That’s acceptable, even desirable! I like a well-presented talk as well as the next person. I like ideas, and even inspiration. But I will complain about bad information. Always. Those are my key notes on keynotes, what are yours? (And I’m available, if you want L&D advice ahead of the curve but grounded in evidence. Particularly contrary takes… ;)

Analyzing analysis

by 1 Comment

Another reflection, triggered by my visit to DevLearn. One of the things that matters, and we don’t discuss enough, is analysis. That is, starting up front to determine what we need! There are nuances here, and I’m not a total expert (paging Dawn Snyder), but certain things are obvious, So let’s take some time analyzing analysis.

Analysis is the first part of the process. Yes, there’re the organizing and managing bits, but the process starts with analysis, whether ADDIE, SAM, LLAMA, or any other acronym. You need to determine what’s going on, what’s the need, and what’s the appropriate remedy.

One of the first things to note is that not everything L&D does fits. As is widely noted (e.g. here), there are lots of reasons courses aren’t the only answer. The real trigger should be a need. That is, there’s a new skillset required to do this thing we’ve identified as wrong or necessary. Or, there’s something we’re doing, but badly. At core, there are two situations: the one where we need to be, and the one where we are. The gap between is what we want to remedy.

Then, it’s matter of determining why we’re not where we want to be. The reason is, there are different interventions for different problems, as Guy Wallace talks about in his tome. It might be a lack of resources, or people get rewards for doing X, even though it’s Y they’re to be doing. These, by the way, aren’t things we deal with! That’s why you do this, so you don’t build a solution where said solution actually isn’t.

When it is a situation where knowledge in the world, or in the head, will help, then we can jump into action. Of course, we need a clear definition of what it is people need to be able to do, under what conditions, etc. BTW, what we need are performance objectives, not ‘learning’ objectives. That is, it’s about doing. Which is why, if the circumstances support, we should be providing job aids, not courses! You’ll usually find that job aids are cheaper to do than courses. If it’s not being performed very frequently, or too frequently, memory will play a role, and external memory is valuable in many such circumstances.

When you’ve determined that a course is needed, you can develop that. HOWEVER, you need certain things from the analysis phase here too. In short, you need to understand the actual performance. That includes what the performance should be, and how you can tell. Essentially, you need to know the decisions people must make to deliver the required outcomes. Which involves knowing the models that describe how the world works in this particular area, what ways people go wrong and why, and why people should care. This is where you need your subject matter experts (SMEs).  Then you can build your practices that align, and the models and examples, and then the hook and closing, and…

Whatever it is, ideally there’s a metric, that says this is what’s needed. You design to that metric, and then test until you achieve it. If you’re not achieving it faster than you’re losing resources, you can consciously evaluate. Is the lower level ok? Can we get more resources? Should we abandon ship? But doing so consciously is better than just going ’til you run out of time and/or money.

Analysis is a necessary first step. What is not is responding with acquiescence to a ‘we need a course on X’ request. Do you trust them to know that a course on X solves their problem? (Not the way to bet.) You can, and should, say, “yes, and…let’s dig in and make sure we’re solving the right problem”. Analysis is, properly, the way to start looking at problems. You understand what the gap is, then the root cause, and then align an intervention, or interventions, to address it. By analyzing analysis, we can figure out what we have to do, and why.

And, yes, I just gave a talk on designing in the real world, and you may have to do inference on resources to determine all the above, but at least you know what you need to come away with.

Beyond LLMs

by Leave a Comment

So, I was recently at the DevLearn conference, and it was, as always, fun. Though, as you might expect, there as non-stop discussion of AI. Of course, even on the panel I was on (with about 130 other Guild Masters; hyperbole, me?) it was termed AI tho’ everyone was talking Large Language Models (LLMs). In preparation, I started thinking about LLMs and their architecture. What I have realized (and argued) is that people are misusing LLMs. What became clear to me, though, is why. And I realize there’s another, and probably better, approach. So let’s talk going beyond LLMs.

As background, you tune LLMs (and the architecture, whether applied to video, audio, or language) to a particular task. Using text/language as an example, their goal is to create good sounding language. And they’ve become very good at it. As has been said, they create what sounds like good answers. (They’re not, as hype has it, revolutionary, just evolutionary, but they appear to be new.)

I made that point on the panel, asking the audience how many thought LLMs made good answers, and there was a reasonable response. Then I asked how many thought it made what sounded like good answers. My point was that they’re not the same. So, they don’t necessarily make good design! (Diane Elkins pointed out that they trained on average, so they create average. If you’re below average, they’re good, if you’re above average, they’ll do worse than what you’d do.) I ranted that tech-enabled bad design is still bad design!

However, I’ve been a fan of predictive coding, as it poses a plausible model of cognition. Then I heard about active inference. And, in a quick search, found out that together, they’re much closer to actual thinking. In particular, combined, they approach artificial general intelligence (AGI for short, and wrongly attributed to our current capability). I admit that I don’t go fully into the math, but conceptually, they build a model of the world (as we do). Moreover, they learn, and keep learning. That is, they’re not training on a set of language statements to learn language, but they’re building explanations of how the world works.

I think that when we want really good systems to know about a domain (say business strategy) and provide good guidance, this is the type of architecture. What I said there, and will say again here, is that this is where we should be applying our efforts. We’re not there yet, and I’m not sure how far the models have evolved. On the other hand, if we were applying the resources going to LLMs… Look, I’m not saying there aren’t roles for LLMs, but too often they’re being used inappropriately I think we can do better when we go beyond LLMs. You heard it here first ;).