Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Small hybrid multi-agent systems

by Leave a Comment

So, I’ve been pretty harshly critical of the current generative artificial intelligence (Gen AI) situation. Again, I have no problem with the tech itself, but the big providers I have several problems with. Not to rehash those objections, but I’ve been investigating alternatives. And, I think I’ve now converged on a solution: small hybrid multi-agent systems. I owe thanks to several informants, including Elizabeth Dalton, Markus Bernhardt, and Scott Parker.

First, despite my other objections, what I see is that the companies are pushing these changes as ‘inevitable’ as well as desirable. And, sadly, way too many orgs have jumped on board. Which, while understandable, isn’t desirable. What we’re now seeing are the results. Companies are hiring folks back that were fired. We’re also seeing that we’ve hollowed out the pathways for new hires to acquire the necessary skills. Which is part of the larger picture of negatively impacting learning.

Yet, I’ve been a proponent of AI in the past. The technologies on tap have real benefits, whether they’re AI or something that we now understand so we can’t call it AI anymore. So I need to converge on an integration. And, that utility for specific needs is the key to my reconciliation.

In interpersonal situations, I remain a strong belief in ‘partnering’. That is, recognizing that others have complementary knowledge to your own, and different perspectives. So, why have one supposedly all-seeing all-knowing solution, instead of smaller more targeted ones?

Ok, yes, there’re issues of cost. Right now, it’s seemingly cheap to get an AI to do your rote tasks. However, two things are relevant here. For one, the costs are going up. Companies are going public, and that means that they’ll have to start generating revenues to repay the venture capital investments. And we’re seeing that, as companies realize that their AI expenses are growing, to the extent in some cases where they’re more than the people they’re replacing. We’re also seeing that they haven’t yet faced all the incumbent costs: IP, environment, labor. That’ll change the game too.

The flip side is that smaller language models are more cost-effective. There are open source models that you can obtain,  install locally (or on private servers), train on specific documents, and use for specific purposes. Yes, there’s the development costs, but they’re more secure, faster, and less costly. As well as addressing the other concerns.

Moreover, you can have more than one. You can have specialized agents to handle X, Y, and Z. If it’s a language task, you can use a language model. If it’s another type of task, you can use another model. You can even mix in symbolic with sub symbolic, and programs versus AIs. If diversity is good for people, it also appears to be useful for systems. A mix of custom agents with an agreed upon interchange protocol is, it appears, more cost-effective, private, maintainable, and more.

Look, if you don’t mind the costs, and the questionable ownership of data, and security, and dumbing down your workforce, and … go with the big ones. They certainly will benefit. But if you want to be secure and sustainable, small hybrid multi-agent systems makes more sense to me, at least in many instances. Sure, language models trained on all the language out there is useful. Er, for general purpose language tasks. So make that one agent in your solution, if necessary (tho’ aren’t your language tasks typically more constrained?). Which, I’ll suggest, are a small subset of all the things you need to get done by technology. As I learned from the Aussies (originally a Brit saying), “horses for courses”, in this case meaning use the right tool for the job. Am I missing something?

Scams & Recommendations

by Leave a Comment

A blurry woman in white blouse and dark jacket holding up a clear hand in the 'stop' position. Of late, and I’m sure you’re seeing it too, there are an increasing number of attempts to inappropriately separate one from one’s money. Which I understand, if they’re ethical. Often, they’re not. So here are some scams & recommendations associated thereto.

For one, it appears that everyone wants to loan me money. I get offers seemingly every day. Texts, emails, and perhaps the calls that I don’t see if they’re from someone I don’t know. And, at least, these are (presumably at least mostly) legit. That is, loaning money isn’t unknown. It’s just not a great deal. I’d far rather earn money than borrow it. So, no. Please, stop! They’re unsolicited, so they are spam. So, I do recommend blocking, reporting, etc.

Then there’re the actual scams. One that’s become ubiquitous of late are the ones for authors. Sure, I get the requests to translate to another language, for books for which I don’t own the copyright (c.f. my books w/ ATD). Talk to the publisher, man?!?!  The new one (always something new), is something about GoodReads and groups. The promise is that they’ll recommend the book to their group. For a fee, of course. There are glowing reflections on the book and how valuable it is. The clue for me is that sometimes it’s for a new, or relatively new, book. But I’ve also gotten it for books more than a decade old, already out of print, and available for free download. Yeah, I really want to boost sales of that! Don’t fall for it.

There more, of course. There’ve been the companies that want to promote my book at book fairs. For a fee, of course. No. I’ve left much of social media (I miss the folks, but I ethically can’t support what they’re doing), but stay on LinkedIn. However, increasingly I’m getting connection requests from folks not in L&D or associated fields. I’ve got canned responses that either welcome the connection, or ask why the reason to connect. The latter, of course, is for anyone not obviously in L&D. There’ve been folks who want to do tech dev, marketing, financial advisement, etc. And, of course, loans ;). Nope, don’t want the assistance of someone who contacts me cold. Heck, I’d do research if I didn’t know someone personally (and in most cases, I do).

So, caveat emptor. There are lots of folks who’re looking for revenue, and not all are scrupulous. Be curious, but also probably skeptical. That’s my overall approach to scams & recommendations. What’re your approaches?

The right questions?

by Leave a Comment

Like others, I’ve been somewhat less than thrilled with the decrease in quality we’re seeing in more and more of our experiences. Such that Cory Doctorow even termed it “enshittification“. Of course, I haven’t read the full expose, but I also see it in what L&D is doing. Which leads me to wonder if we’re asking the right questions, or focusing appropriately on same.

It’s no secret that organizations are focusing on short-term shareholder returns. And I get it, but…I wonder if that’s the best focus? An apt focus would be on generating both revenue and brand loyalty, which should lead to the returns. However, it seems that the focus is different. And the introduction of generative AI seems to exacerbate that, in that it’s use is too often to produce more, cheaper. Oddly, there don’t seem to be similar rewards for doing more with less…

The appearance is that the focus is on driving more output faster and cheaper. That is, indeed, one way to shareholder returns. However, it seems to me, it also undermines brand loyalty, and thus the long term outcomes are worse. Are we too focused on the ‘short term’ aspect? Sure, shareholders want returns, but they can flip to other investments, so maybe it appears good to deliver? If we just focus on extracting more money, it would seem that it would undermine trust.

Another question we could ask, instead of ‘faster and cheaper’ would be better. How can we increase the customer outcomes and satisfaction? How can we make a better experience? When I look at L&D, for instance, the people I see offering better solutions are doing that. Whether it’s Jess Almlie focusing on being a better business partner, or JD Dillon helping the frontline more effectively, Lori Niles-Hoffman with tech, etc, the answer always is to do better. Which I laud.

To be fair, I’m not a stock investor. I don’t play that form of gambling. Yes, I have investments (all too few, TBH), and they’re managed. I indicate my preferences, and they execute them. However, I’m focusing on long-term benefits. I wish more companies would do that too, particularly online. And I wish more folks would be doing that in the areas they control, say L&D ;).

What is learning

by Leave a Comment

There’s been much debate of late about how generative AI (read LLMs) is taking over our thinking. Which, of course, has caused me to think about what’s crucial for us, as thinkers. And, arguably more importantly, as learners. In short, we’re asking what is learning, and what is it that we can’t abrogate if we want to keep doing so.

So, there’s increasing evidence that when we use AI to just get answers, it’s at least keeping us from learning, if not making us stupider (warning: PDF). The point is apt, to develop, we need what’s termed ‘desirable difficulty’. That is, we need to challenge ourselves. You can think of it as keeping an exercise regime, except we don’t plateau, we keep learning. There is, effectively, no limit to what we can learn; we run out of time before we run out of new things to learn!

What happens when we ask an AI for an answer is that we aren’t exercising our own minds. It’s been made clear that we need to do the work ourselves – tackle the task – so that we activate and leverage our cognitive representations for them to strengthen. To be clear, there are things we don’t want to learn. For instance, I’m fine not having learned how I repaired my dryer; I’m not a dryer repairperson. However, I don’t want some arbitrary summary that might be wrong, I prefer to find a document created by an expert who guides me through the steps to diagnose and repair it. And then I’m fine not learning, because I hope to never have to do it again!

The interesting question then becomes, what do we do? Markus Bernhardt has suggested there’re five steps, from working with a contractor. These are briefing, judging, composing, selecting, and building. Only the first two are before you talk agents (I’m not, for reasons). However, I want to simplify it, and suggest that there’s essentially one step that’s critical. As suggested by the above.

The main thing I think one needs to do is first generate our own ideas. I’ll suggest that an LLM can be a good partner for thinking. It can generate ideas you haven’t, though of course it may also generate ideas you reject. So evaluating the output is important (requiring an expert). But what I now think is that you should generate your ideas first. Then see what you’re coming up with. It’s an extension of the idea of self-evaluation of free form responses, with a rubric. Particularly when learning to learn on one’s own, you should be generating responses, but you need feedback. Without an instructor to do so, you need a way to evaluate your own output. Comparing your answers to someone else’s is good! Particularly if you’ve done the work beforehand to know what a good answer should contain or consists of.

Let’s be clear; you don’t have to flesh it all out. What you do need to do is get concrete enough that you know your own ideas. It’s one thing to think you’ve got an answer, and another to actually try to capture it. It’s a pretty common phenomena to realize, when you try to get concrete, that you need to tweak a few things. That thinking is part of the learning. But a quick phrase, a sketch, whatever captures your thinking quickly can be enough, it’s just that you have to generate the idea.

So for any area you want to grow in, you can’t abrogate your responsibility. That’s a negative, not a positive, path. For other things, I prefer the best expert advice, not a potentially wrong summary generated by a stochastic parrot (yeah, I said it ;).  But when it’s important for your learning, and that is something you can and should be doing, you need to do the work. Sure, get assistance, but do the work and then get feedback. And evaluate the feedback to understand how you can improve. But, do the work. That’s where the learning happens. That’s what is learning: action and reflection. So, as the Nike ad says, Just Do It!

 

 

On ‘cognitive presence’

by Leave a Comment

I recently opined that while I thought the ideas in an article by Dr. Page Chen (COO of Elevator 9, the startup I advise, and very smart) were brilliant, I had reservations about the term ‘cognitive presence’. And maybe I should explain why. So let me do some riffing on cognitive presence.

First, it’s a well-regarded concept. In typical formulations, it’s part of a three-element model of learning under the concept of ‘community of inquiry‘, in conjunction with social presence and teaching presence. Further, it has a well-known definition, with specific phases that are part of my concern about going all the way from learning to application. Specifically, it includes triggering of the idea, exploration of the concepts, integration of the ideas, and resolution into a coherent whole. It is a bit academic, in that it doesn’t necessarily imply ‘doing’, but that’s obviously where it’s being taken.

One issue is that it’s well-known in education. That is, it’s known there, but it’s not necessarily cognizant to other audiences. In particular, I wouldn’t expect organization executives to know it. Frankly, I’m not sure all L&D folks know it! Thus, it’s a concept that’s accurate, but maybe not understood when talking to customers (or investors).

So why my concern? Because the label feels a bit ‘inert’. Presence means, to me, “I’m present”. I’m here. Which doesn’t mean I’m engaged, applying, and closing the loop. It’s absolutely the right idea, it’s just not a good label. That is, for those who don’t about the concept, this label isn’t going to inspire them. And, to be fair, I’m totally willing to be wrong on this. It’s just that I fear that the term, without explanation, isn’t going to lead to the insights about what we’re doing that we want.

Thus, I’m leery of the term, and somewhat keen to find another. Of course, I don’t have an immediate alternative to offer. What would be good? It’s about capturing progress along the trajectory from knowledge to application. Is that the right term, application trajectory? Or…? I’m not (yet? ;) at the stage of having answers, but felt like I should explain my thinking (learning out loud, after all!). What’re your thoughts?

The benefits of learning ‘out loud’ (also known as making a mistake)

by Leave a Comment

I just released a blog post this morn. And, as soon as I saw it in the wild I had a brutal realization. What occurred to me, of course, is that I’m going about it totally wrong! Or, at least, totally contrary to what I advocate. Yes, I had activities, but I focused on the content. Not a bad thing, maybe, as a step, but the first step? Maybe not. So I’m experiencing the benefits of learning ‘out loud’, via making a mistake.

To start, I’ve been designing a workshop. And, it’s taking me a while, as I want it to be thorough, and good. For instance, I deliberately have it spread out, so to ensure spaced learning. I have reflection in it. Further, I’ve made ‘ToDo’s to go back and add in some processing activities. I largely have application activities already put in. That’s what I talked about. In particular, ensuring good coverage. However, once it was posted (this morn as I write, two days ago as you read), I realized something.

The design I was talking about was, sadly, very different from what I’ve advocated for in the past. And, still believe in! That is, starting with what I want learners to be able to do, and working backwards. Focus first on activity, then on the necessary content. Which is very similar to Cathy Moore’s Map It, but with a bit more focus on the rationale, and feedback (my take, YMMV).

What that means is what I should do is create the activities first. Then, I figure out the minimal content. From there, I can dig into processing activities ala Richter (via Thiagi). Along the way, I can review the activities, match it against the content I’ve developed, and see if need more or different. Yes, it’s an iterative process. But, to be true to my beliefs, and science, and learners, and…I have to design the activities first. So, I’ll keep my existing deck, and start afresh on one (or, at least, some representation). It’s better to take that time. And, yes, it’s not like the real world pressures most have to work through, but then I’ve the luxury to try to do the right thing, not the expedient things.

Let’s be clear: the workshop I designed would be really good, possibly great. I think reflecting and focusing on the activities, however, has a high probability of making a definitively great workshop. Those are my reflections, what are. yours?

There may be a followup post about the revised process, but no promises right now.

Designing a workshop

by 1 Comment

I’ve written about writing before, but I don’t think I’ve talked about designing a workshop. Yet, that’s what I’m doing right now, and so I’m also reflecting. Which, of course, is very ‘meta’, because reflection is one of the topics of the workshop! Still, I’m pondering what’s different about designing a workshop versus writing, whether book or article or… So here’re some thoughts.

First, as is the case with writing a book (at least for me) is the process of creating the structure. I outline what I’m doing (even both outlined it and separately created a mindmap, to see what overlaps and gaps I found). For instance, I initially saw this as five separate events, spaced out. When I got into it, putting meat onto the bones as the saying goes, I realized that I wanted to combine three and four. So now there are four separate bits.

Then, as is the case with writing, too, I realize I’m alternating between top-down and bottom-up. Which is demonstrably what designers do, but…it’s interesting to break the myth of the waterfall design. I generate low-level elements, which makes me rethink and restructure bits, which means I need to go down into the details some more.

It’s also the case that I can disappear into design for too long (fortunately, the ribs came out ok). I personally go in bursts. I will get going and time will fly by, and then I hit a wall and just have to stop and take a break. It also is mentally taxing, and I find my thoughts become difficult until I take some time off.

And, I recognize it will take several passes. Already I’ve made ‘todo’s for two specific post-hoc passes. First, I’m getting the ideas and the structure right. Then, I want to go back and make sure I’ve put in all the good ideas, hints/tips/tricks, and all that ensures there’s real value in what I’m doing. Finally, I want to make a third pass that ensures I’m helping the audience process the content successfully. That is, getting them to rephrase the ideas into their own words (a form of reflection).

The latter of which is, specifically, what’s at least partly unique to a workshop versus a book. Yes, I’m making it interactive in the first pass by asking an opening question for each section to get folks creating their own thoughts before I present mine, so as to activate and also to generate any improvements that can come from them. While I think I have plenty of relevant background, I’ve not enough hubris to think that there won’t be ideas that are better than mine. I learn when I run workshops, too!

One of the important things here is to develop real new abilities, not just to present ideas. Learning science tells us we need repeated practice, and that’s definitely got to be part of what this workshop does. So, not only do I want some scaffolded practice in the session, but there’s also ‘homework’ where attendees apply the current idea to their own work and share the learnings therefrom in between sessions. (Research emerging from our work at Elevator 9 suggests that when this is done – and it’s hard to do – the results are better.)

The goal is a workshop that delivers real learning, and real value. I don’t want to just make money selling ideas and good feelings. That’s too frequently what we see (*cough* leadership development *cough*), but that’s not what will lead to long term benefit. And I do care about that (perhaps too much! ;). So, there are some thoughts on designing a workshop. What haven’t I made clear? What am I missing?

(And no thanks to Apple for losing my templates and making me create them anew. Ahem.)

Schemas versus Mental Models

by Leave a Comment

For reasons (that elude me, except ongoing curiosity), I’ve been looking at ‘content’. That, of course, has meant models, and examples, and feedback. However…my co-director at the LDA*, Matt Richter, chides me that schemas are a goal of learning. And, of course, I was steeped in schemas as a grad student in cog psych. On the other hand, I’ve pushed for mental models as the complement to examples (and the basis for feedback). So how do I reconcile schemas versus mental models?

I got a boost on my recent explorations via this post from Paul Kirschner distinguishing between the two. His characterization of schemas is as fixed knowledge about the world, while models are systems reflecting the dynamics. He also talks about how misconceptions can come from either, and they’re separate, suggesting they’re distinctive.

My searches, however, led me to this other piece by David Didau that argues that they’re not functionally dissimilar. They’re conceptually distinct, but functionally you need a combination. Which makes sense, because there’re interrelations.

The reconciliation I’m finding is that we need models that explain how the world works. As I’ve maintained, you have to predict the way the world reacts to this perturbation versus that one, and then decide which outcome is better. However, you can’t really do that until you know what the elements are, and the relationships between them.

The argument then becomes how you teach. And I reckon you kind of build them iteratively, actually, because you need some to explain some, and then some more to explain some more. Which isn’t all that unfamiliar. I use ‘builds’ to help people see how a model is built, by exposing the diagram bit by bit.

That is, ultimately, that it’s not about schemas versus mental models, it’s about schemas and mental models. Which is a good resolution. Of course,

* Our society for evidence-informed L&D.

KMN

by 6 Comments

So, I’m compiling a hopefully comprehensive list of learning to learn strategies. As part of that, I searched on meta-learning strategies. And, in two of the top listed sites, I found, indeed, articles on meta-cognitive strategies. And, a myth. More specifically, one that continues to bedevil our industry (and, relatedly, education). I mean, we’re a quarter way into the new century! I sadly feel like KMN (Kill Me Now).

So, the first site had an article that asked “What meta-learning techniques improve skill acquisition?” After first saying why and what, both good things, the very first recommendation is “understand and adapt to your learning style”. What? I mean, this is such a debunked theme that it’s almost ludicrous. I did comment, but yikes.

So, then going through my other tabs opened from the search results (a meta-learning strategy, btw ;), I find another list. This site has an article listing Examples of Metacognitive Strategies. And, it was only item 5 that was “awareness of learning styles”. Still, here’s a myth pretending to be a valid strategy. Here I couldn’t comment, but I did send a politely worded recommendation to pay attention to the research. (And got a response saying they’d moved on from that position. Ok, but then why did I find it when I did the search?)

In both cases, pointing to an approach that has been deeply investigated and resoundingly dismissed undermines anything and everything else they have to say. Why should I trust what you tell me when I know part of it is wrong? There was other good, and bad, advice in the articles, but sorry, you’ve lost my ability to think you know what you’re talking about.

Yes, I get that the idea of learning styles feels right. Anyone who’s taught recognizes that people learn differently. But…what people think is good, and what actually works, has essentially zero correlation. Yes, you should have different representations. This is to increase access, not to address learning styles, however. And deliberately designing for styles is clearly a waste of money. For instance, creating three different versions of the content for say, visual, auditory, and kinesthetic learners, wouldn’t be justified.

A step back provides this lesson: know what you say before you say it, and don’t say wrong things unless you want to lose credibility. Which is kind of a meta-comment, but that’s a separate issue ;). I don’t really want you to KMN, but…sometimes it feels like we are moving backwards. Can we return to using what’s demonstrably known as a basis? Sure, science gets upgrades, and so sometimes is wrong (flat earth, anyone?) when it moves to a better explanation, but it’s still better than a basis of dogma.

Meta-learning-to-learn

by Leave a Comment

So, my attention has swung back to learning to learn. So that requires me to think what about meta-learning is worth paying attention to, promulgating, etc. Which is kind of meta-learning-to-learn, no? So, why, and what?

To begin with, as I was explaining to my LDA co-director, Matt Richter, I’ve been a fan of learning to learn for literally decades. As a grad student, I served as a teaching assistant for a class on critical thinking (using Halpern’s Thought and Knowledge). My own PhD thesis was on teaching thinking skills! I joined up with the late Jay Cross to form the Meta-Learning Lab (MLL), promoting these views, after finding it was a shared interest amongst those of us in the East Bay who knew Bill Daul.

So, it makes sense that this theme, connected strongly to my life-long pursuit of learning through technology, would resurface. I think the MLL was ahead of it’s time, but I’m seeing more resurgence of interest. I think that’s for several reasons, for one orgs are realizing that they need to become more agile, and learning skills are key. Also, generative artificial intelligence is raising questions about what value we add, and I’ll go out on a limb and state that to learn is human. Yes, machines learn, but…they really don’t have an understanding, as I’ve pointed to Harnad’s Symbol Grounding Problem before. Arguably, we do increase understanding through learning.

It’s also clear that we’re not optimal. Some argue that there aren’t ‘learning to learn’ skills, yet point out that certain ways of doing things (across domains, mind you), are more and less efficient. You can be better or worse at reading, experimenting, even interacting with others. Sorry, but to me, that’s learning to learn.

Also, I see more and more folks recognizing that these skills are desirable, and undeveloped. So, if we wanted to turn that around, and develop them, what’s needed? That’s where I’m at, compiling a list of skills, and trying to generate a unified model. Yes, there are components – Jarche’s PKM, cognitivce science’s plan-monitor-evaluate, etc – but I reckon it’s best if they’re integrated. That includes breaking down so-called 21C skills into actual component.

Still underway, and I welcome pointers to comprehensive models. I confess that I quibble if I think they’re insufficient (there are lots of models that have necessary, but not sufficient, coverage). I don’t mind reviewing (and stealing :) any elements I think I’ve missed, but I also won’t tout anything that’s incomplete. That includes, BTW, culture, and importantly, how L&D facilitates this. That’s where I am going, of course; we need to find days to build these skills into what we do. For instance, one of our LDA members talked about making pedagogy clear when you deliver a learning experience. That’s exactly right. So, what else? What’s the comprehensive list? Stay tuned.