Search Results for: align

Today I read that Anthropic has released Claude for Education (thanks, David ;). And, it triggered some thinking. So, I thought I’d share. I haven’t fully worked out my thoughts, so this is preliminary. Still, here’re some triggered reflections on Intelligent Tutoring via models.

So, as I’ve mentioned, I’ve been an AI groupie. Which includes tracking the AI and education field, since that’s the natural intersection of my interests. Way back when, Stellan Ohlsson abstracted the core elements of an intelligent tutoring system (ITS), which include a student (learner) model, a domain (expert on the content) model, and an instruction (tutoring) model. So, a student with a problem takes an action, and then we see what an expert in the domain would do. From that basis, the pedagogy determines what to do next.  They’ve been built, work in research, and even been successfully employed in the real world (see Carnegie Learning).

Now, I’ve largely been pessimistic about the generative AI field, for several reasons. These include that it’s:

• evolutionary, not revolutionary (more and more powerful processors using slight advances on algorithms yields a quantum bump)
• predicated on theft and damage (IP and environmental issues)
• likely will lead to ill use (laying off folks to reduce costs for shareholder returns)
• based upon biz models boosted by VC funds and as yet still volatile (e.g. don’t pick your long term partners yet)

Yet, I’ve been upbeat for AI overall, so it’s mostly the hype and the unresolved issues that are bugging me. So, seeing the features touted for this new system made me think of a potential way in which we might get the desired output. Which is how I (and we) should evolve.

As background, several decades back I was leading a team developing an adaptive learning system. The problem with ITS is that the content model is hard to build; they had to capture how experts reasoned in the field, and then model it through symbolic rules. In this instance I had the team focus on the tutoring model instead, and used a content model based upon learning objects with the relationships between them capturing the knowledge.  Thus, you had to be careful in the content development. (This was an approach we got running. A commercial company subsequently brought it to market successfully a decade after our project. Of course, our project was burned to the ground by greed and ego.)

So, what I realized is that, with the right constraints, you could perhaps do an intelligent tutoring system. So, first, the learner model might be primed by a pre-test, but is built by learner actions. The content model could come from training on textbooks. You could do either a symbolic processing of the prose (a task AI can do), or a machine learning (e.g. LLM) version by training. Then, the tutoring model could be symbolic, capturing the best of our rules, or trained on a (procured, not stolen) database of interventions (something Kaplan was doing, for instance). (In our system, we wrote rules, but had parameters that could be tuned by machine learning over time to get better.)

My thought was that, in short, we can start having cross-domain tutoring. We can have a good learning model, and use the auto-categorization of content. Now, this does beg the problem of knowledge versus skills, which I still worry about. (And, continue to look at.) Still, it appears that the particular solution is looking at this opportunity. I’ll be keen to see how it goes; maybe we can have learning support. If we blend this and a coaching engine…maybe the dream I articulated a long time ago might come to fruition.

I’ve written in praise of the cognitive and learning sciences. I, however, need to take a step back. It’s becoming increasingly clear to me, sadly, that there are attacks on science itself.  Yet, I have a strong belief that it matters. So let me briefly address the question of why science.

As background, I have been steeped in science. It was one of my favorite topics in school, and in college. My PhD is in the underpinnings of how we think. Though it’s been a long while since I was an active scientific researcher, I still apply what’s known. Moreover, I continue to track developments, so I can continue to do so. 

As a result, I’ve been a fan of the work of scientists in the cognitive and learning fields. I’ve not only had training in the methods, but I also continue to explore more broadly the methods and the applications. I also love the translators who take that research written in the original academese and turn it into practical advice. Heck, I’m co-director of a society about evidence-based practices. 

There has been some ‘confusion’ about the scientific process. “How can you trust it if it admits it’s been wrong?” Er, that’s what it’s about, continually creating explanations about the world. When we know more, we may need to change our explanations. We went from the sun circling the earth to the other way around, and we no longer (should) think the world is flat. If you don’t believe in the findings, how (and why) are you reading this? Technologies developed from scientific endeavor. 

To be fair, science has been used for ill as well as good. That’s about people’s ethics, not the outcomes. We have to be mindful of how we apply what we learn. That’s up to our values and morals, which science actually has a lot to say as well. For instance, I’ve made the case that research tells us we do better when we’re inclusive. That’s science telling us what values lead to the best outcomes. When we work with what we know about how we think, work, and learn, we improve the outcomes. 

The evidence says that science is better than any alternative. When we apply evidence-based practices, we get the best results. That’s a win. When we turn our backs on it, we lose. Lives can be negatively impacted or lost. That’s not a win. And for our orgs, ignoring science in marketing, operations, sales, etc doesn’t make sense. So, too, for learning and ‘human resources’ in general. And, that’s true for society and government as well. So let’s make sure we’re making decisions in ways that align with science. It may seem more expedient in the short-term to do otherwise, but the long-term results argue for us doing the right thing. When there’re conflicts between beliefs and the evidence, things go better when we adapt beliefs and go with the evidence. “Why science” is because it works better. 

So, this is a wee bit not my normal post, but…I did want to let you know about the Evidence-Informed Practitioner conference we (the Learning Development Accelerator) are running come April. This won’t be my last post on it, of course!  Still, I’ll entice you with some details, and give you a special deal. It’s too good not to let you know about the Evidence-Informed Practitioner conference deal.

So, first, the conference is a follow-on to the Learning Science Conference we held last fall. That was a great conference, but there was one repeated sentiment: “but how do we do this in practice?” A fair question!  And, frankly, a topic that’s gotten my mind going in other ways (stay tuned ;). So, we decided to offer a conference to address it.

First, the conference follows the well-received format we saw for that last event. We have the important topics, with canned presentations beforehand, discussions forums to discuss, and then live sessions. The presentations were great, and the emerging discussions were really insightful!

Then, we have top presenters, and I mean really top. People who’ve been there, done that, and in many cases wrote the book or built the company. Julie Dirksen, Dawn Snyder, Will Thalheimer, Lori Niles-Hoffman, Dave Ferguson, Emma Weber, Maarten Vansteenkiste, and Nigel Paine, along with Nidhi Sachdeva and Kat Koppett. These are folks we look to for insight, and it’s a real pleasure to bring them to you.

I get to offer you 10% off. You can use my code to get 10% off the regular price. The secret password is EIP10CQ. That’s EIP (the conference acronym), 10 (percent), CQ (my initials).

I realize I should’ve mentioned this all before, but it’s not TOO late. Hope to see you there, it’ll be great (as my firstborn used to say)! Look, I don’t usually do such a promotion, but I really am excited to offer Evidence-Informed Practitioner conference deals. Hope to see you there!

Two recent things have prompted some reflection. For one, the LDA had another workshop with Emma Weber, in this case on transfer of learning. At the same time, Dave Snowden, on LinkedIn, was pointing to a post suggesting being wary of the latest management infatuation. How are they related? Well, to me it’s about fads and foundations.

So Emma’s workshop was about how to use coaching to facilitate post-event transfer. Her approach had a domain-independent coaching model. In it, the coaching is applied for roughly 30 minutes over a period of time, with at least a week between. She was looking to drill into what people wanted to accomplish and keep them on track. Also, doing so without being expert in the area of endeavor. In fact, to the contrary. Which I laud, with a caveat. As I’ve opined before, I think that we need domain-specific feedback until learners have a level of capability. They have to be able to know  what they don’t know and acquire it. They also need to critique their own performance. (She believes that the course should get people to that level; I’m a bit more cautious. Should.)

Now, what the post suggested was that the big consulting companies had a pattern of boosting the latest management approach. They then indicate expertise, and get businesses to follow them. The consultants then move on, without checking to see whether the fad has led to any improvement. (A small plug here for using your friendly neighborhood consultant for a reality check before embarking on heavy investment.) This reminds me of Alex Edman’s book May Contain Lies where he demonstrated how many management books took a biased data set and used that to make sweeping generalizations that weren’t justified. Nor checked for continuing success.

The link is that too often, folks will bring in a new executive, even CEO, who isn’t in their business but has had success elsewhere. A reliable situation is that they will have learned some MBA-spiel, like cost-cutting, and successfully applied it in a particular instance. (The ones who aren’t successful we don’t hear about.) Then, their approach doesn’t work in the new situation. Because it’s a new situation! They don’t have the foundational knowledge. Another recent item I saw said how a business had failed with a new CEO, and had to then hire another who knew the business to set it right. (If only I could remember where!)

The underlying message is that the world is contextual (see Brian Klaas’ Fluke). Without the knowledge of how the world works here, we’re liable to apply too-general approaches that aren’t matched to the current situation. When we acquire the contextual knowledge, we can then self-help. Yet, we do better when we know the situation. We need informed analysis and aligned interventions! This is something we can, and should, do.

(Have I talked about looking across contexts for learning before? I looked and couldn’t find it. Though I’m pretty good about sharing diagrams?!? So, here it is; if again, please bear with me).

In our recent learning science conference, one topic that came up was about contexts. That is, I suggest the contexts we see across examples and practice define the space of transfer. We know that contextual performance is better than abstract (c.f. Bransford’s work at Vanderbilt with the Cognitive Technology Group). The natural question is how to choose contexts. The answer, I suggest, is ad hoc: choose the minimal set of contexts that spans the space of transfer. What we’re talking about is looking for a set chosen across contexts that support the best learning.

So, in talks I’ve used the diagram to say that if you choose the set of contexts represented by the ‘o’s, you’ll be unlikely to transfer to A, whereas if you choose the ‘x’s, you’re much more likely. Let me make that concrete: let’s talk negotiation (something we’re all likely to experience). If all your contexts are about vendors (e.g. ‘o’s,) you may not apply the principles to negotiating with a customer, A. If, however, you have contexts negotiating with vendors, customers, maybe even employers (‘x’s), you’re more likely to transfer to other situations. (Though your employer might not like it! ;)

The point that was asked was how to choose the set. You can be algorithmic about it. If you could measure all dimensions of transfer, and ensure you’re progressing from simple to complex along those, you’d be doing the scientific best. It might lead you to choose too many, however. It may be that you can choose a suite based upon a more heuristic approach to coverage. Here I mean picking ones that provide some substantive coverage based upon expertise (say, from your SME or supervisors of performance). I suspect that you’ll have to make your best first guess and then test to see if you’re getting appropriate transfer, regardless.

It’s important to ensure that the set is minimal. You don’t want too many contexts to make the experience onerous. So pick a set that spans the space, but also is slim. The right set will illuminating the ways in which things can vary without being too large. Another criteria is to have interesting contexts. You are, I’ll suggest, free to exaggerate them a little to make them interesting if they’re not inherently so.

You may also need some times when the context says not to use the focus here. What I mean is that while it could seem appropriate to extend whatever’s being learned to this situation, you shouldn’t. Some ideas support over-generalization, and you’ll need to help people learn where those limits are.

Note that the contexts are those across both examples and practice. So, learners will see some contexts in examples, then others in practice. It may be (if it’s complex, or infrequent, or costly) that you need to have lots of practice, and this isn’t a worry. Still, making sure you’re covering the right swatch across contexts will support achieving the impact in all appropriate situations.

I’m less aware of research on the spread of contexts for transfer (PhD topic, anyone?), and welcome pointers. Still, cognitive theory suggests that this all makes sense. It does to me, how about you?