Learnlets

Secondary

Clark Quinn’s Learnings about Learning

You are here: Home / Archives for strategy

AI and Meaningful Practice

by 1 Comment

Again, a video of an idea I want to talk about. This time about AI and Meaningful Practice (just around 2 minutes). I welcome your thoughts.

By the way, I’m experimenting with video as a blog mechanism. A colleague mentioned that no one remembers the author of an article or post, but they do remember the speaker in a video. And much as I hate to do it, I do want people to associate my ideas with me! I welcome, very much, your feedback on this too!

Script:

Hi, I’m Clark Quinn, Executive Director of Quinnovation (my vehicle for learning experience design strategy).  Today I want to talk about an insight I had, sparked in a conversation I had with a colleague. We were talking about learning (of course), and the difference between knowledge versus practice.

I was reminded that we’re now seeing AI technologies that can parse content and then answer questions about it. We even see ones that can ask you questions about the content! Which is part of learning. But not all.

My realization was that, increasingly, these systems will take over this form of content presentation. That is, we’ll write a white paper, and an AI will parse it, then present it, and drill it. Which is, after all, way too much of corporate learning. See, for instance, the Serious eLearning Manifesto as a response.

Now, I’ve always maintained that such systems aren’t sufficient for real learning. Meaningful learning includes more: motivation and contextualized practice. Content presentation and quiz questions may be necessary, but by no means are they sufficient. And that for now and the foreseeable future, AI will not be able to create those elements. 

This is the job of LXD: integrating learning science and deep engagement into experiences that transform us. Which means that what L&D needs to do is stop doing information dump and knowledge test, and learn how to do real learning experience design! That, I suggest, is a noble pursuit (and, to be fair, what we should have been doing all along). 

Of course, there’s also the necessary new role, per my last post/video, of being a facilitator of informal learning. Coupling the optimal execution with continual innovation. But, for now, I’m suggesting we truly have to master everything that makes learning work, in particular meaningful practice.

That’s my take, I welcome hearing yours. Thanks for watching!

 

Flow, workflow, and learning

by 1 Comment

On LinkedIn, a colleague asked “Why do people think that integrating content in the flow of work equals learning in the flow of work?” An apt question. My (flip) response was “because marketing”. And I think there’s a lot to that. But, a comment prompted me to think a little bit deeper, because ‘flow’ is its own meaningful concept and we need to be careful about meaning. So here are some reflections on flow, workflow, and learning.

The response that triggered my reflection was:

I can’t recall the last time I told someone that I was in the “flow” of work today and learned so much!!

Flow state(Which is pretty funny!) The comment was a bit pointed, but it made me think about being in the ‘flow’ state, and the relationship with learning. I’ve previously pointed out how Csikszentmihalyi’s Flow and Vygotsky’s Zone of Proximal Development (ZoPD) are essentially the same. If the difficulty is too far above your skill level, the experience is frustrating. If it’s too easy, it’s boring. And in between is the flow state, and where learning happens.

Now, when we’re in the ‘flow’ at work (which is different than being in the workflow), we’re performing optimally. And I’m not sure learning happens there. Similarly with the ZoPD. You’re working and I’m not sure learning happens then. When I state that learning is action and reflection, I think reflection is a necessary component.

Now, the original complaint talked about learning in the workflow, and opined that content in the workflow won’t necessarily equal learning. Another comment pointed out what I believe is often conflated with “workflow learning”, and that’s performance support. There are lots of reasons that we might want content in the workflow to help us succeed, but it may have nothing to do with learning. If, indeed, learning is to happen, it might need some content, and feedback, and so actually break the flow!

Now, I also recognize that many times we’re in the flow of work, but not in the ‘flow’ zone. So, we could definitely be learning in the workflow. And it happens by deciding to look up the answer to some contextually relevant question. Or from a comment from a person. But it’s a bit different than being in the zone, and we’d like to be there in our work too!

And, I wonder whether Vygostky’s ZoPD really aligns with the Flow Zone, or if it needs to be coupled with some offline reflection. It’s certainly possible. Maybe the flow zone is a superset of the ZoPD. More to ponder.

There isn’t a real revelation here about flow, workflow, and learning, other than we have to keep our concepts straight. We need to recognize when we’re supporting performance, and when we’re learning. And we need to be clear about workflow, and being in the flow zone. And there may be more here to unpack. Thoughts?

 

In Defense of Science

by 1 Comment

I previously wrote in defense of cog psych. Here, I want to go broader. Not my usual topic, but… I feel the need to rail in defense of science.

When I’m talking about science, I’m talking about the systematic exploration of how things work. It’s about theorizing, and testing, and refining. It’s about having rigor in that, being systematic and principled. And, importantly, it’s about sharing the results and building on the works of others. Interestingly, it’s not a human universal, but instead emerged relatively recently. However, there’s a reason it’s been so successful.

And, let’s be honest, science has flaws. There have been recent problems in replicability. Another problem is academic politics; new ideas can struggle to get recognized. Pressure to publish can lead to fake data. Folks with money can influence what research gets done, and the outcomes. Similarly, politics can play a role. Like democracy, it’s not perfect, but…and this is an important but…there’s nothing better.

The evidence for science are the things that we’ve come to count on: sanitation, transportation, medicine, the list goes on. Your ability to read this depends on science. Most of what we use all day every day has been improved by science. So, too, some bad things, like how successful marketing can be at snagging your attention. Yes, we need to use it wisely. Science can help there, too! And, importantly, most scientists are ethical, caring, diligent individuals. They do what they do for science, not for wealth, not for fame (except amongst their colleagues, which goes back to doing it for science), and most certainly not to support conspiracies.

So, trying to pick and choose what science to believe isn’t a great bet. Unless you have a deep background in a particular domain, trying to ascertain the validity is challenging. You may listen to disparate voices, but not if they’re flying the face of a concerted viewpoint of people who have spent the requisite time to be true experts. In my mind, you’re either for science, or not. Saying “well, I’m not for this science because someone said it’s controversial”, then, is just not on.

Yes, there are controversies around most science: that’s how it advances. But there’s also essential truths that most every reputable scientist in the field will agree to. And that’s how we build products, services, and ultimately societies. I was trained as a scientist (though I’m more of an engineer, tracking it and applying it to solve real problems). I know, in my field, what makes sense and what’s silly. And then, in other fields, I look to what the received wisdom is. And I know what sorts of people to listen to, and it’s not politicians, or pundits. Unless they listen to science. The best guidance comes from the folks who know the field in question. And that holds true for medicine as well as meteorology.

And sure, I too could wish I lived in a world where magic worked. But if you think about it, they, too, use systematic experimentation to find out what works. Whether Earthsea or Hogwarts, they go to schools to learn and there the professors are studying. But here, magical thinking doesn’t work. Science is what has let us knock back polio, generate electricity from sunlight, and walk on the moon.

So, if you do want to go against what the scientists or reliable interpreters tell you, don’t do it piecemeal. Abandon all the science, because you’re unlikely to get it right in a domain that’s not your expertise. If anyone is telling you contrary to what’s known, question their motives! People mislead for lots of reasons, from money to mischief. If you let them, they’ll hurt you in ways that may be stark or subtle.

If they’re steering you away from something that has been shown to be better than the alternative, you should be wary. Their tricks are myriad: lack of context, distortions, selected subsets, and outright lying. For instance, our brains are wired to see patterns. If we’re pointed to them, we’ll see them. We’re also biased to look for evidence that confirms our beliefs, and avoid what contradicts it. Thus, it’s easy to gin up potential conspiracies, despite the incredible challenge in actually pulling them off!

I’m putting it out there. I can say that, in defense of science, it’s better than any other approach. That’s my stance, what’s yours?

 

Ritual

by 2 Comments

I’ve talked before about the power of ritual, but while powerful, it also seemed piecemeal. That is, there were lots of hints, but not a coherent theory. That has now changed. I recently found a paper by Nicholas Hobson & colleagues (Schroeder, Risen, Xylagatas, & Inzlicht; warning, PDF) titled The Psychology of Rituals that creates an integrated framework. And while my take simplifies it down, I found it interesting.

At core, what the model suggests is that there are two components that are linked together. The first element are things that involve the senses. The second element are the semantics we’re looking to create allegiance and adherence too. And there are important elements about this relationship.

There are a number of elements that are on tap for involving the senses. Certain movements, sounds, and words said or to be spoken can be used. There can also be food, drink, smells, and more. Objects also. Timing is an element; at the micro level of things in order, and at the macro level of the triggers for the ritual.

Semantics come, of course, from your needs. It can be about things you want people to believe, or a set of values you want people to subscribe to. Or, of course, both. From the design purpose, I’d suggest it’s about agreeing to be a member of a community of practice; to undertake certain actions when appropriate, and to uphold certain values.

Interestingly, according to their model, the relationship between the two is effectively arbitrary. That is, there is no intrinsic relationship between what you’re signifying, and how you do so. Rituals are about the practices. Which means you could in theory do just about anything to make the relationship.

The other thing is that the ritual has to be invariable in its aspects. You define it, and so do it. Note that the execution can vary considerably; from several times a day to upon certain triggering conditions. So, for instance, having completed a course, or before engaging in certain activities.

While such a definition gives us lots of freedom, it also doesn’t necessarily serve as a guide for design. Still, thinking about it in this way does suggest the utility in developing deeply held beliefs and appropriately practiced behaviors. At least, that’s how I see it. You?

What is wrong with (higher) education?

by 7 Comments

I was having a conversation with a colleague, sparked by dropping enrollments in unis. Not surprisingly, we ended up talking about flaws in higher education. He suggested that they don’t get it, and I agreed. He was thinking that they get the tech, but not the learning. I think it’s more complex. There are those that get some parts of the learning right. Just not enough, and not all of it right. Thinking further, post-convo, it occurs to me that there is a layer beneath the surface that matters. So I want to consider what is wrong with higher education.

And, let’s be clear, I’m not talking about the problems with tuition and administration. Yes, tuition’s risen faster than the cost of living. And yes, there’s little commercial pressure to keep universities free from the persistent creep of increasing administration. I saw an interesting article talking about how universities without a solid financial foundation, and ones without a good value proposition, will perish. It’s the latter I’m talking about.

I previously mentioned the three pillars I think create a valid learning offer:

  • a killer learning experience,
  • being a partner in your success
  • and developing you as an individual.

I suggest that all three are doable, but it occurred to me that there’s a bit more to unpack.

The ‘being a partner in your success’ bit is most frequently seen. Here, it’s about looking for signs of trouble and being proactive about reaching out and assisting. It’s not ‘sink or swim’, but recognizing there can be troubles and helping learners cope. The Predictive Analytics work that Ellen Wagner did is the type of opportunity we have here.

The ‘developing you as an individual’ is really building your more general skills: communicating, working with others, a positive attitude, knowing how to search, etc. And, of course, knowing how to continue to learn. Given the rate of change, most of what you learn as the core of a degree may well be out of date in short order!  But you can’t address these skills on their own, they’re specifically about how you do domain things.  And that’s a layer I’ve yet to see.

And the ‘killer learning experience’ is a second area where I think folks still aren’t doing well. My short (and admittedly cheeky) statement about education is that they’re wrong on two things, the curriculum and pedagogy, other than that they’re fine. Most universities aren’t doing a good job of curriculum, focusing on knowledge instead of skills. And some are moving in a good direction. Startups are addressing this area as well.

The other problem is the pedagogy. There’re two elements here: the learning design, and engagement. Too often, it’s still the ‘information dump and knowledge test’. But even when that’s right, making it truly meaningful for the learners is sadly neglected. Even professors who care often forget to put the ‘why’ into the syllabus.

In short, what is wrong with higher education is the ability to successfully execute on all these points. (It’s true for other education, too, but…) I’ve seen efforts that address one, or two (and plenty that get none right). However, as of yet, I have not seen anyone doing it the way it could be done. It’s doable, but not without some serious attention to not only the elements, but their successful integration. And it’s important enough that we should be doing it. At least, that’s what I think. So, what do you think?

Personalized and adaptive learning

by 1 Comment

For reasons that are unclear even to me, I was thinking about personalized versus adaptive learning. They’re similar in some ways, but also different. And a way to distinguish them occurred to me. It’s kinda simplistic, but I think it may help to differentiate personalized and adaptive learning.

As background, I led a project to build an intelligently adaptive learning platform. We were going to profile learners, but then also track their ongoing behavior. And, on this basis, we’d serve up something appropriate for learner X versus learner Y. (We’d actually recommend something, and they could make other choices.)

It was quite the research endeavor, actually, as the CEO had been inspired by Guilford’s learning model. I dug into that and all the learning styles literature, and cognitive factor analysis, and content models around learning objectives, and revisited my interest in intelligent tutoring, and more. I was able to hire a stellar team, and create an approach that was scientifically scrutable (e.g. no learning ‘styles’ :). We got it up and running before, well, 2001 happened and the Internet bubble burst and…

In some sense, the system was really both, in the way I’m thinking about it. I’ve seen different definitions, and one has adaptive as a subset of personalized, but I’m going a different way. I think of personalized as pre-planned alternatives for different groups, whereas adaptive reacts to the learner’s behavior.

Our use of initial profiling, if we only used that, would be personalized. The ongoing adaptation is what made it adaptive. We had rules that would prioritize preferences, but we’d also use behavior to update the learner model. It’s something they’re doing now, but we had it a couple of decades ago.

So, my simple way of thinking about personalized versus adaptive is that personalized is based upon who you are: your role, largely. We’d swap out examples on marketing for people selling services versus those selling products, for instance. Or if we’re talking negotiation, a vendor might get a different model than a lawyer.

Adaptive, on the other hand, is based upon what you do. So, for instance, if you did poorly on the last problem, we might not give you a more difficult one, but give you another at the same level. Twice in a row doing badly, we might bring you another example, or even revisit the concept. This is what intelligent tutoring systems do, they just tend to require a rigorous model of expertise.

Of course, you could get more complicated. Personalization might have a more and less supportive path, depending on your anxiety and confidence. Similarly, adaptive might throw in an encouraging remark while showing some remedial materials.

At any rate, that’s how I differentiate personalized and adaptive learning. Personalized is pre-set based upon some determined differences that suggest different learning paths. Adaptive calculates on the fly and changes what the learner sees next.  How do you see it?

Skills, competencies, and moving forward

by 3 Comments

I was asked, recently, about skills versus competencies. The context was an individual who saw orgs having competency frameworks, but only focusing on skill development. The question was where the focus should be. And I admit I had to look up the difference first! But then I could see where the emphasis should be on skills, competencies, and moving forward.

Now, the reason I joined with IBSTPI (the International Board for Standards in Training, Performance, and Instruction) was to learn more about competencies. So I didn’t feel inadequate looking it up (and probably should’ve asked my colleagues), but my search revealed a consistent viewpoint that kept me from having to bother them. The story was that there are individual skills, but that it takes more to do a job.

Competencies are suites of skills, knowledge, and attitudes* that create the ability to apply them in context to accomplish goals.  So you may be able to address customer objections, but there’s more to closing a sale than that. Competencies are aggregates of skills; they’re not just focused on what, but how. They’re a richer picture, based upon performance.

Should you care? It seems to me that you should. The clear implication is that if you only focus on skills, you may be missing other elements. You could develop skills and still not develop the ability to succeed. Thus, organizations are increasingly needing to focus on contextualized abilities to perform.

I’ll go further. In the days of optimizing performance, skills could potentially be sufficient. You knew what you had to do, and you had to do it. However, increasingly optimal execution is only the cost of entry, and continual innovation is the only sustainable differentiator. And that, I suggest, comes from competencies beyond skills.

Increasingly, you see orgs moving to competency-based hiring as well as development. Performance management likewise benefits from focusing on competencies.

Overall, my take is that when you’re looking at skills, competencies, and moving forward, competencies offer more power.

*”attitude” added based upon sound critique from Paul Kirschner.

Addressing fear in learning

by 4 Comments

One of my mantras in ‘make it meaningful‘ is that there’re three things to do. And one of those was kind of a toss away, until a comment in a conversation with a colleague brought it home. So here’s a first take at addressing fear in learning.

The mantra, to be clear, is that you have 3 major hurdles to overcome in getting someone to ‘buy in’ to learning:

  • I do need this
  • I don’t know it already
  • and I trust this experience will address that

I’ve focused mostly on the first, to date. The second is for the case where someone’s overconfident in their own abilities. However, the latter one was a toss-away, until…

My colleague mentioned how in trying to train data analysis, you could be coming up against decades of a belief such as ‘I don’t do well at math’. And I saw how you could have anxiety or a lack of confidence that this learning could address it.

Which makes it clear that you need to know the audience, and anticipate barriers. How can you address such a situation? I think you have to make sure that you make it steady and slow enough, or that it’s misperceived. So here, I could see either suggesting “we’ll take it slow enough and make it simple enough that you’ll find it easy” or “you may think data analytics is about math, but that’s the least part of it, it’s really about asking and answering questions”.

The point being, you need that trust, and that means addressing any barriers. It’s addressable, but you need to be aware. I also wonder if the typical elearning experience might have undermined trust such that there would have to be a series of successes to reestablish the trust that a learning unit should have. However, if we start regularly addressing all three, we have a start. That includes establishing the need, removing false conceptions, and addressing fear in learning. Those are my thoughts, what are yours?

Authentic Marketing

by Leave a Comment

I’m not a marketing expert, or even a marketer, so take the following with the proverbial boulder of salt. Still, I have to market Quinnovation, and I’ve advised orgs on marketing (learning) products, and I’ve taken down a lot of bogus marketing. So when something prompted me to reflect, I realized I had some thoughts on authentic marketing.

First, I’ve argued that good marketing is really good customer education. That is, you should be helping your customers understand why your product is the right thing for their needs. Of course, you should be first designing to ensure that it is the answer. Or, perhaps, an answer, and then helping your customers to understand if they’re the right customer for this solution.

And, when I’ve studied marketing for services, there are several steps that make sense. First, the clear thing is knowing your customer’s pain, and being able to articulate how you solve that pain. You want to help articulate clearly what the problem is, and what’s it’s costing, so that then you can suggest a solution and the benefits.

At core it’s about building up a solid, scrutable, case. Which is, in essence, building up trust that you know what you’re talking about, and that you can truly meet the need.

And that may not be the quick easy way. It appears to be the case that some folks would rather use clickbait-style advertising. Perhaps to cover up from not having a defensibly different product or solution? When there are hundreds of LMSs around, how do you differentiate yourself?  And I guess it works, because I keep finding new examples of marketing that goes for the cheap ploy rather than authentic education.

So I guess this is a plea for being an aware consumer. It’d be great if orgs started building products that really do make their customers awesome, and then use authentic marketing to sell them. In lieu of that, be wary. Look for unsubstantiated hype, buzzword bandwagon behavior, and style over substance. Know what you need, take your time to do due diligence, and spend wisely. Caveat emptor, after all.

The plusses and minuses of learning science research

by 1 Comment

A person who I find quite insightful (and occasionally inciteful ;) is Donald Clark. He built and sold Epic, an elearning company, and now he leads a learning AI company, Wildfire. He’s knowledgeable (for instance, having read up and summarized centuries of learning theorists), willing to call out bad learning, and he’s funny. And so, when he reported on a new study, I of course looked into it. And I find that it points out the plusses and minuses of learning science research.

To be clear, this is about his product, so there’s a vested interest. However, he’s got integrity; he’s not going to sully his reputation with a bad study. And, it’s a good study. It rightly demonstrates an important point. It’s just that it stops short of what we need for full learning.

So, his product does something pretty amazing. You give it content, and it can not only answer questions about the content (as, for instance, some chat tools do), it can turn the tables and ask you questions about the content. That is, it can serve as a sort of tutor. Which is all to the good.

What it can’t do, of course, is design meaningful practice. As Van Merriënboer’s Four Component Instructional Design (4C/ID) points out, you need to know the information, and you also need practice applying it. And I reckon we’re still far from that. So, while this is part of a whole solution (and Donald knows this), it’s not the full solution. He’s subsequently let me know it can do language tasks, which is impressive. I’m thinking more of contextualized scenarios, however.

The study demonstrates, as you might expect, that breaking up a video into reasonable chunks, and having system-generated questions asked in-between, led to 61% better retrieval, going from getting 8 to 14 questions right. That is a big improvement. it’s also impressive, since it’s generating those questions from video! That is, it parses the video, establishes a transcript, and then uses that to generate a knowledge base. Very cool.

And it’s a well-designed study. It’s got a control group, and a reasonable number of subjects. It uses the same test material, for an AB comparison. Presumably, the video chunking was done by hand, into four pieces. The chunking and break might account for the difference, which wasn’t controlled for, but it’s still a big improvement. Granted, we know that watching a video alone isn’t necessarily going to improve retention (except, perhaps, over some other non-interactive way of dumping content). But still, this is good as it’s an improvement and a lot of work was saved.

What I quibble about, however, is the nature of the retrieval. The types of questions liable to be asked (and it’s not indicated), are knowledge questions. As suggested above, knowledge is a necessary component. But using that knowledge to make decisions in context is typically what our goals are. And to achieve such goals, you basically have to practice making decisions in context. (Interestingly, the topic here was equality and diversity, a topic he has complained about!)

Knowledge about a topic isn’t likely to impact your ability to apply it. What will make a difference are actually doing things about it, like calling it out, having consequences, and actively working to remedy imbalances. And that requires separate practice. Which he’s acknowledged in the past, and rightly points out that his solution means you can devote more resources to that end.

Thus, the plusses of learning science research are we nibble away at the questions we need to answer, and find answers about the questions we ask. The minus, of course, is not necessarily asking the most important questions. It’d be easy to see this and say: “we’ve improved retention, and we’re done”. However, it won’t necessarily lead to reducing the behaviors being learned about, or building ability to deal with it. There are plusses and minuses of learning science research, and we need to know the strengths, and limitations, of it when we hear it.