Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Web 3.0 and system-generated content

by Leave a Comment

Not quite 15 years ago, I proposed that Web 3.0 would be system-generated content. There was talk about the semantic web, where we started tagging things, even auto-tagging, and then operating on chunks by rules connecting tags, not hard wiring. I think, however, that we’ve reached a new interpretation of Web 3.0 and system-generated content.

Back then, I postulated that Web 1.0 was producer-generated content. That is, the only folks who could put up content had all the skills. So, teams (or the rare individual) who could manage the writing, the technical specification, and the technical implementation. They had to put prose into html and then host it on the web with all the server requirements. These were the ones who controlled what was out there. Pages were static.

Then, CGIs came along, and folks could maintain state. This enabled some companies to make tools that could handle the backend, and so individuals could create. There were forms where you could type in content, and the system could handle posting it to the web (e.g. this blog!). So, most anyone could be a web creator. Social media emerged (with all the associated good and bad). This was Web 2.0, user-generated content.

I saw the next step as system-generated content. Here, I meant small chunks of (human-generated) content linked together on the fly by rules. This is, indeed, what we see in many sites. For instance, when you see recommendations, they’re based upon your actions and statistical inferences from a database of previous action. Rules pull up content descriptions by tags and present them together

There is another interpretation of Web 3.0, which is where systems are disaggregated. So, your content isn’t hosted in one place, but is distributed (c.f. Mastodon or blockchain. Here, the content and content creation are not under the control of one provider. This disaggregation undermines unified control, really a political issue with a technical solution.

However, we now see a new form of system-generated content. I’ll be clear, this isn’t what I foresaw (though, post-hoc, it could be inferred). That is, generative AI is taking semantics to a new level. It’s generating content based upon previous content. That’s different than what I meant, but it is an extension. It has positives and negatives, as did the previous approaches.

Ethics, ultimately, plays a role in how these play out. As they say, Powerpoint doesn’t kill people, bad design does. So, too, with these technologies. While I laud exploration, I also champion keeping experimentation in check. That is, nicely sandboxing such experimentation until we understand it and can have appropriate safe-guards in place. As it is, we don’t yet understand the copyright implications, for one. I note that this blog was contributing to Google C4 (according to a tool I can no longer find), for instance. Also, everyone using ChatGPT 3 has to assume that their queries are data.

I think we’re seeing system-generated content in a very new way. It’s exciting in terms of work automation, and scary in terms of the trustworthiness of the output. I’m erring on the side of not using such tools, for now. I’m fortunate that I work in a place of people paying me for my expertise. Thus, I will continue to rely on my own interpretation of what others say, not on an aggregation tool. Of course, people could generate stuff and say it’s from me; that’s Web 3.0 and system-generated content. Do be careful out there!

Skills to move into L&D?

by 2 Comments

Our colleague, Connie Malamed, posed a question on Twitter. She asked: “What areas and skills are essential (in order of importance if possible) for those who want to transition into learning and development.”  So, I used some of my tiny brain and generated an initial list. I’m sharing it for her (I responded with just the categories on Twitter), and for discussion. I welcome your thoughts on skills to move into L&D.

The first category I thought of was the basics of cognitive science: perception, consciousness & context, elaboration, retrieval, etc. I think (strongly) that if you know the basics of the Human Information Processing (HIP) loop, you have a foundation to understand so much more. Not just instructional design, but basically designing for people: graphic, interface, marketing, etc design. I think anyone who designs for people ought to know it. I’m surprised, frankly, that it’s not more central.

From there, the next step, and this is where we get L&D-specific, is learning science. Here I mean the role of models, examples, emotion, etc. These are phenomena that emerge from our HIP system, but are unique to learning (there are others that aren’t as specific to our roles). This is what we have to understand so that we can understand the precepts of learning experience design (LXD).

I think that in L&D, you also have to understand the process of design itself. Particularly the things that are our adaptions to the flaws in our cognitive architecture, but as learners and as designers. Here I’m talking about things like the need for iteration and formative and summative evaluation, (proper) brainstorming, etc. For instance, I got a lot of mileage out of bringing what was known from interface design to instructional design; that field was much more astute about design than was ours at the time.

Integrating the above is LXD, which I think we ought to know. That is, taking the principles above about design and learning and turning that into a process that reliably yields experiences that develop real performance. If we’re designing solutions, we need to know the science and the engineering thereof.

There are also some additional fields that have a big impact on the ability to be successful in L&D in the real world. Thus, you need things like good project management (scheduling, resource allocation, risk management, etc), for one. Even if assigned a project manager, you still need to be personally effective. Which incorporates, I suppose, time management similarly, effectively using calendaring, ‘todo’ lists, etc. There’s also working with others: being a good teammate, knowing how to work with SMEs and other stakeholders, etc. You also need to know the basics of business: planning, budgeting, strategy, also things like supply/demand, and change management in particular (interventions are org changes, and need to be treated as same).

Connie also said: “I have an opinion and would like to hear yours. ” So, here’ve been my thoughts on the skills to move into L&D. What are yours?

Two steps for L&D

by Leave a Comment

In a conversation, we were discussing how L&D fares. Badly, of course, but we were looking at why. One of the problems is that L&D folks don’t have credibility. Another was that they don’t measure. I didn’t raise it in the conversation, but it’s come up before that they’re also not being strategic. That came up in another conversation. Overall, there are two steps for L&D to really make an impact on.

Now, I joke that L&D isn’t doing well what it’s supposed to be doing, and isn’t doing enough. My first complaint is that we’re not doing a good job. In the second conversation, up-skilling came up as an important trend. My take is that it’s all well and good to want to do it, but if you really want persistent new skill development, you have to do it right! That is, shooting for retention and transfer. Which will be, by the way, the topic of my presentation at DevLearn this year, I’ve just found out. Also the topic of the Missing LXD workshop (coming in Asia Pacific times this July/Aug), in linking that learning science grounding to engagement as well.

I’ve argued that the most important thing L&D can do is start measuring, because it will point out what works (and doesn’t). That’s a barrier that came up in the first conversation; how do we move people forward in their measurements. We were talking about little steps; if they’re doing learner surveys (c.f. Thalheimer), let’s encourage them to move to survey some time after. If they’re doing that, let’s also have them ask supervisors. Etc.

So, this is a necessary step. It’s not enough, of course. You might throw courses at things where they don’t make sense, e.g. where performance support would work better. Measurement should tell you that, in that a course isn’t working, but it won’t necessarily point you directly to performance support. Still, measurement is a step along the way. There’s another step, however.

The second thing I argue we should do is start looking at going beyond courses. Not just performance support, but here I’m talking about informal and social learning, e.g. innovation. There are both principled and practical reasons for this. The principled reason is that innovation is learning; you don’t know the answer when you start. Thus, knowing how learning works provides a good basis for assisting here. The practical reason is it gives a way for L&D to contribute to the most important part of organizational success. Instead of being an appendage that can be cut when times are tough, L&D can be facilitating the survival and thrival strategies that will keep the organization agile.

Of course, we’re running a workshop on this as well. I’m not touting it because it’s on offer, I’m behind it because it’s something I’ve organized specifically because it’s so important! We’ll cover the gamut, from individual learning skills, to team, and organizational success. We’ll also cover strategy. Importantly, we have some of the best people in the world to assist! I’ve managed to convince  Harold Jarche, Emma Weber, Kat Koppett, and Mark Britz (each of which alone would be worth the price of entry!), on top of myself and Matt Richter. Because it’s the Learning Development Accelerator, it will be evidence-based. It’ll also be interactive, and practically focused.

Look, there are lots of things you can do. There are some things you should do. There are two steps for L&D to do, and you have the opportunity to get on top of each. You can do it any way you want, of course, but please, please start making these moves!

A placebo effect?

by 1 Comment

I was thinking about what too often we see as elearning. That is, the usual content dump and knowledge test. There’s good reason to believe that it isn’t effective. So, why are we seeing it continue? Is it a placebo effect?

I tend to view this as a superstition. That is, the belief that information presentation will lead to behavior change is held implicitly. I think it originates from a legacy perspective that we’re logical, and therefore new information will yield impact. (Not.) Regardless, it exists.

I was inclined to wonder if, really, it’s a placebo. That is, doing something with a hope that things change, but the onus is on the individual, not the intervention. There’s not going to be any actual effect, but it makes people feel better. Of course, the role is different here; the placebo makes the doctor feel better! (Or the health system? I’m muddling my metaphor…:)

It may not be that in practice, of course. There is a ‘faith’ that “if we build it, it is good”. So, biz units can ask for a course, and get one. They’ve provided content and access to SMEs. However, they push back when asked “what’s the actual problem”, let alone asked for measures.  It’s like they think the job can be done with information. They are happy with the appearance of a solution, because it’s easy, and no one’s checking.

We, of course, have to change this perception. If we continue to let folks believe they can give us content and we’ll deliver meaningful change, shame on us. Of course they don’t care about the measures, and they want things to be easy. We have to, however.  It may as well be a placebo effect, because the ultimate impact is a likely null as sugar pills, unless the patient wants to change. It’s probably not a great metaphor, but somehow it still seems apt. Thoughts?

The Role of a Storyboard?

by 1 Comment

In a recent conversation, the issue of storyboards came up. It was in the larger context of representations for content development. In particular, for communicating with stakeholders (read: clients ;). The concern was how do you communicate the content versus representing the experience. So, the open question is what is the role of a storyboard?

So, there are (at least) several elements that are important in creating a learning experience. One is, of course, the domain. What are the performance objectives? Moreover, how are you providing content support. Specifically, what models are you using, and what examples. Then, of course, there’s the practice. Ideally, practice aligns with the performance objectives, and the models and examples support the practice.

There’s also the experience flow. How are you hooking learners up front? We are concerned with balancing the quantity of content with the actual practice, keeping engagement throughout.

In both cases, we need to communicate these to the clients. Too often, of course, clients raise concerns about making sure ‘the content’ is covered. In many cases, they really don’t understand that less is better, and have a large amount they’ve heard from subject matter experts (SMEs). Not knowing, of course, that SMEs have access to what they know about the domain, but less about what they do! Thus, they’re looking to make sure the content’s covered.

There will also be concern about the experience. This, likely not ideally, comes after assurance about the content. I personally have experienced situations where stakeholders say ‘ok’ to a storyboard, but then balk at the resulting experience. Some (surprisingly high) proportion of folks can’t infer an experience from a storyboard.  This has been echoed by others.

The question is what is the role of a storyboard. In game design, there is a (dynamic) design document that captures everything as it develops. Is this the right representation to communicate the experience? It  communicates to developers, but is it good for clients? I argue that we want more iterative representations, for instance getting sign-off on what we’ve heard from the analysis and documenting what will be the focus of the design. We also want to separate out the domain from the experience.

Overall, I advocate representing the experience, for instance in (mocked up) screenshots with narration to represent a sample interaction. That can accompany the storyboard, but when folks have to sign-off on an experience, and they can’t get it from the usual representations, you’ll need an augment. I wonder whether we should fight against presenting the content that’ll be covered.

We should show the objectives, models, and examples, but fight against content ‘coverage’. Cathy Moore does a good job in her ‘Action Mapping’ to argue that what the minimum is to achieve success on appropriate performance tasks is a good goal. I agree, as does learning science. The role of a storyboard is to capture development for developers. It may not be the right communication tool for stakeholders. I welcome your thoughts.

Grounded in practice

by Leave a Comment

Many years ago, I was accused of not knowing the realities of learning design. It’s true that I’ve been in many ways a theorist, following what research tells us, and having been an academic. I also have designed solutions, designed design processes, and advised orgs. Still, it’s nice to be grounded in practice, and I’ve had the opportunity of late.

So, as you read this, I’m in India (hopefully ;), working with Upside Learning. I joined them around 6 months ago to serve as their Chief Learning Strategist (on top of my work as Quinnovation, as co-director of the Learning Development Accelerator, and as advisor to Elevator9). They have a willingness to pay serious attention to learning science, which as you might imagine, I found attractive!

It’s been a lot of marketing: writing position papers and such. The good news is it’s also been about practice. For one, I’ve been running workshops for their team (such as the Missing LXD workshop with the LDA coming up in Asia-friendly times this summer). We’ve also created some demos (coming soon to a sales preso near you ;). I’ve also learned a bit about their clients and usual expectations.

It’s the latter that’s inspiring. How do we bake learning science into a practical process that clients can comprehend? We’re working on it. So far, it seems like it’s a mix of awareness, policy, and tools. That is, the design team must understand the principles in practice, there need to be policy adjustments to support the necessary steps, and the tools should support the practice. I’m hoping we have a chance to put some serious work into these in my visit.

Still, it’s already been eye-opening to see the realities organizations face in their L&D roles. It only inspires me more to fight for the changes in L&D that can address this. We have lots to offer orgs, but only if we move out of our comfort zone and start making changes. Here’s to the revolution L&D needs to have!

 

Curriculimb

by 3 Comments

Ok, so I’m going to go out on a limb here, and talk a wee bit about what I’ve been learning about designing curricula. I care about doing it right (and probably haven’t always). It’s not the average course that’s the issue, but big ones, or multiple courses addressing skill gaps. It’s been challenging to find a systematic approach, which is why I’m teetering on a curriculimb.

So, the issue is how to develop a curriculum. I know in higher ed (I was there once) it tends to be a process of figuring out what content they need, and distributing across courses. It’s probably more art than science, where you move stuff around until it feels like you’ve got the right sized amount of content for each subject and it covers the ‘right stuff’. How people meet the criteria can vary. In a more research institution, I could design my HCI course my way. In more teaching-focused institutions, people may actually be given course syllabi to teach to!

My problem is when I have an uncertain amount of content, say for a large domain, and I want to develop specific capabilities. On principle, we should work backwards from the final performance. Which might include some very rich types of capabilities, so we might have a lot of concepts and practice involved. We’d need to create a large map. We might even break it up into conceptual stages (e.g. with programming: learning conditionals and then loops), and addressing them separately.

You probably also need to provide some practice to deal with misconceptions. That is, where are folks likely to get off track and maybe discouraged? Then you want to create practice for that. The things you’d rather they learned before it matters.

When I looked for good principles around this, it seemed like most of what I found basically said it’s iterative, there are no overarching principles (except work backwards and iterate ;). Which was less than satisfying, and some evidence-based practice would be nice.

Now, one of the things I was pondering in the dark of the night was how AI could help. I’ve been hearing how it can parse content and create maps. However, I also realized that to do so, it needs well-structured content. Kind of a circular argument. I think we need people to define it then AI can align it.

Again, right now it seems more like an art than a science. And I get that; it’s a lot like designing in engagement: create a first best guess and then test. Still, there are some solid results in engagement that give us some grounds for the first pass. I feel less like that at the next level up. So, I’m out on a curriculimb, and welcome help getting down!

Attention is underrated

by 1 Comment

Attention is a complex phenomena. Thinking that we can simply address is probably naive. Worse, there is at least one pervasive myth about it. Trivial attention is probably overrated, but meaningful attention is underrated.

Attention, I’ll suggest, is how we pay conscious awareness to our thinking. We pay attention to the sensory stream that’s available, and as working memory is has limits, our attention chooses what ends up being in working memory (which is where we see conscious thought). This is the picture I paint in Learning Science for Instructional Designers,  my recent book on how we learn. That’s how I learned it in grad school, and little seemed to change that.

As an aside, I suggest that the basic human information processing loop is something that is critical to understand. This is true for learning designers, but I would suggest there’s broader applicability. Knowing how information flows:

  • from sensory store to working memory via attention
  • from working memory to long term memory via elaboration
  • back to working memory via retrieval
  • and to decision from working memory

as a simplified story, shows how humans work in many ways. It gets more complex in important ways, but this is a key basis. On top of it comes aspects of how we think, and learn, but this is the core.  It benefits anyone dealing with people, basically: UI, marketing, etc. In short, most everyone.

Recent pictures of the information processing loop suggest, however, that attention has a bigger purview. They have it influencing most of the above. Which may be more accurate, in that if you need to attend to what’s in working memory, and manage the process of attending to information while evaluating what decision to make. You must maintain conscious focus on what you want to learn.

The myth, which still persists, is that our attention span has dropped to 8 seconds. Which folks tout as less than that of a goldfish. (How do we know what the attention span of a goldfish is?) The origin of this myth came from StatBrain misinterpreting a study, and was amplified since it was published by Microsoft Canada.  Marketing, mind you, not their research group! A myth I busted in a previous book!

There is apparently some evidence that our attention span has dropped (to 4o-something seconds, not eight), but we can still disappear into movies, novels, and games for hours. I reckon it’s about how engaging it is. Which, not completely surprisingly, is the topic of my most recent book, Make It Meaningful.

So, please, avoid the myths, and learn the core. Attention is underrated, as is the whole human information processing loop. Learn it, and benefit.

Misleading Malarkey

by 2 Comments

Recently, I saw a claim that was, well, a tad extreme. Worse, I think it was wrong, and possibly harmful. Thus, I feel it’s right to address it, to avoid misleading malarkey.

So, here’s the claim that riled me up:

Short-form edutainment is the most effective teaching method for both children and adults. TikTok and YouTube shorts will ultimately replace high schools and universities. Employment sector will phase out LMS systems and replaced with AI-powered compliance tools. If you are considering instructional design as a career, you may want to become a YouTuber or TikToker instead.

If you’ve tuned in at all, you’ll know that I’m a fan of engagement, properly construed.  Heck, it’s the topic of my most recent book! So, talking about the value of engagement in learning is all to the good. However…

…this claim goes over the top. Most notably, there’s the claim that edutainment is the most effective teaching method. If only! That puts me off, because teaching should yield a learning outcome, and just watching video shorts won’t do that (under most circumstances). Not surprisingly, I asked for research.

The author pointed to a study where mice genetically low on dopamine learned better when given dopamine. Yes, but the study had the mice do more than just watch videos, they performed tasks! I tried to go deeper, saying that engagement may be desirable, but it’s not sufficient. Without practice, watching entertaining and informative material (e.g. edutainment) isn’t a path to learning outcomes.

The conversation was derailed by my comment that edutainment had gotten a bad name from games. In the 80s, in an industry I was in, this was the case! I was accused of having a ‘gamification’ mindset! (Ahem.)  I tried steering the conversation back to the point it’s not about gamification, it’s about engagement combined with practice.

Interestingly, there was an almost parallel conversation about how engagement wasn’t the same as learning (which I pointed to in the exchange). The general take is that engagement is desirable but insufficient. Yes! Yet here we see the claim that engagement is all we need!

I believe in engagement for learning. I just don’t believe that by itself it will lead to learning. Learning science supports both the value of engagement, and the necessity of practice and feedback. That’s all. But claims like the above are misleading malarkey. It may be we’re talking an outrageous marketing claim (infamy is better than not being known at all?), but when it misleads, it’s a problem. Am I missing something?

Missing LXD Workshop

by Leave a Comment

We interrupt your regularly scheduled reading for this commercial announcement:

What is Learning Experience Design (LXD)? Further, why should you care? Finally, (and arguably most important) what does it mean you should do differently? Those, to me, are important questions. My short answer is that LXD is the elegant integration of learning science and engagement. Which, to me, implies some important nuances on top of what’s traditionally done in instructional design (ID). How to address it? There’s actually quite a lot in LXD, but it’s also a lot of overlap with traditional ID practices and processes. I reckon the easiest (and best) way to address it is to talk about the delta. That is, what’s different between the two. So, in my role for Upside Learning, I developed a missing LXD workshop. We ran it internally to good outcomes, and now, you can take it!

I believe that the difference starts with objectives; you can’t make a meaningful experience if you don’t have learners acquiring relevant new skills (not just an information dump). From there, there are nuances on designing individual practice activities, and then aggregated into practices (that is, putting practices together). Moving on, we look at the content elements of models and examples, and then the emotional aspects of learning. The workshop closes by looking at a design process that accommodates these. Recognizing that folks don’t want to throw out their whole process to start anew, it works from a generic model.

In the workshop, I cover each of those topics in a week; so it’s a six week experience. In between, I ask attendees to do some interim processing to both cement their understanding and to change their practices. Each week we’ll cover underlying concepts, see examples of what we’re talking about, actively process the information, and do a major application task.

To make this available more broadly, Upside’s partnered with the Learning Development Accelerator (LDA) to deliver it. Full disclosure: I’m co-director of the LDA, and Chief Learning Strategist for Upside Learning (in addition to my ongoing role for Quinnovation). (So, it’s all about me! :) Seriously, I think this puts together the tools I believe are necessary to lift our industry.

To be clear, since the advance notice timeframe puts this in summer, we’re offering it in Asia time-frames first (tho’ anyone is welcome!):

Australian Eastern Standard Time: July 7, 14, 21, 28, August 4 and 11 from 12h00 to 14h00 each day
Singapore Time: July 7, 14, 21, 28, August 4 and 11 from 10h00 to 12h00 each day
India Standard Time: July 7, 14, 21, 28, August 4 and 11 from 07h30 to 09h30 each day
New York Time: July 6, 13, 20, 27, August 3 and 10 from 22h00 to 24h00 each day

We’re offering it for US$100 to LDA members, and US$350 to non-members (for only $40 more, you get the full LDA offerings as well).

We’re planning to offer the missing LXD workshop again at a later date at East Coast/Europe friendly times (probably at a steeper price, we’ll have worked the bugs out ;). You can find out more at the LDA site. It’s got learning science and engagement bundled up into a coherent whole, for those who’ve already been doing ID and want to lift their game. I hope you’ll find it worth your while.

We now return you to your regularly scheduled reading until next week at the usual time.