Learnlets

Secondary

Clark Quinn’s Learnings about Learning

You are here: Home / Archives for design

Getting brainstorming wrong

by Leave a Comment

There’s a time when someone takes a result, doesn’t put it into context, and leads you to bad information. And we have to call it out. In this case, someone opined about a common misconception in regards to brainstorming. This person cited a scientific study to buttress an argument about how such a process should go. However, the approach cited in the study was narrower than what brainstorming could and should be. As a consequence, the article gave what I consider to be bad information. And that’s a problem.

Brainstorming

Brainstorming, to be fair, has many interpretations.  The original brought people into a room, had them generate ideas, and evaluate them.  However, as I wrote elsewhere, we now have better models of brainstorming. The most important thing is to get everyone to consider the issue independently, before sharing. This taps into the benefits of diversity. You should have identified the criteria of the problem to be addressed or outcome you’re looking for.

Then, you share, and still refrain from evaluation, looking for ideas sparked from the combinations of two individual ideas, extending them (even illogically). the goal here is to ensure you explore the full space of possibilities. The point here is to diverge.

Finally, you get critical and evaluate the ideas. Your goal is to converge on one or several that you’re going to test. Here, you’re looking to surface the best option under the relevant criteria. You should be testing against the initial criteria.

Bad Advice

So, where did this other article go wrong? The premise what that the idea of ‘no bad ideas’ wasn’t valid. They cited a study where groups were given one of three instructions before addressing a problem: not to criticize, free to debate and criticize, or no instructions.  The groups with instructions did better, but the criticize group were. best.  And that’s ok, because this wasn’t an optimal brainstorming design.

What the group with debate and criticizing were actually tasked with doing most of the whole process: freewheeling debate and evaluation, diverging and converging. The second instruction group was just diverging. But, if you’re doing it all at once, you’re not getting the benefit of each stage! They were all missing the independent step, the freewheeling didn’t have evaluation, and the combined freewheeling and criticizing group wouldn’t get the best of either.

This simplistic interpretation of the research misses the nuances of brainstorming, and ends up giving bad advice. Ok, if the folks doing the brainstorming in orgs are violating the premise of the stages, it is good advice, but why would you do suboptimal brainstorming?  It might take a tiny bit longer, but it’s not a big issue, and the outputs are likely to be better.

Doing better

We can, and should, recognize the right context to begin with, and interpret research in that context. Taking an under-informed view can lead you to misinterpret research, and consequently lead you to bad prescriptions.  I’m sure this article gave this person and, by association, the patina of knowing what they’re talking about. They’re citing research, after all!  But if you unpack it, the veneer falls off and it’s unhelpful at the core. And it’s important to be able to dig deep enough to really know what’s going on.

I implore you to turn a jaundiced eye to information that doesn’t come from someone with some real time in the trenches. We need good research translators. I’ve a list of trustworthy sources on the resources page of my book on myths. Tread carefully in the world of self-promoting media, and you’ll be less hampered by the mud ;).

The wisdom of instruction

by Leave a Comment

I was listening in to a webinar on trends in higher education. The speakers had been looking at different higher ed pedagogy models, within and external to institutions. It became clear that there was a significant gap between a focus on meeting corporate needs and the original goals of education. Naturally, it got me to think, and one link was, not surprisingly, wisdom. So what does that mean?

In the ‘code academy’ models that are currently challenging to higher education, there’s very much a ‘career’ focus. That is, they’re equipping students to be ready to take jobs.  Which is understandable, but there’s a gap. A not-for-profit initiative I was involved with wanted to get folks a meaningful job. My point was that I didn’t want them to get a job, I wanted them to keep a job!  And that means also learning about learning to learn skills, and more. That more is where we make a substantial shift.

The shift I want to think about is not just what corporations need, but what society needs. The original role of institutions like Oxford and Harvard was to create the next generation leaders of society. That is, to give the philosophical (in the broad sense) and historical perspective to let them do thinking like what delivered the US Constitution (as an example). And there’s plenty of lip service to this, but little impact. For example, look at the success of teaching ethics separately from other business classes…let’s move on.

It seems like there’s several things we need to integrate. As pointed out, treating them separately doesn’t work. So how do we integrate them and make them relevant.  Let’s take Sternberg’s model of Wisdom, where you think about decisions:

  • for the short term and long term
  • for you, yours, and society as a whole
  • and also explicitly discuss the value assumptions underpinning the decision.

This gives us a handle. We need to find ways to naturally embed these elements into our tasks. Our tasks need to require 21C skills and understanding the societal context as well.

In my ‘application-based instruction’ model, I talk about giving learners challenges that do require 21 C skills in natural ways. In this model, tasks mimic world tasks, asking for things like presentations, RFPs, problem recommendations, and more.  Then, how do we also include the societal aspects?  I suppose by putting those decisions in situations where there are implications not just for the business but for society.

Ok, it may be too much to layer this on every assignment (major assignment, not the accompanying knowledge check), but it should be covered in every subject (yes, even introductory) in some way. This thinking has already led me to create a question on evaluating policy tradeoffs for the mobile course I’m developing.

We need to keep the societal implications involved. Ensuring that at least a subset of the assignments do that is one approach. Doing so in a natural way requires some extra thinking, but the consequences are better. Particularly if the instructor actually makes a point of it (making a note to myself…).  A separate course doesn’t do it. So let’s get wise, and develop in deeper ways that will deliver better outcomes in the domain, and for the greater good. Shall we?

What to evaluate?

by 4 Comments

In a couple of articles, the notion that we should be measuring our impact on the business is called out. And being one who says just that, I feel obligated to respond.  So let’s get clear on what I’m saying and why.  It’s about what to evaluate, why, and possibly when.

So, in the original article, by my colleague Will Thalheimer, he calls the claim that we should focus on business impact ‘dangerous’!  To be fair (I know Will, and we had a comment exchange), he’s saying that there are important metrics we should be paying attention to about what we do and how we do it. And no argument!  Of course we have to be professional in what we do.  The claim isn’t that the business measure is all we need to pay attention to. And he acknowledges that later. Further, he does say we need to avoid what he calls ‘vanity metrics’, just how efficient we are. And I think we do need to look at efficiency, but only after we know we’re doing something worthwhile.

The second article is a bit more off kilter. It seems to ignore the value of business metrics all together. It talks about competencies and audience, but not impacting the business. Again, the author raises the importance of being professional, but still seems to be in the ‘if we do good design, it is good’, without seeming to even check to see if the design is addressing something real.

Why does this matter?  Partly because, empirically, what the profession measures are what Will called ‘vanity’ measures. I put it another way: they’re efficiency metrics. How much per seat per hour? How many people are served per L&D employee?  And what do we compare these to?  Industry benchmarks. And I’m not saying these aren’t important, ultimately. Yes, we should be frugal with our resources. We even should ultimately ensure that the cost to improve isn’t more than the problem costs!  But…

The big problem is that we’ve no idea if that butt in that seat for that hour is doing any good for the org.  We don’t know if the competency is a gap that means the org isn’t succeeding!  I’m saying we need to focus on the business imperatives because we aren’t!

And then, yes, let’s focus on whether our learning interventions are good. Do we have the best practice, the least amount of content and it’s good, etc. Then we can ask if we’re efficient. But if we only measure efficiency, we end up taking PDFs and PPTs and throwing them up on the screen. If we’re lucky, with a quiz. And this is not going to have an impact.

So I’m advocating the focus on business metrics because that’s part of a performance consulting process to create meaningful impacts. Not in lieu of the stuff Will and the other author are advocating, but in addition. It’s all too easy to worry about good design, and miss that there’s no meaningful impact.

Our business partners will not be impressed if we’re designing efficient, and even effective learning, if it isn’t doing anything.  Our solutions need to be targeted at a real problem and address it. That’s why I’ll continue to say things like “As a discipline, we must look at the metrics that really matter… not to us but to the business we serve.”  Then we also need to be professional. Will’s right that we don’t do enough to assure our effectiveness, and only focus on efficiency. But it takes it all, impact + effectiveness + efficiency, and I think it’s dangerous to say otherwise.  So what say you?

Redesigning Learning Design

by 2 Comments

Of late, a lot of my work has been designing learning design. Helping orgs transition their existing design processes to ones that will actually have an impact. That is, someone’s got a learning design process, but they want to improve it. One idea, of course, is to replace it with some validated design process. Another approach, much less disruptive, is to find opportunities to fine tune the design. The idea is to find the minimal set of changes that will yield the maximal benefit. So what are the likely inflection points?  Where am I finding those spots for redesigning?  It’s about good learning.

Starting at the top, one place where organizations go wrong right off the bat is the initial analysis for a course. There’s the ‘give us a course on this’, but even if there’s a decent analysis the process can go awry. Side-stepping the big issue of performance consulting (do a reality check: is this truly a case for a course), we get into working to create the objectives. It’s about how you work with SMEs. Understanding what they can, and can’t, do well means you have the opportunity to ensure that you get the right objectives to design to.

From there, the most meaningful and valuable step is to focus on the practice. What are you having learners do, and how can you change that?  Helping your designers switch to good assessment writing is going to be useful. It’s nuanced, so the questions don’t seem that different from typical ones, but they’re much more focused for success.

Of course, to support good application of the content to develop abilities, you need the right content!  Again, getting designers to understand what the nuances of useful examples from just stories isn’t hard but rarely done. Similarly knowing why you want models and not just presentations about the concept isn’t fully realized.

Of course, making it an emotionally compelling experience has learning impact as well. Yet too often we see the elements just juxtaposed instead of integrated. There are systematic ways to align the engagement and the learning, but they’re not understood.

A final note is knowing when to have someone work alone, and when some collaboration will help.  It’s not a lot, but unless it happens at the right time (or happens at all) can have a valuable contribution to the quality of the outcome.

I’ve provided many resources about better learning design, from my 7 step program white paper to my deeper elearning series for Learnnovators.  And I’ve a white paper about redesigning as well. And, of course, if you’re interested in doing this organizationally, I’d welcome hearing from you!

One other resource will be my upcoming workshop at the Learning Solutions conference on March 25 in Orlando, where we’ll spend a day working on learning experience design, integrating engagement and learning science.  Of course, you’ll be responsible for taking the learnings back to your learning process, but you’ll have the ammunition for redesigning.  I’d welcome seeing you there!

Bringing Transformation to Life

by 1 Comment

I’m going to be delivering a mobile learning course for a university this spring. Consequently, I’m currently beginning the design. I need to practice what I preach in the sense of good learning design, so I’m working through the usual decisions. The real question I have is whether I can make it transformative. There are limitations, but one of my mantras is about design having many possible development forms. So…I’ve got to make a good stab at it. Here’s my preliminary thinking on bringing the course to life.

I’ve actually been working a lot on the design of university learning experiences. There have been several instances in the past year or two that have really pushed my thinking in this space. This naturally includes application-based instruction, as well as meaningful (and minimal) content, and good assessment design. It’s been handy for this!

Naturally, Designing mLearning will be the text (The Mobile Academy is focused on formal education, and these students are focused on the workplace).  As it turns out, the book isn’t written in the order I want to deliver the course. So while I’ll have specific readings each week, I’ll instead recommend that the students read it in one go (it’s not a long book), and that two different cuts through the material will be a better learning experience.

I’m also thinking about the assignments: having them do meaningful things. E.g. designing solutions. Also, in the right order, to facilitate useful processing. I suppose I should worry about whether the workload will be too much However, as it’s a compressed course the expectations per week are higher. And I will argue that I’m having them do more than consume, so the workload’s ok.

The important thing, to me, is getting the emotional trajectory right. I reckon I need an aspirational goal up front, and then ensure I deliver.  It needs to work on a week-by-week basis and overall.  I’m making sure they’re doing the right things, and then I’ll fine-tune. I like the chance to integrate my thinking and put a stake in the ground. It’ll be interesting to see how it goes.

As a side note, mobile seems (to me) to be resurrecting. I would think it’s now mainstream, but some folks are still getting started. Hey, it makes sense, so better late than never! Where are you in mobile?

A foolish inconsistency

by Leave a Comment

Here, a foolish inconsistency is the hobgoblin of my little mind. While there are some learnings in here (for me and others), it’s really just getting stuff off my chest. Feel free to move along. This is just a lack of consistency that I suggest is unnecessary and ill-conceived.

I’ve hinted at this before, but I don’t think I’ve gone into detail. I like LinkedIn. It’s a useful augment for business networking. However, what drives me nuts is the inconsistency between the device app and the web interface.  One instance is sufficient: messaging someone you’ve just connected to.

So, on the device, if you link to someone, you immediately get a notice and a link to send them a message. And I like that, since I like to send a quick followup to everyone I link to (a trick I learned from a colleague).  On the device, it goes straight to the messaging interface. Perfect. Now, from the invitations on the app that I want to query (e.g. it’s not clear why they’ve linked) or to explain why I won’t (I generally don’t link to orgs, for instance), I can’t do that, but that’s ok, it can wait ’til I’m on my laptop using the (richer) web app.

On the web version, when I accept a link, I’m also offered the chance to message them, but here’s the trick: it’s not a message, it’s an InMail!  And, of course, those are limited. I don’t want to use my InMails on messaging someone I’m already linked to.  (I don’t use them in general, but that’s a separate issue.).  WHY can’t they go to messages like the app?  That’d be consistent, and this is a worse default than using messages.  I get that the app would have more limited functionality in return for being an app (there’re benefits, like notifications), but why would the full web version do things that are contrary to your interests and intentions?!?!

Good design says consistency is a good thing, generally; certainly aligning with user expectations and best interests. It’s bad design to do something that’s unnecessarily wasteful.  There are lots of such irritations: web forms that only tell you the expected format after you get it wrong instead of making it easy to point to the answer or give you a clue and sites with mismatched security (overly complex for unessential data or vice-versa) are just two examples.  This one, however, continues to be in my face regularly.

This inconsistency is instead a hobgoblin of a sensible mind. Has this irritated you, or what other silly designs bedevil you?

 

Fun, Hard Fun, & Engagement

by 3 Comments

At Online Educa in Berlin, they apparently had a debate on fun in learning. The proposition was “all learning should be fun”.  And while the answer is obviously ‘no’, I think that it’s too simplistic of a question. So I want to dig a bit deeper into fun, engagement, and learning, how the right alignment is ‘hard fun’.

Donald Clark weighs in with a summary of the debate and the point he thought was the winner. He lauds Patti Shank, who pointed out that research talks about ‘desirable difficulty’. And I can’t argue with this (besides, Patti’s usually spot-on).  He goes on to cite how you read books that aren’t funny, and that how athletes train isn’t particularly giggle-inducing.  All of which I agree with, except this “Engagement and fun are proxies and the research shows that effort trumps fun every time.”  And I think tying engagement and fun together is a mistake.

There is the trivial notion of fun, to be fair.  The notion that it’s breezily entertaining.  But I want to make a distinction between such trivial attention and engagement.  For instance, I would argue that a movie like Schindler’s List is wholly engaging, but I’m not sure I would consider it ‘fun’.  And even ‘entertaining’ is a stretch. But I think it’s compelling. Similarly with even reading books for entertainment: many aren’t ‘fun’ in the sense of light entertainment and humor, but are hard to put down. So what’s going on here?

I think that cognitive (and emotional) immersion is also ‘engagement’.  That is, you find the story gripping, the action compelling, or the required performance to be a challenge, but you persist because you find it engaging in a deeper sense.

Raph Koster wrote A Theory of Fun about game design, but the underlying premise was that why games were ‘fun’ is that they were about learning. The continually increasing challenge, set in a world that you find compelling (we don’t all like the same games), is what makes a game fun. Similarly, I’ve written about engagement as a far more complex notion than just a trivial view of fun.

The elements of the alignment between effective education practice and engaging experiences demonstrate that learning can, and should, be hard fun. This isn’t the trivial sort of ‘fun’ that apparently is what Donald and Patti were concerned about.  It is all about ‘desirable difficulty’, having a challenge in the zone that’s Czikszentmihalyi’s Flow and Vygotsky’s Zone of Proximal Development.

I agree that just making it fun (just as putting high production values on under-designed content dump) isn’t the answer. But just making it ‘work’ doesn’t help either.  You want people to see the connection between what they’re doing and their goals. Learners should have a level of challenge that helps them know that they’re working toward that goal. You want them to recognize that the tasks are for achieving that goal. It’s about integrating the cognitive elements of learning with the emotional components of engagement in a way that the whole is greater than the sum of the parts. The synergy is what is needed.

I think fun and engagement aren’t the same thing. So while I agree with the premise that learning shouldn’t be the trivial sense of fun, I think the more rigorous sense should be the goal of learning. We want learning to be a transformation, not just a trudge nor a treat.  I’ll argue that the athletes and the readers and the others who are learning are engaged, just not amused. And that’s the important distinction. This is, to me, what Learning Experience Design should be, designing hard fun. And I think we can do this; my upcoming workshop at Learning Solutions is about doing just that. Hope to see you there!

Application-based Instruction

by 2 Comments

A number of years ago, I wrote about activity-based learning.  And I firmly stand behind the model there. It’s not a major campaign, but quietly permeates the things that I do. However, I realize that there’re two misnomers in the label, and it’s time to rectify that. It’s about instruction, and it’s about application. I need to make those distinctions clear.

The original point was to find a way to make it hard for content (read: info dump and knowledge test) to be the basis of curricula. So, instead of a series of content, or knowledge, to be a curriculum,  a curriculum is a series of activities.

There are, of course, specific constraints around the activities, and I realize it’s about knowledge application. There are lots of activities that aren’t going to lead to meaningful learning. The key is retrieval, but I think what’s important is retrieval to do something. Hence application, applying knowledge to make decisions. It can be behavioral decisions or the decisions inherent in creating meaningful work product, but it’s about cognitive skills in context.

The focus on decisions is because I believe what will make the difference to organizations is making right, or better, decisions. Not, for instance, just knowing things. And part of the application core is about doing things.

I argue that learning is acton and reflection. And, consequently, instruction is designed action and guided reflection. Similarly, application is designed actions to require decisions. And this is about instruction. It’s not learning, though that’s the goal. But instruction is a probabilistic game; as I’ve paraphrased Dorothy Parker, you can lead learners to learning, but you can’t make them think.

‘Application-based instruction’ it is. It’s like problem-based learning, or case-based, but it’s instruction not learning, and it’s an umbrella for those and other such initiatives. The real question is whether this labeling makes important distinctions. I think it does, but I’m biased. What do you say?

Transformation!

by 1 Comment

butterfly cocoonI’m a fan of the notion of ‘learning experience design’ (not so sure about platforms; I need to investigate them more ;).  The idea of integrating effective education and engaging experiences is something I’ve been on about for a very long time. And I want to push it a little further. I want to talk about transformation.

What am I talking about? So, I’ve previously referred to Pine & Gilmore’s concept of the Transformation Economy. That is, going beyond experiences (e.g. themed restaurants) to ones that change us. And I argue that’s what we do; we create (or should) experiences that give us new skills, new abilities to do.  But I want to push it further.

Here I’m talking about deliberately using the idea of transformation as a learning design goal. Not just change, but leveraging the emotions as well as cognition to have the learner not just feel empowered, but transformed!  This may sound like a lofty goal, fine for a TED Talk (just read the book; recommended), but is it practical for elearning?  Well, that’s an interesting question.

Let me spin it another way: I do not think we should be shooting for an information dump and knowledge test. For two reasons: one is that it’s not inspiring. More importantly, however, it also isn’t effective. You end up with what cognitive scientists call ‘inert knowledge’. You’ll learn it and pass a test on it, but when it’s relevant in practice it won’t even get activated!  Because you’ve never used it in ways like you practice.

I think if we are actively thinking about transformation as a goal, we might do a better job of thinking about the necessary practice and the emotional engagement.  We can focus on thinking “what will lead to the transformation we want”, and “how do we make people want it and celebrate when they’ve made the breakthrough?”  And I think this is a useful perspective.

Even for things like compliance, I’d suggest that we should be having visceral reactions like “Ok, I get it <bad behavior> is pretty heinous”, and “safety is important, and I commit to following these rules”.  For more important things, you’d like them to feel “yes, I see, this will change how I do this!”

Yes, it’s ambitious. But why set ourselves limited goals? When I was teaching interface design, I maintained that if I accommodated the engineers lack of background in Psych, I’d get them only so far. If I pushed them, they’d end up farther than if I was conciliatory. Similarly, here, I think we’ll do a better job if we think ‘ambitious’, and end up not as far as we’d like. I’ll suggest that’s better than satisfactorily achieving mediocrity. Most importantly, I truly think we’ll do a better job of design if we strive for transformation.

And, if there’s nothing transformative about what we’re covering, should we really be using our resources?  Let me put it another way: why shouldn’t we do this? Seriously, I’m asking.  So, what’s your answer?

 

Content Confusion

by 1 Comment

I read, again and again, about the importance of ‘content’ in learning. And I don’t disagree, but…I think there’s still a problem about it. And where I get concerned is about what is meant by the term.  Just what do mean by ‘content’?  And why should we care about the distinction?

My worry is twofold. For one, I get concerned that talking about content foregrounds ‘information’. And that’s a problem. I’ve been concerned for a while about how it’s too easy to allow knowledge to dominate learning objectives. Know, understand, etc are generally not meaningful objectives. Objectives should be ‘able to use to ___’.  Talking about content, as I’ve talked about before, leads us down a slippery slope to curriculum being defined as content.

My second concern is related. It’s about content being meant to include concepts, examples, and practice. Yet, if we don’t separate out interactivity separate from consumption, we can make nonsensical learning interactions instead of meaningful applications of concepts to contexts. Recognition is not powerful learning.

Look, I get it.  From a technical point of view, e.g. a content management system perspective, it’s all content. It’s addressable files. They may just report access, or they can report success/failure, or many other things. However, again, this view can make it easy to do bad things. And, as the book Nudge I just read suggests, we want to make it easy to do the right things, and make you have to work to do things inappropriately.

So I may be being a pedant about this, but I have a reason. It won’t be when we’re all on the same page about good learning design, practice foregrounded and concepts and examples as learning resources, not the goal. But I don’t think we’re there yet.  And language matters in shaping thinking. It may not be the Whorfian Hypothesis, but it does influence how we think and what we do.

For principled and practical reasons, I think we want to distinguish between content (concepts and examples) and interactives (practice).  At least as designers. Others can focus differently, but we have our own language for other things (I’d argue our use of the term ‘objectives’ is different than business folks, for example), and I argue we should do so here as well.  What say you?