Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

27 July 2016

Serious Comics

Clark @ 8:11 am

I attended  ComicCon  again this year, and addition to the wild costumes, crowded exhibit hall, and over-priced food, there are a series of sessions. They cover television, movies, and print in a wide variety of markets.  And I like the sessions that aren’t associated with popular media (as waiting in lines is something I’m fairly averse too).  One I saw this year (not all of, for several reasons) was particularly thought-provoking.

As background, when I was approached by the Australian Children’s Welfare Agency, many years ago, to do a game to help kids who grow up in ‘non-parental’ situations, they’d already spent their money on a video, and a comic book, and a poster.  As far as I know, it was the first serious game you could play on the web (and I’m happy to have that disconfirmed, but as I’ve thought about it and tried to find out to the contrary, I haven’t found to the contrary). And back then we didn’t even have the label ‘serious game’!

And I’ve been a fan of serious games since before then (my first job out of college was designing and programming educational computer games).  In fact, one of the reasons I went to grad school was because  I saw the connection between adventure games and learning, but it wasn’t clear they were commercially viable (at that time).

But I didn’t think about the comic book much.  I got a copy as part of the overall launch when the game was released along with the other materials, so I’m sure I read it (it may even be lurking somewhere in a cubbyhole somewhere, though could also have been the victim of a move or a tidiness binge).  And I’ve argued before about how graphic novel and such formats aren’t used enough in learning.

So this session was on serious comics, and it of course resurrected those thoughts. One panelist opened about how they were using comics to spark reading, and I was reminded how apparently the original Pokemon games (not Go, though that was obsessing my kids on the trip) required and consequently sparked lots of reading. The second speaker introduced how he was using comics to spread STD/HIV awareness. These are actually both serious issues.

Of course, I was also reminded of an interactive comic book I once read on my iPad that had games interspersed that advanced the storyline (I couldn’t finish because I couldn’t complete one of the games: I’ve little time to spend developing the necessary ‘twitch’ skills).  However, more serious games, requiring applying the knowledge available through the comic, could provide an embedded practice environment.  It’s sort of a blend between a pure comic and a pure game, for important outcomes.  And this is very doable in ebook formats, even if the ‘game’ is just a mini-scenario or several, but with HTML 5 embedded you could do more.

I once wrote that in the future there would be lots of little interactive ‘learnlets’ that would teach you anything you needed to know (including how to make learnlets ;) and games or even interactive comics are what I meant and what could be pretty close to ideal.  It’s been doable for a while, but now it’s doable pretty much with commercially available tools (e.g. not requiring custom programming).  We can make learning ‘hard fun’, and we should. So, what are you waiting for?

26 July 2016

Quinnovation Fall 2016 Schedule

Clark @ 9:51 am

My fall  schedule is coalescing, so I thought I’d provide pointers to when and where I’ll be for the rest of this year:

I’m doing two  webinars for a government agency, one at the end of August, and one at the end of September.

I’ll be in Beijing running a mobile learning workshop on the 6th of September, and keynoting the CEFE conference on the 7th.

The week after I’ll be keynoting a private event in Connecticut.

And I’ll be delivering a virtual keynote for a different government agency in November.

I’ll be running an elearning strategy (read: Revolution) workshop at DevLearn in Las Vegas come mid-November, and presenting on elearning myths.

Then, on the very last day of November, I’ll be running an elearning design workshop at Online Educa in Berlin.

So, some availability  in late September through October, or mid-December, if you’d like access to Quinnovation as well.

I hope that if you’re near Beijing, Las Vegas, or Berlin, you’ll be attending. If so, say hi!


20 July 2016

The wrong basis

Clark @ 8:08 am

Of late, I’ve been talking about the approach organizations take to learning.  It’s come up in presentations on learning design, measurement, and learning technology strategy.  And the point is simple: we’re not using the right basis.

What we’re supposed to be doing is empirically justifiable:

  • doing investigations into the problem
  • identifying the root cause
  • mapping back to an intervention design
  • determining how we’ll know the intervention is working
  • implementing our intervention
  • testing to see if we’ve achieved the necessary outcome
  • and revising until we do

Instead, what we see is what I’ve begun to refer to as ‘faith-based learning’: if we build a course, it is good!  We:

  • take orders for courses
  • document what the SME tells us
  • design a screen-friendly version of the associated content
  • and add a knowledge test

Which would be well and good except that this approach has a very low likelihood of affecting anything except perhaps our learners’ patience (and of course our available resources). Orders for courses have little relation to the real problems, SMEs can’t tell you what they actually do, content 0n a screen doesn’t mean learners know how to or will apply it, and a quiz isn’t likely to lead to any meaningful change in behavior (even if it is tarted up with racing cars).

The closer you are to the former, the better; the closer to the latter, the more likely it is that you’re quite literally wasting time and money.

Faith may not be a bad thing for spirituality, but it’s not a particularly good basis for attempting to develop new skills.  I’ve argued that learning design really is rocket science, and we should be taking an engineering approach.  To the extent we’re not – to the extent that we are implicitly accepting that a course is needed and that our linear processes are sufficient – we’re taking an approach that very much is based upon wishful thinking. And that’s not a good basis to run a business on.

It’s time to get serious about your learning.  It’s doable, with less effort than you may think.   And the alternative is really unjustifiable. So let’s get ourselves, and our industry, on a sound basis.  There’s a lot more we can do as well, but we can start by getting this part right.  Please?

19 July 2016

‘Form’ing learning

Clark @ 8:04 am

Last week I ran a workshop for an online university that is working to improve it’s learning design.  Substantially.  They’re ramping up their staff abilities, and we’d talked about how I could help.  They have ‘content’, but wanted to improve the learning design around this.  While there are a number of steps to take (including how you work with SMEs, the details you attend to in your content, etc), their internal vocabulary talks about ‘knowledge checks’ and the goal was to do those better as they migrate existing courses to a new platform with a suite of assessment types.

So, first of all, my focus was on formative evaluation.  If we take activity-based learning seriously, we need to ensure that there are meaningful tasks set that can provide feedback.  They are fans of Make It Stick (mentioned in my Deeper eLearning reading list), so it was easy to help them recognize that good activities require learners retrieve the information in context, so each formative evaluation should be a situation requiring a decision.

Ok, so not every formative evaluation should be such a situation. But for things that need to be known by rote, I recommend tarted-up ‘drill and kill’. And it became clear, they’re fine at developing standard knowledge checks, it’s the more important ones that needed work.

I started out reviewing the principles, not least because they had a larger audience they wanted to appreciate the background being applied.  Then we moved on to more hands-on work.  First we worked through the different types of assessment types (moving from true/false to more complex assessments like ‘submit and compare’).  We then proceeded to review a first pass to understand the overall course requirements and likely important milestone assessments. We concluded by working through some examples of tough challenges (they’d submitted) and workshopping how to revise them.

There was more behind this, including my understanding more of their context and task, but overall it appeared to develop their understanding of how to take formative evaluation and turn it into an opportunity to truly develop learners in ways that will benefit them after the learning experience.

Of course, focusing on decisions was a key component, and we visited and revisited the issues of working with SMEs. This included getting contexts, and how exaggeration is your friend.  The result is that they’re much better equipped to develop ‘knowledge checks’ that go far beyond knowledge, and actually develop skills that are critical to success after graduation.

This is the type of thinking that organizations from K12 through higher ed and workplace learning (whether corporate, not-for-profit, or government) need to adopt if they’re going to move to learning experiences that actually develop meaningful new abilities.  It’s also about good objectives and more, but what the learner actually does, how they are required to use the knowledge, is critical to the outcome. So, are you ready to make learning that works?

13 July 2016

‘Checking’ In

Clark @ 8:03 am

As a personal reflection, the value of checklists and forcing functions can definitely be understated.  As I mentioned, last week I went into the woods for a few days.  And while the trip didn’t live up to our plans, it was a great experience.  However, there was a particular gap that points out our cognitive limitations.

So, I have a backpacking checklist. And I look at it from time to time. What I didn’t do this time was check it before the trip.  And I found out once I got away from home was that I’d forgotten both my bandana and my towel!  Both are useful, and while I was able to purchase a bandana ($15! but it is microfiber and large, so I’ll keep using it), I had to do without the towel (which the bandana was a poor but necessary substitute for).

We often swim or wade in the river (and did this trip too), and a towel’s handy to get dry before the breeze chills you or the horseflies descend. The bandana, well it served as a sun cover, mosquito deterrent, towel (see above), and glasses wipe. Amongst others.

Let me add that I almost left on today’s overnite biz trip without my sleep clothes!  Fortunately, I had one of those middle-of-the-nite epiphanies, and remedied this morning.

And this just isn’t a consequence of advancing age (hey, I’m still [barely] < 60!).  It’s a natural consequence of our cognitive architecture, and we have well-established processes/tools to support these gaps.  These include checklists to help us remember things, and forcing functions whereby we place things in ways that it’s hard to forget things.

As a consequence, I’m going to do two things going forward. One is to make sure I do check my checklist. I’ll review it for comprehensiveness in the meantime, and have developed it in conjunction with another list from an experience colleague. I have another wilderness trip, and I’ll definitely check it beforehand.  Second, I’ve now put the bandana and a towel in my backpack. So I’d actually have to take it out to forget it!

Here’s to knowing, and applying, tools to help us overcome our cognitive deficits.  What are you doing to help not make mistakes?  And what could you do similarly for your learning design processes?

21 June 2016

eLearning Process Survey results!

Clark @ 8:05 am

So, a few weeks ago I ran a survey asking about elearning processes*, and it’s time to look at the results (I’ve closed it).  eLearning process is something I’m suggesting is ripe for change, and I thought it appropriate to see what people thoughts.  Some caveats: it’s self-selected, it’s limited (23 respondents), and it’s arguably readers of this blog or the other folks who pointed to it, so it’s a select group.  With those caveats, what did we see?

SQ1The first question was looking at how we align our efforts with business needs. The alternatives were ‘providing what’s asked for’ (e.g. taking orders), ‘getting from SMEs’, and ‘using a process’.  These are clearly in ascending order of appropriateness. Order taking doesn’t allow for seeing if a course is needed and SMEs can’t tell you what they actually do. Creating a process to ensure a course is the best solution (as opposed to a job aid or going to the network), and then getting the real performance needs (by triangulating), is optimal.  What we see, however, is that only a bit more than 20% are actually getting this right from the get-go, and almost 80% are failing at one of the two points along the way.

SQ2The second question was asking about how the assessments were aligned with the need. The options ranged from ‘developing from good sources’, thru ‘we test knowledge’ and ‘they have to get it right’ to ‘sufficient spaced contextualized practice’, e.g. ’til they can’t get it wrong.  The clear need, if we’re bothering to develop learning, is to ensure that they can do it at the end.  Doing it ‘until they get it right’ isn’t sufficient to develop a new ability to do. And, we see more than 40% are focusing on using the existing content! Now, the alternatives were not totally orthogonal (e.g. you could have the first response and any of the others), so interpreting this is somewhat problematic.  I assumed  people would know to choose the lowest option in the list if they could, and I don’t know that (flaw in the survey design).  Still it’s pleasing to see that almost 30% are doing sufficient practice, but that’s only a wee bit ahead of those who say they’re just testing knowledge!  So it’s still a concern.

SQ3The third question was looking at the feedback provided. The options included ‘right or wrong’, ‘provides the right answer’, and ‘indication for each wrong answer’.  I’ve been railing against one piece of feedback for all the wrong answers for years now, and it’s important. The alternatives to the wrong answer shouldn’t be random, but instead should represent the ways learners typically get it wrong (based upon misconceptions).  It’s nice (and I admit somewhat surprising) that almost 40% are actually providing feedback that addresses each wrong answer. That’s a very positive outcome.  However, that it’s not even half is still kind of concerning.

SQ4The fourth question digs into the issue of examples.  There are nuances of details about examples, and here I was picking up on a few of these. The options ranged from ‘having’, thru ‘coming from SMEs’ and ‘illustrate the concept and context’, to ‘showing the underlying thinking’.  Again, obviously the latter is the best.  It turns out that experts don’t typically show the underlying cognition, and yet it’s really valuable for the learning. We see that we are getting the link of concept to context clear, and together with showing thinking we’re nabbing roughly 70% of the examples, so that’s a positive sign.

SQ5The fifth question asks about concepts.  Concepts are (or should be) the models that guide performance in the contexts seen across examples and practice (and the basis for the aforementioned feedback). The alternatives ranged from ‘using good content’ and ‘working with SMEs’ to ‘determining the underlying model’.  It’s the latter that is indicated as the basis for making better decisions, going forward.  (I suggest that what will helps orgs is not the ability to receive knowledge, but to make better decisions.)  And we see over 30% going to those models, but still a high percentage still taking the presentations from the SMEs. Which isn’t totally inappropriate, as they do have access to what they learned. I’m somewhat concerned overall that much of ID seems to talk about practice and ‘content’, lumping intros and concepts and examples and closing all together into the latter (without suitable differentiation), so this was better than expected.

SQ6The sixth question tapped into the emotional side of learning, engagement. The options were ‘giving learners what they need’, ‘a good look’, ‘gamification’, and ‘tapping into intrinsic motivation’.  I’ve been a big proponent of intrinsic motivation (heck, I effectively wrote a book on it ;), and not gamification. I think an appealing visual design, but just ‘giving them what they need’ isn’t sufficient for novices: they need the emotional component too. For practitioners, of course, not so much.  I’m pleased that no one talked about gamification (yet the success of companies that sell ‘tart up’ templates suggests that this isn’t the norm). Still, more than a third are going to the intrinsic motivation, which is heartening. There’s a ways to go, but some folks are hearing the message.

SQ7The last question gets into measurement.  We should be evaluating what we do. Ideally, we start from a business metric we need to address and work backward. That’s typically not seen. The questions basically covered the Kirkpatrick model, working from ‘smile sheets’, through’ testing after the learning experience’ and ‘checking changes in workplace behavior’ to ‘tuning until impacting org  metrics’.  I was pleasantly surprised to see over a third doing the latter, and my results don’t parallel what I’ve seen elsewhere. I’m dismayed, of course, that over 20% are still just asking learners, which we know in general isn’t of particular use.

This was a set of questions deliberately digging into areas where I think elearning falls down, and (at least with this group of respondents), it’s not good as I’d hope, but not as bad as I feared.  Still, I’d suggest there’s room for improvement, given the constraints above about who the likely respondents are.  It’s not a representative sample, I’d suspect.

Clearly, there are ways to do well, but it’s not trivial. I’m arguing that we can do good elearning without breaking the bank, but it requires an understanding of the inflection points of the design process where small changes can yield important results. And it requires an understanding of the deeper elements to develop the necessary tools and support. I have been working with several organizations to make these improvements, but it’s well past time to get serious about learning, and start having a real impact.

So over to you: do you see this as a realistic assessment of where we are? And do you take the overall results as indicating a healthy industry, or an industry that needs to go beyond haphazard approaches and start practicing Learning Engineering?

*And, let me say, thanks very much to those respondents who bothered to take the time to respond.  It was quick, but still, the effort was completely appreciated.


16 June 2016

John Black #ICELW Keynote Mindmap

Clark @ 8:06 am

Professor John Black of Columbia Unveristy gave a fascinating talk about how games can leverage “embodied cognition” to achieve deeper learning. The notion is that by physical enaction, you get richer activation, and sponsor deeper learning.  It obviously triggered lots of thoughts (mine are the ones in the bubbles :). Lots to ponder.

1 June 2016

The Quinnovation eLearning Process Survey

Clark @ 8:08 am

In the interests of understanding where the market is, I’m looking to benchmark where organizations are. Sure, there are other data points, but I have my own questions I would like to get answered. So I’ve created a quick survey of seven questions (thanks, SurveyMonkey) I’d love for you to fill out.

My interest is in finding out about the processes used in designing and delivering elearning. While I’ve my own impressions, I thought it would be nice to bolster it with data. So here we are.
And I’m not asking what org you’re working for, because I’d appreciate honest answers. Please feel free to respond and circulate to those you know in other organizations (but try to only have one person from your org fill it out).

This is an experiment (hey, that’s what innovation is all about ;), so we’ll see how it goes. I’ll report out what happens when responses start petering out (or when I hit my 100 response cap ;). I welcome your comments or questions as well. Thanks!

Create your own user feedback survey

31 May 2016

Where do comics/cartoons fit?

Clark @ 8:07 am

I’ve regularly suggested that you want to use the right media for the task, and there are specific cognitive properties of media that help determine the answer.  One important dimension is context versus concept, and another is dynamic versus static.  But I realized I needed to extend it.

MediaPropertiesNewTo start with, concepts are relationships, such as diagrams (as this one is!).  Whereas context is the actual setting. For one, you want to abstract away, for the other you want to be concrete.  Similarly, some relationships, and settings, are static, whereas others are dynamic. Obviously, here we’re talking static relationships, but if we wanted to illustrate some chemical process, we might need an animation.

So, for contextualization, we can use a photo capturing the real setting. Unless, of course, it’s dynamic and we need a video. Similarly, if we need conceptual relationships, we use a diagram, unless again if it’s dynamic and we need an animation. (By animation, I mean a dynamic diagram, not a cartoon, just as a video is a dynamic recording of a live setting, not a cartoon.)

Audio’s a funny case, in that it can be static as text or dynamic as audio. The needs change depending on where you need your attention represented: you can’t (and shouldn’t) put static text on a dynamic visual, and you can’t use video if the attention can’t be visually distracted. Audio is valuable when you can’t take your eyes away (e.g. the audio guidance on a GPS, “now turn left”).

Note that there are halfway points. You can capture a sequence of static images in lieu of a video (think narrated slide show).  Similarly, a diagram could be shown in multiple states.  And this is all ignoring interactives.  But there’s a particular place I want to go, hinted above.

I was reflecting that comics (static) and cartoons (dynamic) are  instances that don’t naturally fall out of my characterization, and realized I needed a way to consider them.  I posit that comics/cartoons are halfway between context and concept.  They strip away unnecessary context, so that it’s easier to see what’s important, and have the potential (via, say, thought balloons) to annotate the world with the concept.  So they’re semi-conceptual, and semi-contextual.  I’ve regularly argued that we don’t use them often enough for a number of reasons, and it’s important to think where they fit.

This is my proposal: that they help focus attention on important elements without unnecessary details and the ability to elaborate (as well as the rest of the benefits: familiarity, bandwidth, etc).  So, what do you say?  Does this fit and make sense?  Are you going to use more comics/graphic novels/cartoons?

26 May 2016

Heading in the right direction

Clark @ 8:06 am

Most of our educational approaches – K12, Higher Ed, and organizational – are fundamentally wrong.  What I see in schools, classrooms, and corporations are information presentation and knowledge testing.  Which isn’t bad in and of itself, except that it won’t lead to new abilities to do!  And this bothers me.

As a consequence, I took a stand trying to create a curricula that wasn’t about content, but instead about action.  I elaborated it in some subsequent posts, trying to make clear that the activities could be connected and social, so that you could be developing something over time, and also that the output of the activity produced products – both the work and thoughts on the work – that serve as a portfolio.

I just was reading and saw some lovely synergistic thoughts that inspire me that there’s hope. For one, Paul Tough apparently wrote a book on the non-cognitive aspects of successful learners, How Children Succeed, and then followed it up with Helping Children Succeed, which digs into the ignored ‘how’.  His point is that elements like ‘grit’ that have been (rightly) touted aren’t developed in the same way cognitive skills are, and yet they can be developed. I haven’t read his book (yet), but in exploring an interview with him, I found out about Expeditionary Learning.

And what Expeditionary Learning has, I’m happy to discover, is an approach based upon deeply immersive projects that integrate curricula and require the learning traits recognized as important.  Tough’s point is that the environment matters, and here are schools that are restructured to be learning environments with learning cultures.  They’re social, facilitated, with meaningful goals, and real challenges. This is about learning, not testing.  “A teacher’s primary task is to help students overcome their fears and discover they can do more than they think they can.”

And I similarly came across an article by Benjamin Riley, who’s been pilloried as the poster-child against personalization.  And he embraces that from a particular stance, that learning should be personalized by teachers, not technology.  He goes further, talking about having teachers understand learning science, becoming learning engineers.  He also emphasizes social aspects.

Both of these approaches indicate a shift from content regurgitation to meaningful social action, in ways that reflect what’s known about how we think, work, and learn. It’s way past time, but it doesn’t mean we shouldn’t keep striving to do better. I’ll argue that in higher ed and in organizations, we should also become more aware of learning science, and on meaningful activity.  I encourage you to read the short interview and article, and think about where you see leverage to improve learning.  I’m happy to help!

Next Page »

Powered by WordPress