Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

3 March 2015

On the road again

Clark @ 7:42 am

Well, some more travels are imminent, so I thought I’d update you on where the Quinnovation road show would be on tour this spring:

  • March 9-10 I’ll be collaborating with Sarah Gilbert and Nick Floro to deliver ATD’s mLearnNow event in Miami on mobile
  • On the 11th I’ll be at a private event talking the Revolution to a select group outside Denver
  • Come the 18th I’ll be inciting the revolution at the ATD Golden Gate chapter meeting here in the Bay Area
  • On the 25th-27th, I’ll be in Orlando again instigating at the eLearning Guild’s Learning Solutions conference
  • May 7-8 I’ll be kicking up my heels about the revolution for the eLearning Symposium in Austin
  • I’ll be stumping the revolution at another vendor event in Las Vegas 12-13
  • And June 2-3 I’ll be myth-smashing for ATD Atlanta, and then workshopping game design

So, if you’re at one of these, do come up and introduce yourself and say hello!



8 October 2014

The resurgence of games?

Clark @ 8:44 am

I talked yesterday about how some concepts may not resonate immediately, and need to continue to be raised until the context is right.  There I was talking about explorability and my own experience with service science, but it occurred to me that the same may be true of games.

Now, I’ve been pushing games as a vehicle for learning for a long time, well before my book came out on the topic.  I strongly believe that next to mentored live practice (which doesn’t scale well), (serious) games are the next best learning opportunity.  The reasons are strong:

  • safe practice: learners can make mistakes without real consequences (tho’ world-based ones can play out)
  • contextualized practice (and feedback): learning works better in context rather than on abstract problems
  • sufficient practice: a game engine can give essentially infinite replay
  • adaptive practice: the game can get more difficult to develop the learner to the necessary level
  • meaningful practice: we can choose the world and story to be relevant and interesting to learners

the list goes on.  Pretty much all the principles of the Serious eLearning Manifesto are addressed in games.

Now, I and others (Gee, Aldrich, Shaffer, again the list goes on) have touted this for years.  Yet we haven’t seen as much progress as we could and should.  It seemed like there was a resurgence around 2009-2010, but then it seemed to go quiet again. And now, with Karl Kapp’s Gamification book and the rise of interest in gamification, we have yet another wave of interest.

Now, I’m not a fan of the extrinsic  gamification, but it appears there’s a growing awareness of the difference between extrinsic and intrinsic. And I’m seeing more use of games to develop understanding in at least K12 circles.  Hopefully, the awareness will arise in higher ed and corp too.

As some fear, it’s too costly, but my response is twofold:

  • games aren’t as expensive as you fear; there are lots of opportunities for games in lower price ranges (e.g. $100K), don’t buy into the $1M and up mentality
  • they’re actually likely to be effective (as part of a complete learning experience), compared to many if not most of the things being done in learning

So I hope we might finally go beyond Clicky Clicky Bling Bling, (tarted quiz shows, cheesy videos and more) and get to interaction that actually leads to change.  Here’s hoping!

28 August 2014

Kris Duggan #LnDMeetup Gamification Mindmap

Clark @ 8:05 am

Kris Duggan spoke on gamification at the Bay Area Learning Design & Technology MeetUp. He talked about some successes at his Badging role and then his new initiative bringing gamification more intrinsically into organizations. He proposed five Goal Science rules that resonated with other principles I’ve heard for good organizations.


27 May 2014

Setting Story

Clark @ 8:15 am

I’ve been thinking about the deep challenge of motivating uninterested learners.  To me, at least part of that is making the learning of intrinsic interest.  And one of those elements is practice, and this is arguably the most important element to making learning work.  So how to do we make practice intrinsically interesting?

One of the challenging but important components of designing meaningful practice is choosing a context in which that practice is situated.  It’s really about finding a story line that makes the action meaningful to both the learner and the learning. It’s creative (and consequently fun), but it’s also not intrinsically obvious (which I’ve learned after trying to teach it in both game design and advanced ID workshops). There are heuristics to be followed (there’s no guaranteed formula except brainstorm, winnow, trial, and refine), however, that can be useful.

While Subject Matter Experts (SMEs) can be the bane of your existence while setting learning goals (they have conscious access to no more than 30% of what they do, so they tend to end up reciting what they know, which they do have access to), they can be very useful when creating stories. There’s a reason why they’ve spent the requisite time to become experts in the field, and that’s an aspect we can tap into. Find out why it’s of interest to them.  In one instance, when asking experts about computer auditing, a colleague found that auditors found it like playing detective, tracking back to find the error.  It’s that sort of insight upon which a good game or practice exercise can hinge.

One of the tricks to work with SMEs is to talk about decisions.  I argue that what is most likely to make a difference to organizations is that people make better decisions, and I also believe that using the language of decisions helps SMEs focus on what they do, not what they know.  Between your performance gap analysis of the situation, and expert insight into what decisions are key, you’re likely to find the key performances you want learners to practice.

You also want to find out all the ways learners go wrong.  Here you may well hear instructors and/or SMEs say “no matter what we do, they always…”. And that’s the things you want to know, because novices don’t tend to make random errors.  Yes, there’s some, owing to our cognitive architecture (it’s adaptive), which is why it’s bad to expect people to do rote things, but it’s a small fraction of mistakes.  Instead, learners make patterned mistakes based upon mistakes in their conceptualizations of the performance, aka misconceptions.  And  you want to trap those because you’ll have a chance to remediate them in the learning context. And they make the challenge more appropriately tuned.

You also need the consequences of both the right choice and the misconceptions. Even if it’s just a multiple choice question, you should show what the real world consequence is before providing the feedback about why it’s wrong. It’s also the key element in scenarios, and building models for serious games.

Then the trick is to ask SMEs about all the different settings in which these decisions embed. Such decisions tend to travel in packs, which is why scenarios are better practice than simple multiple choice, just as scenario-based multiple choice trumps knowledge test.  Regardless, you want to contextualize those decisions, and knowing the different settings that can be used gives you a greater palette to choose from.

Finally, you’ll want to decide how close you want the context to be to the real context.  For certain high-stakes and well-defined tasks, like flying planes or surgery, you’ll want them quite close to the real situation.  In other situations, where there’s more broad applicability and less intrinsic interest (perhaps accounting or project management), you may want a more fantastic setting that facilitates broader transfer.

Exaggeration is a key element. Knowing what to exaggerate and when is not yet a science, but the rule of thumb is leave the core decisions to be based upon the important variables, but the context can be raised to increase the importance.  For example, accounting might not be riveting but your job depends on it.  Raising the importance of the accounting decision in the learning experience will mimic the importance, so you might be accounting for a mob boss who’ll terminate your existence if you don’t terminate the discrepancy in his accounts!  Sometimes exaggeration can serve a pedagogical purpose as well, such as highlighting certain decisions that are rare in real life but really important when they occur. In one instance, we had asthma show up with a 50% frequency instead of the usual ~15%, as the respiratory complications that could occur required specific approaches to address.

Ultimately, you want to choose a setting in which to embed the decisions. Just making it abstract decreases the impact of the learning, and making it about knowledge, not decisions, will render it almost useless, except for those rare bits of knowledge that have to absolutely be in the head.  You want to be making decisions using models, not recalling specific facts. Facts are better off put in the world for reference, except where time is too critical. And that’s more rare than you’d expect.

This may seem like a lot of work, but it’s not that hard, with practice.  And the above is for critical decisions. In many cases, a good designer should be able to look at some content and infer what the decisions involved should be.  It’s a different design approach then transforming knowledge into tests, but it’s critical for learning.  Start working on your practice items first, aligned with meaningful objects, and the rest will flow. That’s my claim, what say you?

4 February 2014

Exaggeration and Alignment

Clark @ 11:05 pm

In addition to my keynote and session at last week’s Immersive Learning University event, I was on a panel with Eric Bernstein, Andy Peterson, & Will Thalheimer. As we riffed about Immersive Learning, I chimed in with my usual claim about the value of exaggeration, and Will challenged me, which led to an interesting discussion and (in my mind) this resolution.

So, I talk about exaggeration as a great tool in learning design. That is, we too often are reigned in to the mundane, and I think whether it’s taking it a little bit more extreme or jumping off into a fantasy setting (which are similar, really), we bring the learning experience closer to the emotion of the performance environment (when it matters).

Will challenged me about the need for transfer, and that the closer the learning experience is to the performance environment, the better the transfer. Which has been demonstrated empirically. Eric (if memory serves) also raised the issue of alignment to the learning goals, and that you can’t overproduce if you lose sight of the original cognitive skills (we also talked about when such experiences matter, and I believe it’s when you need to develop cognitive skills).

And they’re both right, although I subsequently pointed out that when the transfer goal is farther, e.g. the specific context can vary substantially, exaggeration of the situation may facilitate transfer. Ideally, you would have practice across contexts spanning the application space, but that might not be feasible if we’re high up on the line going from training to education.

And of course, keeping the key decisions at the forefront is critical. The story setting can be altered around those decisions, but the key triggers for making those decisions and the consequences must map to reality, and the exaggeration has to be constrained to elements that aren’t core to the learning. Which should be minimized.

Which gets back to my point about the emotional side. We want to create a plausible setting, but one that’s also motivating. That happens by embedding the decisions in a setting that’s somewhat ‘larger than life’, where we’re emotionally engaged in ways consonant with the ones we will be when we’re performing.

Knowing what rules to break, and when, here comes down to knowing what is key to the learning and what is key to the engagement, and where they differ. Make sense?

20 January 2014

Gaming Learning

Clark @ 7:08 am

Remember the game Where in the World Is Carmen San Diego? The game had you chasing an international fugitive, and you had to decipher clues about world facts to figure out where to go next to catch her, using an included world almanac. The claim for learning was that it developed knowledge of world facts.  And that was patently shown to be wrong by Cathie Sherwood, then at Griffith University (if memory serves).  What she showed was that kids learned how to use an almanac, but didn’t remember the information pointed to by the clues.  And this is a consistent problem with educational software.

I’ve been thinking about games for the simple reason that I’m keynoting and doing a panel and a session about gaming and learning at NexLearn’s Immersive Learning University conference next week.  I’ll be talking about how to design them, and lessons from games for the design of learning and assessment.  So when I read this recent article, while generally supportive, I had a problem.

The good thing with the article is that it argues that we should be doing more with games to support learning, and I couldn’t agree more.  When properly designed, games provide deep and meaningful practice.  And we could be tapping into much more of the facets of games for designing learning experiences. Challenge, decisions, and consequences in a safe environment.

So what bothered me?  At one point, the article does on about what skills are required in computer games, things like problem-solving, strategy, etc.  And, yes, games do require those skills. However, what many have done wrongly is say that the games develop those skills, and this is wrong.  For instance, when Kurt Squire was touting the learning outcomes of Civilization, it came from a teacher who scaffolded that understanding, not intrinsically from the game. Similarly, when my kids were playing Pajama Sam (a great series of games with interesting stories and appropriate challenges), we were scaffolding the learning.

For some, requiring skills will develop them. For the 10% or so who survive despite what we do to them ;).  But if you want to be sure they’re getting developed, you need to do more than require them, you need to scaffold them. And we could do this if we wanted to.  But we don’t. The existence of coaching for higher-level learning skills in the game environment is essentially non-existent. And I just think this is a shame. (Many years ago I was proposing research to develop a coaching environment on top of a game engine, so it could be available in any game designed with that engine, but of course it was deemed too ambitious.  Hmmph.)

And don’t get me wrong, the article didn’t make wrong statements, it just reminded me of the problem that has bugged me and also I think damaged the industry (think: why is the term ‘edutainment’ tainted?).  But we need to be careful what we say and how we talk about it. We can develop meaningful learning games, but we have to know how to do it, not just put game and instructional designers in a room together and expect them to know how to create a success.  You need to understand the alignment of elements of learning and leverage those to achieve success.  Don’t settle for less.

14 November 2013

Really Useful eLearning Instruction Manual

Clark @ 7:02 am

Rob Hubbard organized a suite of us to write chapters for a use-focused guide to elearning. And, now it’s out and available!  Here’s the official blurb:

Technology has revolutionised every aspect of our lives and how we learn is no exception. The trouble is; the range of elearning technologies and the options available can seem bewildering. Even those who are highly experienced in one aspect of elearning will lack knowledge in some other areas. Wouldn’t it be great if you could access the hard-won knowledge, practical guidance and helpful tips of world-leading experts in these fields? Edited by Rob Hubbard and featuring chapters written by global elearning experts: Clive Shepherd, Laura Overton, Jane Bozarth, Lars Hyland, Rob Hubbard, Julie Wedgwood, Jane Hart, Colin Steed, Clark Quinn, Ben Betts and Charles Jennings – this book is a practical guide to all the key topics in elearning, including: getting the business on board, building it yourself, learning management, blended, social, informal, mobile and game-based learning, facilitating online learning, making the most of memory and more.

And here’s the Table of Contents, so you can see who wrote what:

  1. So What is eLearning? – Clive Shepherd
  2. Getting the Business on Board – Laura Overton
  3. Build In-House, Buy Off -the-Shelf or Outsource? – Jane Bozarth
  4. Production Processes – Making it Happen! – Lars Hyland
  5. Making the Most of Memory – Rob Hubbard
  6. Blended Learning – Julie Wedgwood
  7. Informal and Social Learning – Jane Hart
  8. Facilitating Live Online Learning – Colin Steed
  9. Mobile Learning – Clark Quinn
  10. Game-Based Learning – Ben Betts
  11. Learning Management – Charles Jennings

If you’d like to purchase the book, VBF11 is the promotion code to get 15% discount when you buy the book at www.wiley.com, or you can get it through Amazon as a book or on Kindle.  I look forward to getting my copy in the mail!

18 June 2013

Christopher Pirie #mlearncon Keynote Mindmap

Clark @ 10:15 am

Christopher Pirie opened the eLearning Guild’s mLearnCon mobile learning conference with a fair overview of technology for learning.  He talked about the usual trends, and pointed to some interesting game apps for learning.  Kodu, in particular, is an interesting advancement on things like Scratch and StageCast’s Creator.

I was somewhat surprised by his pointer to Bloom as the turning point to modern learning design, as I’d be inclined to point more to Collins & Brown’s Cognitive Apprenticeship. I also think he should take a look at Donald Clark’s criticisms of Mitra’s Hole in the Wall.  Finally, the characterization between the overhead projector as characteristic of 1991 and the Kinect for 2012 is a bit spurious: in 1991 we also had HyperCard, and in 2012 I don’t see the Kinect in many classrooms yet, but his point is apt about the potential for change we have at our fingertips.

Overall, a nice kickoff for the conference.


9 May 2013

Assessing online assessments

Clark @ 5:00 am

Good formal learning consists of an engaging introductions, rich presentation of concepts, annotated examples, and meaningful practice, all aligned on cognitive skills. As we start seeing user-generated online c, publishers and online schools are feeling the pressure. Particularly as MOOCs come into play, with (decreasingly) government funded institutions giving online content and courses for free. Are we seeing the demise of for-profit institutions and publishers?

I will suggest that there’s one thing that is harder to get out of the user-generated content environment, and that’s meaningful practice. I recall hearing of, but haven’t yet seen, a seriously threatening repository of such. Yes, there are learning object repositories, but they’re not yet populated with a rich suite of contextualized practice.

Writing good assessments is hard. Principles of good practice include meaningful decisions, alternatives that represent reliable misconceptions, relevant contexts, believable dialog, and more. They must be aligned to the objectives, and ideally have an increasing level of challenge.

There are some technical issues as well. Extensions that are high value include problem generators and randomness in the order of options (challenging attempts to ‘game’ the assessment). A greater variety of response options for novelty isn’t bad either, and automarking is desirable for at least a subset of assessment.

I don’t want to preclude essays or other interpretive work like presentations or media content, and they are likely to require human evaluation, even with peer marking. Writing evaluation rubrics is also a challenge for untrained designers or experts.

While SMEs can write content and even examples (if they get pedagogical principles and are in touch with the underlying thinking, but writing good assessments is another area.

I’ve an inkling that writing meaningful assessments, particularly leveraging interactive technology like immersive simulation games, is an area where skills are still going to be needed. Aligning and evaluating the assessment, and providing scrutable justification for the assessment attributes (e.g. accreditation) is going to continue to be a role for some time.

We may need to move accreditation from knowledge to skills (a current problem in many accreditation bodies), but I think we need and can have a better process for determining, developing, and assessing certain core skills, and particularly so-called 21st century skills. I think there will continue to be a role for doing so, even if we make it possible to develop e necessary understanding in any way the learner chooses.

As is not unusual, I’m thinking out loud, so I welcome your thoughts and feedback.

17 April 2013

Sach’s Winning the Story Wars

Clark @ 6:47 am

On a recommendation, I’ve been reading Jonah Sach’s Winning the Story Wars.   While it’s ostensibly about marketing/advertising, which interests me not, I was intrigued by the possibilities to understand stories from a different perspective.   I was surprised to find that it offered much more.

The book does cover the history of advertising, going through some classic examples of old-style advertising, and using some surprisingly successful examples to elicit a new model.  Some personal stories and revelations make this more than a conceptual treatise.

The core premise is turning your customer into a potential hero of an important journey.  You play the role of the mentor, providing the magic aid for them to accomplish a goal that they know they need, but for a variety of reasons may have avoided.  The journey is motivated from core values, a feature that resonates nicely with my personal quest for using technology to facilitate wisdom.

The book also provides, as one of the benefits, a nice overview of story, particularly the hero’s journey as synthesized by Joseph Campbell across many cultures and time periods.  If you find Campbell a tough read, as many do, this is a nicely digested version.  It talks in sensible ways about the resistance, and trials, and ultimate confrontation.

The obvious focus is on new way to build your brand, tapping into higher purpose, not the more negative fears of inadequacy.  So this book is valuable for those looking to market in a higher way.  And I do intend to rethink the Quinnovation site as a consequence.  But I suggest there’s more.

The notion of the individual being offered the opportunity to play a transformative role seems to be a useful framing for learning. We can, and should, be putting learners in meaningful practice roles, and those roles can be coming from learners’ deep motivators. One of the heuristics in learning game design is Henry Jenkins’ “put the player in a role they’d like to be in”.  This provides a deeper grounding, put the learner in a role they aspire to be in.

I think this book provides not only practical marketing advice, but also guidance for personal journeys and learning.  I think that the perspective of designing stories and roles that are based on personal values to be a great opportunity to do better design. I haven’t completely finished it yet, but I’ve already found enough value in the majority of it to recommend it to you.

Next Page »

Powered by WordPress