Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

27 May 2014

Setting Story

Clark @ 8:15 am

I’ve been thinking about the deep challenge of motivating uninterested learners.  To me, at least part of that is making the learning of intrinsic interest.  And one of those elements is practice, and this is arguably the most important element to making learning work.  So how to do we make practice intrinsically interesting?

One of the challenging but important components of designing meaningful practice is choosing a context in which that practice is situated.  It’s really about finding a story line that makes the action meaningful to both the learner and the learning. It’s creative (and consequently fun), but it’s also not intrinsically obvious (which I’ve learned after trying to teach it in both game design and advanced ID workshops). There are heuristics to be followed (there’s no guaranteed formula except brainstorm, winnow, trial, and refine), however, that can be useful.

While Subject Matter Experts (SMEs) can be the bane of your existence while setting learning goals (they have conscious access to no more than 30% of what they do, so they tend to end up reciting what they know, which they do have access to), they can be very useful when creating stories. There’s a reason why they’ve spent the requisite time to become experts in the field, and that’s an aspect we can tap into. Find out why it’s of interest to them.  In one instance, when asking experts about computer auditing, a colleague found that auditors found it like playing detective, tracking back to find the error.  It’s that sort of insight upon which a good game or practice exercise can hinge.

One of the tricks to work with SMEs is to talk about decisions.  I argue that what is most likely to make a difference to organizations is that people make better decisions, and I also believe that using the language of decisions helps SMEs focus on what they do, not what they know.  Between your performance gap analysis of the situation, and expert insight into what decisions are key, you’re likely to find the key performances you want learners to practice.

You also want to find out all the ways learners go wrong.  Here you may well hear instructors and/or SMEs say “no matter what we do, they always…”. And that’s the things you want to know, because novices don’t tend to make random errors.  Yes, there’s some, owing to our cognitive architecture (it’s adaptive), which is why it’s bad to expect people to do rote things, but it’s a small fraction of mistakes.  Instead, learners make patterned mistakes based upon mistakes in their conceptualizations of the performance, aka misconceptions.  And  you want to trap those because you’ll have a chance to remediate them in the learning context. And they make the challenge more appropriately tuned.

You also need the consequences of both the right choice and the misconceptions. Even if it’s just a multiple choice question, you should show what the real world consequence is before providing the feedback about why it’s wrong. It’s also the key element in scenarios, and building models for serious games.

Then the trick is to ask SMEs about all the different settings in which these decisions embed. Such decisions tend to travel in packs, which is why scenarios are better practice than simple multiple choice, just as scenario-based multiple choice trumps knowledge test.  Regardless, you want to contextualize those decisions, and knowing the different settings that can be used gives you a greater palette to choose from.

Finally, you’ll want to decide how close you want the context to be to the real context.  For certain high-stakes and well-defined tasks, like flying planes or surgery, you’ll want them quite close to the real situation.  In other situations, where there’s more broad applicability and less intrinsic interest (perhaps accounting or project management), you may want a more fantastic setting that facilitates broader transfer.

Exaggeration is a key element. Knowing what to exaggerate and when is not yet a science, but the rule of thumb is leave the core decisions to be based upon the important variables, but the context can be raised to increase the importance.  For example, accounting might not be riveting but your job depends on it.  Raising the importance of the accounting decision in the learning experience will mimic the importance, so you might be accounting for a mob boss who’ll terminate your existence if you don’t terminate the discrepancy in his accounts!  Sometimes exaggeration can serve a pedagogical purpose as well, such as highlighting certain decisions that are rare in real life but really important when they occur. In one instance, we had asthma show up with a 50% frequency instead of the usual ~15%, as the respiratory complications that could occur required specific approaches to address.

Ultimately, you want to choose a setting in which to embed the decisions. Just making it abstract decreases the impact of the learning, and making it about knowledge, not decisions, will render it almost useless, except for those rare bits of knowledge that have to absolutely be in the head.  You want to be making decisions using models, not recalling specific facts. Facts are better off put in the world for reference, except where time is too critical. And that’s more rare than you’d expect.

This may seem like a lot of work, but it’s not that hard, with practice.  And the above is for critical decisions. In many cases, a good designer should be able to look at some content and infer what the decisions involved should be.  It’s a different design approach then transforming knowledge into tests, but it’s critical for learning.  Start working on your practice items first, aligned with meaningful objects, and the rest will flow. That’s my claim, what say you?

4 February 2014

Exaggeration and Alignment

Clark @ 11:05 pm

In addition to my keynote and session at last week’s Immersive Learning University event, I was on a panel with Eric Bernstein, Andy Peterson, & Will Thalheimer. As we riffed about Immersive Learning, I chimed in with my usual claim about the value of exaggeration, and Will challenged me, which led to an interesting discussion and (in my mind) this resolution.

So, I talk about exaggeration as a great tool in learning design. That is, we too often are reigned in to the mundane, and I think whether it’s taking it a little bit more extreme or jumping off into a fantasy setting (which are similar, really), we bring the learning experience closer to the emotion of the performance environment (when it matters).

Will challenged me about the need for transfer, and that the closer the learning experience is to the performance environment, the better the transfer. Which has been demonstrated empirically. Eric (if memory serves) also raised the issue of alignment to the learning goals, and that you can’t overproduce if you lose sight of the original cognitive skills (we also talked about when such experiences matter, and I believe it’s when you need to develop cognitive skills).

And they’re both right, although I subsequently pointed out that when the transfer goal is farther, e.g. the specific context can vary substantially, exaggeration of the situation may facilitate transfer. Ideally, you would have practice across contexts spanning the application space, but that might not be feasible if we’re high up on the line going from training to education.

And of course, keeping the key decisions at the forefront is critical. The story setting can be altered around those decisions, but the key triggers for making those decisions and the consequences must map to reality, and the exaggeration has to be constrained to elements that aren’t core to the learning. Which should be minimized.

Which gets back to my point about the emotional side. We want to create a plausible setting, but one that’s also motivating. That happens by embedding the decisions in a setting that’s somewhat ‘larger than life’, where we’re emotionally engaged in ways consonant with the ones we will be when we’re performing.

Knowing what rules to break, and when, here comes down to knowing what is key to the learning and what is key to the engagement, and where they differ. Make sense?

20 January 2014

Gaming Learning

Clark @ 7:08 am

Remember the game Where in the World Is Carmen San Diego? The game had you chasing an international fugitive, and you had to decipher clues about world facts to figure out where to go next to catch her, using an included world almanac. The claim for learning was that it developed knowledge of world facts.  And that was patently shown to be wrong by Cathie Sherwood, then at Griffith University (if memory serves).  What she showed was that kids learned how to use an almanac, but didn’t remember the information pointed to by the clues.  And this is a consistent problem with educational software.

I’ve been thinking about games for the simple reason that I’m keynoting and doing a panel and a session about gaming and learning at NexLearn’s Immersive Learning University conference next week.  I’ll be talking about how to design them, and lessons from games for the design of learning and assessment.  So when I read this recent article, while generally supportive, I had a problem.

The good thing with the article is that it argues that we should be doing more with games to support learning, and I couldn’t agree more.  When properly designed, games provide deep and meaningful practice.  And we could be tapping into much more of the facets of games for designing learning experiences. Challenge, decisions, and consequences in a safe environment.

So what bothered me?  At one point, the article does on about what skills are required in computer games, things like problem-solving, strategy, etc.  And, yes, games do require those skills. However, what many have done wrongly is say that the games develop those skills, and this is wrong.  For instance, when Kurt Squire was touting the learning outcomes of Civilization, it came from a teacher who scaffolded that understanding, not intrinsically from the game. Similarly, when my kids were playing Pajama Sam (a great series of games with interesting stories and appropriate challenges), we were scaffolding the learning.

For some, requiring skills will develop them. For the 10% or so who survive despite what we do to them ;).  But if you want to be sure they’re getting developed, you need to do more than require them, you need to scaffold them. And we could do this if we wanted to.  But we don’t. The existence of coaching for higher-level learning skills in the game environment is essentially non-existent. And I just think this is a shame. (Many years ago I was proposing research to develop a coaching environment on top of a game engine, so it could be available in any game designed with that engine, but of course it was deemed too ambitious.  Hmmph.)

And don’t get me wrong, the article didn’t make wrong statements, it just reminded me of the problem that has bugged me and also I think damaged the industry (think: why is the term ‘edutainment’ tainted?).  But we need to be careful what we say and how we talk about it. We can develop meaningful learning games, but we have to know how to do it, not just put game and instructional designers in a room together and expect them to know how to create a success.  You need to understand the alignment of elements of learning and leverage those to achieve success.  Don’t settle for less.

14 November 2013

Really Useful eLearning Instruction Manual

Clark @ 7:02 am

Rob Hubbard organized a suite of us to write chapters for a use-focused guide to elearning. And, now it’s out and available!  Here’s the official blurb:

Technology has revolutionised every aspect of our lives and how we learn is no exception. The trouble is; the range of elearning technologies and the options available can seem bewildering. Even those who are highly experienced in one aspect of elearning will lack knowledge in some other areas. Wouldn’t it be great if you could access the hard-won knowledge, practical guidance and helpful tips of world-leading experts in these fields? Edited by Rob Hubbard and featuring chapters written by global elearning experts: Clive Shepherd, Laura Overton, Jane Bozarth, Lars Hyland, Rob Hubbard, Julie Wedgwood, Jane Hart, Colin Steed, Clark Quinn, Ben Betts and Charles Jennings – this book is a practical guide to all the key topics in elearning, including: getting the business on board, building it yourself, learning management, blended, social, informal, mobile and game-based learning, facilitating online learning, making the most of memory and more.

And here’s the Table of Contents, so you can see who wrote what:

  1. So What is eLearning? – Clive Shepherd
  2. Getting the Business on Board – Laura Overton
  3. Build In-House, Buy Off -the-Shelf or Outsource? - Jane Bozarth
  4. Production Processes – Making it Happen! - Lars Hyland
  5. Making the Most of Memory - Rob Hubbard
  6. Blended Learning - Julie Wedgwood
  7. Informal and Social Learning - Jane Hart
  8. Facilitating Live Online Learning - Colin Steed
  9. Mobile Learning - Clark Quinn
  10. Game-Based Learning - Ben Betts
  11. Learning Management - Charles Jennings

If you’d like to purchase the book, VBF11 is the promotion code to get 15% discount when you buy the book at www.wiley.com, or you can get it through Amazon as a book or on Kindle.  I look forward to getting my copy in the mail!

18 June 2013

Christopher Pirie #mlearncon Keynote Mindmap

Clark @ 10:15 am

Christopher Pirie opened the eLearning Guild’s mLearnCon mobile learning conference with a fair overview of technology for learning.  He talked about the usual trends, and pointed to some interesting game apps for learning.  Kodu, in particular, is an interesting advancement on things like Scratch and StageCast’s Creator.

I was somewhat surprised by his pointer to Bloom as the turning point to modern learning design, as I’d be inclined to point more to Collins & Brown’s Cognitive Apprenticeship. I also think he should take a look at Donald Clark’s criticisms of Mitra’s Hole in the Wall.  Finally, the characterization between the overhead projector as characteristic of 1991 and the Kinect for 2012 is a bit spurious: in 1991 we also had HyperCard, and in 2012 I don’t see the Kinect in many classrooms yet, but his point is apt about the potential for change we have at our fingertips.

Overall, a nice kickoff for the conference.


9 May 2013

Assessing online assessments

Clark @ 5:00 am

Good formal learning consists of an engaging introductions, rich presentation of concepts, annotated examples, and meaningful practice, all aligned on cognitive skills. As we start seeing user-generated online c, publishers and online schools are feeling the pressure. Particularly as MOOCs come into play, with (decreasingly) government funded institutions giving online content and courses for free. Are we seeing the demise of for-profit institutions and publishers?

I will suggest that there’s one thing that is harder to get out of the user-generated content environment, and that’s meaningful practice. I recall hearing of, but haven’t yet seen, a seriously threatening repository of such. Yes, there are learning object repositories, but they’re not yet populated with a rich suite of contextualized practice.

Writing good assessments is hard. Principles of good practice include meaningful decisions, alternatives that represent reliable misconceptions, relevant contexts, believable dialog, and more. They must be aligned to the objectives, and ideally have an increasing level of challenge.

There are some technical issues as well. Extensions that are high value include problem generators and randomness in the order of options (challenging attempts to ‘game’ the assessment). A greater variety of response options for novelty isn’t bad either, and automarking is desirable for at least a subset of assessment.

I don’t want to preclude essays or other interpretive work like presentations or media content, and they are likely to require human evaluation, even with peer marking. Writing evaluation rubrics is also a challenge for untrained designers or experts.

While SMEs can write content and even examples (if they get pedagogical principles and are in touch with the underlying thinking, but writing good assessments is another area.

I’ve an inkling that writing meaningful assessments, particularly leveraging interactive technology like immersive simulation games, is an area where skills are still going to be needed. Aligning and evaluating the assessment, and providing scrutable justification for the assessment attributes (e.g. accreditation) is going to continue to be a role for some time.

We may need to move accreditation from knowledge to skills (a current problem in many accreditation bodies), but I think we need and can have a better process for determining, developing, and assessing certain core skills, and particularly so-called 21st century skills. I think there will continue to be a role for doing so, even if we make it possible to develop e necessary understanding in any way the learner chooses.

As is not unusual, I’m thinking out loud, so I welcome your thoughts and feedback.

17 April 2013

Sach’s Winning the Story Wars

Clark @ 6:47 am

On a recommendation, I’ve been reading Jonah Sach’s Winning the Story Wars.   While it’s ostensibly about marketing/advertising, which interests me not, I was intrigued by the possibilities to understand stories from a different perspective.   I was surprised to find that it offered much more.

The book does cover the history of advertising, going through some classic examples of old-style advertising, and using some surprisingly successful examples to elicit a new model.  Some personal stories and revelations make this more than a conceptual treatise.

The core premise is turning your customer into a potential hero of an important journey.  You play the role of the mentor, providing the magic aid for them to accomplish a goal that they know they need, but for a variety of reasons may have avoided.  The journey is motivated from core values, a feature that resonates nicely with my personal quest for using technology to facilitate wisdom.

The book also provides, as one of the benefits, a nice overview of story, particularly the hero’s journey as synthesized by Joseph Campbell across many cultures and time periods.  If you find Campbell a tough read, as many do, this is a nicely digested version.  It talks in sensible ways about the resistance, and trials, and ultimate confrontation.

The obvious focus is on new way to build your brand, tapping into higher purpose, not the more negative fears of inadequacy.  So this book is valuable for those looking to market in a higher way.  And I do intend to rethink the Quinnovation site as a consequence.  But I suggest there’s more.

The notion of the individual being offered the opportunity to play a transformative role seems to be a useful framing for learning. We can, and should, be putting learners in meaningful practice roles, and those roles can be coming from learners’ deep motivators. One of the heuristics in learning game design is Henry Jenkins’ “put the player in a role they’d like to be in”.  This provides a deeper grounding, put the learner in a role they aspire to be in.

I think this book provides not only practical marketing advice, but also guidance for personal journeys and learning.  I think that the perspective of designing stories and roles that are based on personal values to be a great opportunity to do better design. I haven’t completely finished it yet, but I’ve already found enough value in the majority of it to recommend it to you.

8 April 2013

Games & Meaningful Interactivity

Clark @ 6:35 am

A colleague recently queried: “How would you support that Jeopardy type games (Quizzes, etc.) are not really games?”  And while I think I’ve discussed this before, I had a chance to noodle on it on a train trip.  I started diagramming, and came up with the following characterization.

GameSpacesI separated out two dimensions. The first  is differentiating between knowledge and skills.  I like how Van Merriënboer talks about the knowledge you need and the complex problems you apply that knowledge to.  Here I’m separating ‘having’ knowledge from ‘using’ knowledge, focusing on application.  And, no surprise, I’m very much on the side of using, or doing, not just knowing.

The second dimension is whether the learning is essentially very true to life, or exaggerated in some way.  Is it direct, or have we made some effort to make it engaging?

Now, for rote knowledge, if we’re contextualizing it, we’re making it more applied (e.g. moving to the skills side), so really what we have to do is use extrinsic motivation.  We gamify knowledge test (drill and kill) and make it into Jeopardy-style quiz shows.   And while that’s useful in very limited circumstances, it is not what we (should) mean by a game.  Flashy rote drill, using extrinsic motivation, is a fall-back, a tactic of last resort.  We can do better.

What we should mean by a game is to take practice scenarios and focus on ramping up the intrinsic motivation, tuning the scenario into a engaging experience.  We can use tools like exaggeration, humor, drama, and techniques from game design, literature, and more, to make that practice more meaningful.  We align it with the learners interests (and vice-versa), making the experience compelling.

Because, as the value chain suggests, tarting up rote knowledge (which is useful if that’s what we need, and sometimes it’s important, e.g. medical terminology) is better than not, but not near as valuable as real practice via scenarios, and even better if we tune it into a meaningful experience.  Too often we err on the side of knowledge instead of skills, because it’s easy, because we’re not getting what we need from the SME, because that’s what our tools do, etc, but we should be focusing on skills, because that’s what’s going to make a difference to our learners and ultimately our organizations.

What we should do is be focusing on better able to do, moving to the skill side. Tarted up quiz shows are not really games, they’re simplistic extrinsic response trainers.  Real, serious, games translate what Sid Maier said about games – “a series of interesting decisions” – into a meaningful experience: a series of important decisions.  Practicing those are what will make the difference you care about.

21 March 2013

Signs of hope?

Clark @ 6:32 am

Attending the SolutionsFest at the Learning Solutions conference last week, despite my earlier rant, I saw signs of hope.  A quick spot check revealed a number of approaches going above and beyond.

One direction I was pleased to see was a move to more performance support. I saw several solutions that were more focused on providing information as needed, or letting you  navigate to it, rather than thinking everything had to be ‘in the head’. This is definitely a promising sign.  They’re not hard to build, either.

The second promising sign was the use of scenarios. Several different solutions were focused on putting learners into contexts and asking them to perform. This is definitely the direction we need to see more of.  And, again, it’s not that hard to do!

One interesting takeaway was that the innovative solutions seemed to come more from small or internal groups rather than the big teams.  Which only reinforces my overall concern with the industry as a whole.  I wonder if it’s easier for small teams to adapt to advice of folks like Michael Allen (no more boring elearning), Julie Dirksen (Design for How People Learn) and Will Thalheimer than it is for big teams, who not only have to change processes but also educate their customers.

This is an unscientific sample; I did a quick tour of the displays, but couldn’t see all as there were some that were just too crowded.  I also looked at them relatively briefly and didn’t make comprehensive notes, so this is just a read of my state of mind as I finished.  It doesn’t ameliorate the overall concern, but it does provide some hope that things are changing in small pockets.

14 March 2013

Daniel Coyle #LSCon Keynote Mindmap

Clark @ 7:12 am

Daniel Coyle gave a wonderfully funny, passionate, and poignant keynote, talking about what leads to top performance. Naturally, I was thrilled to hear him tout the principles that I suggest make games such a powerful learning environment: challenge, tight feedback, and large amounts of engaging practice. With compelling stories to illustrate his points, he balanced humor and emotional impact to sell a powerful plea for better learning.


Next Page »

Powered by WordPress