Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

16 February 2017

Tackling the tough stuff

Clark @ 8:07 am

tacklingI was reflecting a wee bit on my books (and writings in general), and realized that there’s somewhat of a gap when I talk about games, and mobile, and more.  And it’s not unconscious, but instead principled, even if it arises somewhat implicitly. So I thought I’d talk briefly about why I tend to focus on the design, and not the practical implementations. Briefly, I think the places we fall short are not in executing, but in conceptualizing. And so I focus on tackling what I think is the tough stuff. I think we need to address the things that are more complex. My claim is that if we understand them, we have a better chance of achieving our goals and delivering the necessary outcomes.

I have stated before that I think we can implement most anything we can conceive, the problem is that our conceptions are limited.  So, I talk about design based on knowing how we think, work, and learn. I think we need these foundations if we’re truly going to realign what we do to actually work.  Frankly, I think we’re working under some misapprehensions (read: myths) that are limiting our ability to succeed.

When I talk about thinking, the myth is that it’s all in our head and logically principled.  It turns out that, instead, our thinking is very biased by circumstance and pre-existing beliefs, and we avoid effortful work.  We trust our instincts in far more circumstances than we should!  Similarly, we distribute our thinking across the world: our tools and representations assist us, and yet we don’t focus enough effort on ensuing that those are effectively designed.  There’s a real possibility for a valuable shift here.

My focus in working is to recognize that it’s not as individual as our business processes would assume. The ‘individual innovator’ myth is busted, and the empirical results are that we get better outputs when we work together. Certainly for innovation and creative work. Yet we isolate our work, assigning individual resources.  Similarly, people work best when given meaningful goals, but instead we micromanage too often. Again, there are big opportunities to improve our outcomes by reviewing our approaches.

And on learning, I’ve railed time and again about what’s not working, and been joined by colleagues in opposition.  We learn through designed action and guided reflection, not information dump and knowledge test.  Yet that’s not what we see. And again I suggest only small changes are needed to have a substantial impact.

So, in my books, I don’t talk about so much about how to build a game, or the ways to implement mobile learning, or social learning tools.  These will change. What you want to get your mind around is about our minds. Then you can design solutions that can be implemented in any  number of ways.  I may not be successful at communicating the solutions, but in general when I speak, run workshops, or yes write, people seem to convey that I’ve had some effect on helping them get a handle on these new approaches.  In addition, figuring out how to apply them is why I’m here.

What I’ve been able to do, successfully across years and organizations, is help align processes, products, services, and more with how our brains work.  And then work within the available resources to create solutions that reflect those insights in innovative and yet practical solutions.  It takes time to develop the type of thinking I want organizations to adopt, but it’s doable, and I’ve worked with a number of organizations to do just that. Taking the time to address the tough stuff is a bit of an effort. I think it’s an investment in success.  It’s doable, so the only real open question is whether you’re ready to make a shift in thinking, that leads to a shift in doing, that leads to a better impact for your organization.  And only you can answer that.

9 February 2017


Clark @ 8:05 am

So yesterday I talked about the value of diagrams, but I thought I’d add a bit about the process of actually creating diagrams. Naturally, I created a diagram about it.

Diagram of diagram designI created this diagram for a session I ran on diagramming a number of years ago.  In that session I talked about our cognitive architecture, why we need models, how diagrams work, properties and design issues, and more. At the end, I proposed a potential process for it.  It captures an ideal picture of how diagramming could work.

So, you need to know the elements of the model you want to relate, identify the relationships, and any dimensions that characterize differences between the elements. Then you have to choose how you’re going to represent them: shape, color, weight, font, and more.

With your elements, you can then place them, connect them, and add the visual coding.  Then, of course, you tune.

This is an ideal process, but in reality it’s much more flexible, at least when I’m creating a model as a way of understanding.  I typically iterate, creating placeholder elements, and moving them around until I think I have the right ones.  Then I go about connecting them to make sure I have the relationships right. Then I work on adding dimensions, and colors, and aligning them, and grouping them, and… Except that I might add some elements, then group, or connect, then add more, and…  it’s a very iterative process.  It’s a creative process that involves lots of experimentation, revision, and more.

Sometimes, I even use a diagram and then realize it’s not working and revise it. So, for instance, I blogged about a representation of social process.  I got some feedback that it wasn’t very clear.  So, I made a second stab at it, and I think it worked better. Certainly, I continue to use it without complaints.

And I’m the first to admit that my diagrams may not look as good as the ones that professional graphic designers could create, but they’re good enough (and OmniGraffle does a good job of making it easy for me to make them up to a standard I think is at least acceptable and useful; it’s probably overkill but I’ve stuck with it for years now).  And that’s the point.  If they help you think better, it’s good enough. If it helps you communicate effectively with someone else, even better.  Diagrams are cognitive tools, offloading conceptual complexity to graphic relationships and visual processing.  And with the complex problems we increasingly face, I reckon the more tools the better.


8 February 2017


Clark @ 8:05 am

One of the things that I feel is a really useful tool in my ongoing learning, in my ‘making sense of the world’ is diagramming.  I find diagrams to be a really powerful way to understand not just elements, but relationships.  And yet it doesn’t feel like diagramming gets enough respect.  So I want to make a case for the diagram.

Language is good. Our brains have evolved to use it. But it has trouble communicating complex relationships.  For an example, once I wrote this:

They found that while subjects would rate the analogies, from best to worst, as literally similar, true analogy, mere appearance, and false analogy, their recall for stories, from best to worst, was literally similarly, mere appearance, true analogy, and false analogy.

Try discerning the important difference!  My PhD advisor kindly pointed out that actually parsing this was hard, and recommended a diagram instead. Here’s a rendition of what resulted:

structure task outcomes diagram

In this case it’s much easier to see how the two differed.  (If you want to find out what’s important in the diagram, I’m happy to talk about analogical reasoning for as long as you can stand it! ;)

The point I’m making is that there are times when diagrams are very useful for communicating.  And, if you’ve followed this blog for a fair amount of time, you’ve seen I use diagrams a lot. I use them to think ‘out loud’, and I think it’s important.  As Larkin & Simon argued in their Cognitive Science article, Why a Diagram is (Sometimes) Worth Ten Thousand Words, diagrams let us map conceptual relationships to spatial ones. And so if I want to understand the conceptual relationships, I start laying out spatially, and adjust until they make sense to me.

And my concern is that we aren’t using this powerful visual tool enough.  Sketchnotes are really nice ways to capture presentations, and depending on the skill of the noter, they may communicate it all, or help recall if you’ve seen it. Similarly, my mindmaps of keynotes capture the flow of the discussion and the relationships (at least as I parsed it), but may only make sense if you heard the talk.

But representing things with diagrams is not only a personal thinking tool, it can be a powerful way to communicate concepts, and that’s an important component of a good learning experience design, providing a conceptual model to guide performance.

So I’m surprised we don’t talk about diagrams more. It may seem hard (certainly trying to create an infographic is harder than it seems, from my experience ;), but there’s some systematicity to it. There are principles, and types of diagrams, and more to explore.  And tools that make it easier (though even Powerpoint or Keynote can be used to make diagrams).  Diagrams aren’t the only visuals that help (c.f. graphs and tables), but they’re an important tool in your thinking toolbox.  I encourage you, as part of your meta-learning toolkit, to play around and get your mind around diagrams. Your thinking, and your learning design, can be better as a consequence.

2 February 2017

Reordering the Serious eLearning Manifesto

Clark @ 8:09 am

So, as you may know (and if you don’t, you should), almost three years ago now, I teamed up with colleagues Michael Allen, Julie Dirksen, & Will Thalheimer (all worth knowing about) and put together the Serious eLearning Manifesto.  And I believe it’s a good thing. But it needs an update.

So, we were (and are) frustrated with what was and is happening under the rubric of eLearning.  Michael was intrigued by the concept of Serious Games, and wondered why we didn’t treat elearning seriously as well. (A rant I’ve made before ;). He came up with the idea of a manifesto, and we agreed to work with him on it.  And we finalized a list of 8 ways in which typical elearning differed from what we call Serious eLearning, and 22 research concepts behind it (drawn from work across decades and around the world, we don’t claim to own it). And we put it out there for free (Michael graciously sponsored it through his company with no attribution).

We don’t claim that these are the only ways that good elearning differs from what’s typically seen, of course, we just feel that these are the eight most serious ones that, if followed, have the biggest impact on your learning outcomes.  It wasn’t easy getting the four of us to agree, and we’ve received quite a few suggestions of how it could be expanded or improved, but we’re comfortable that this is a reasonable stance to take.

And it’s gotten a reasonable amount of attention. We had 30+ ‘trustees’ who put their names to it (and many more worthies would have), as well as sponsorship by the appropriate societies.  We’ve been given opportunities to speak and present about it. And we’ve got an ever-growing list of signatories. People recognize that it’s right, even if it hasn’t gotten the traction we’d like (e.g. everyone making a concerted effort to shift to it since it’s release).


When I explain it to others, I realize that I have a trouble with the ordering.  Most of it’s great, but one element somehow slipped out of position, in my mind.  So I’ve made an attempt to remedy that, reordering the list. I’ve made this as similar to the original graphic as possible, except that I’m not using the right fonts. So sue me.

8 differences between typical elearning and serious elearningWhat’s different is that I’ve grouped Real-World Consequences with Authentic Contexts and Realistic Decisions. The consequences naturally follow from realistic decisions made in authentic contexts. Then we can talk about Spaced Practice and Individualized Challenges. The latter of which, by the way, is the only thing that is (mostly) specific to elearning, otherwise it’s applicable to learning in general. The rest is the same.

So this is the version that I’ll be using, going forward.  I still hope you’ll visit the site, sign on, and work towards it. No one expects you to get all the way right away, but it is the right way to go.  If you need help, I’m happy to assist.


1 February 2017

Other writings

Clark @ 8:04 am

It occurs to me to mention some of the other places you can find my writings besides here (and how they differ ;).  My blog posts are pretty regular (my aim is 2/week), but tend to have ideas that are embryonic or a bit ‘evangelical’. First, I’ve written four books; you can check them out and get sample chapters at their respective sites:

Engaging Learning: Designing e-Learning Simulation Games

Designing mLearning: Tapping Into the Mobile Revolution for Organizational Performance

The Mobile Academy: mLearning For Higher Education

Revolutionize Learning &  Development: Performance and Information Strategy for the Information Age

They’re designed to be the definitive word on the topic, at least at the moment.

I’ve also written or co-written a number of chapters in a variety of books.  The books include The Really Useful eLearning Instruction ManualCreating a Learning Culture, Michael Allen’s eLearning Annual 2009,  and a bunch of academic handbooks (Mobile Learning, Experiential Learning, Wiley Learning Technology ;).  These tend to be longer than an article, with a pretty thorough coverage of whatever topic is on tap.

Then there are articles in a variety of magazines.  These tend to be aggregated thoughts that are longer than a blog post, but not as through as a chapter. In particular, they are things I think need to be heard (or read).  So, my writing has shown up in:


Learning Solutions


The topics vary. (For the eLearnMag ones, you’ll have to search for my name owing to their interface, and they tend to be more like editorials.)

And then there are blog posts for others that are a bit longer than my usual blog post, and close to an article in focus:

The Deeper eLearning series for Learnnovators

A monthly article for Litmos.

These, too, are more like articles in that they’re focused, and deeper than my usual blog post.  For the latter I cover a lot of different topics, so you’re likely to find something relevant there in many different areas.

I’m proud of it all, but for a quick update on a topic, you might be best seeing if there’s a Litmos post on it first.  That’s likely to be relatively short and focused if there is one. And, of course, if it’s a topic you’re interested in advancing in and I can help, do let me know.

19 January 2017

Errors and misconceptions

Clark @ 8:08 am

explosionWhen I was a grad student, my advisor looked a lot at error. HIs particular focus was to prevent it through good interface design.  He characterized them as of two types: slips and mistakes.  Slips are when you have the right intent, but from elements of our architecture end up making the wrong move.  Mistakes are when your intentions are wrong. From the learning perspective, it’s the latter we want to address.

So, I’ve argued that there’s some randomness in our architecture. We’re bad at doing rote things, even when we know what to do.  Athletes, for instance, practice for hours a day, day after day and their competitions are typically decided by who makes the fewest errors.  In fact, we’ve identified and created support tools to minimize these errors, like checklists.  We’re supposed to design systems so they minimize the opportunity for slips. These don’t always work; I was embarassed in my most recent presentation to find a couple of typos. They clearly had come in via auto-correct, but I’d missed them subsequently!  There may well be one in this post, too, some form of slip or another.

More importantly are what are termed ‘mistakes’.  Here we’re starting with the wrong intent.  This, from a learning perspective, is what I term a ‘misconception’.  Here, we’re supposed to be using a particular mental model to guide our performance, but we might not have the right model, or no model and we import a wrong one. Again, we should design to minimize these as well, but we also want to make sure we’ve got the right mental models to begin with.  This happens if we haven’t been given the right one, or activated an irrelevant one inappropriately.

Ideally, we want to identify the most likely ways we go wrong, and then make sure we understand what those are and why. From there, then make them available as options in practice. Then we can remediate them at the moment.  Of course, we’re also supposed to have the right model, and highlight how the model guides performance in examples, and then again through the feedback on both correct  and incorrect performance.

Designing practice that supports making the right decisions in context, and also the opportunity to make the wrong choice and get useful feedback, is a key learning design skill. Quiz questions that just test knowledge aren’t likely to lead to meaningful difference in performance.  Practice where the alternative to the right answer are silly or obvious (unless you really know the model) is a waste of your resources and your learner’s time.  Make practice meaningful. Not doing so is an error, too!

28 December 2016

2016 Reflections

Clark @ 8:07 am

2016 out, 2017 inThis is the last Learnlet for 2016, and so it’s time for some reflections on what has been an ‘interesting’ year.  I’ll admit it’s been rough, what with losing so many people known through popular media. I guess you get to an age where more and more people who’ve you’ve grown up with in one way or another begin to pass on. And of course serious changes nationally and internationally.  But there are some learnings as well.

So, I did a fair bit of speaking in 2016, keynoting conferences in New York and Beijing, as well as more private events live and online. I spoke about mobile learning, deeper learning design, innovation, as well as the L&D revolution.  And, of course, I attended the usual suite of industry conferences, notably the eLearning Guild events and Online Educa.  I also was engaged in a number of consulting engagements, working with folks to deepen their understanding (and mine), to achieve meaningful outcomes.

One learning is the value of travel outside the US.  I actually lived outside the US for 7 years (in Australia), and the perspective of seeing how others live, and looking at the rest of the world (and back at the US) from other perspectives is a valuable grounding.  The view I had of China before my recent trips was quite different than the reality. I can say the same from previous experience with India.  It’s too easy to be insular.  Instead, it’s helpful to be curious.

And that’s an industry comment too.  I continue to talk (e.g. my workshop in Berlin) and write about deeper learning design.  And I continue to evangelize about it (c.f. the Serious eLearning Manifesto with my colleagues, and the recent Future of Work project). And yet, the industry seems to continue on in ignorance.  The tools still reflect more of a focus on content instead of experience, for instance. Things get better, but surprisingly slowly. How long until we start treating learning design with the appropriate respect? We need to get out of our comfort zone!

There are positive signs. My engagements with Learnnovators has demonstrated that at least some folks care about quality. And I had several client engagements specifically focused on better learning design.  There just need to be more efforts in this area. It’s not hard to tweak processes to generate outcomes that not only look like good elearning, but actually have a high likelihood of an impact.

I’ve done a lot of reading this year (most recently The Fifth Discipline, which puts lots of what I’ve learned about organizations into a context).  It amazes me that with robust science at the organizational level as well as the learning science level, we still see so much action in organizations (and society) contrary to what’s demonstrably known. There are positive signs here too, but still too few.  It’s challenging, as it involves crossing discipline and business boundaries, yet the benefits are promising.

And I think the hype about technology improvements are premature. Wearables continue, of course. And VR has reached the stage where it’s easy to experiment.  Yet in each case, we’re still in the stage before standards emerge that will make a real market.  AR and content strategy are still nascent, but there’s much potential.  Fortunately, analytics is seeing a boon from the standardization around xAPI. We need to stick to the core learning affordances of new technology to truly grasp the potential.

Looking forward, I see much opportunity, as implied by the gaps indicated above. There’s real opportunity for improvement in the use of technology to facilitate outcomes. We can do personal and organizational learning better.  We can leverage technology in ways that are closer aligned with how our brains work. As a precursor, we’ll need a broader understanding of cognition, but that’s doable.  I’m happy to help ;).

And let me just add a very heartfelt thanks to those of you who I’ve interacted with, this year and in the past. Whether reading the blog, making comments, engaging on social media, attending sessions or workshops, and of course via engagements, I’m very grateful. I hope to connect with you in the future, in any of the above ways or any other. I continue to learn through and with you, and that’s a gift. Again, thank you.

Goodbye 2016, and here’s to making positive changes in the new year.  May it be your best yet.


15 December 2016

(When) Is pattern-matching enough?

Clark @ 8:09 am

In the course of my research, I came across the project shown here, as represented by the accompanying video.  In the video, they show (and tout) the value of their approach to developing pattern recognition around  mathematics.  Further, they argue that it’s superior to the typical rule presentation and practice. And I can buy that, but with many caveats that I want to explore.

So it’s clear that we learn by abstracting patterns across our experiences. We can provide models that guide, but ultimately it’s the practice that works. An extreme example is chicken-sexing (mentioned in the transcript); determining the gender of new-born chicks.  Here, no one can articulate the rationale, it’s merely done by attempts and correct/incorrect feedback!  And the clear implication is that by having learners do repetitive tasks of looking for patterns, they get better at it.

And, yes, they do.  But the open question is what is the learning benefit of that.  Let’s be clear, there are plenty of times we want that to happen. As I learned during my graduate studies, pilots are largely trained to react before their brains kick in: the speed at which things happen are faster than conscious processing.  When speed and accuracy is important, nay critical, we want patterned responses. And it does work for component skills to more complex ones in well-defined domains.  But…

When we need transfer, and things are complex, and we aren’t needing knee-jerk responses, this doesn’t work.  I would like to train myself to recognize patterns of behavior and ways to deal with them effectively, for instance (e.g. in difficult presentation situations, or negotiations).   On the other hand, in many instances I want to preclude any immediate responses and look for clues, ponder, explore, and more.

The important question is when we want rote performance and when we don’t .  Rote ability to do math component skills I’m willing to accept.  But I fear a major problem with math instruction in schools is about doing math, not about thinking like a mathematician (to quote Seymour Paper).  And I don’t want students to be learning the quadratic equation (one of Roger Schank’s most vivid examples) instead of how math can be used a problem-solving tool. The nuances are subtle, to be sure, but again I’m tired of us treating learning like color-by-numbers instead of the rocket science it should be.

Look, it’s great to find more effective methods, but let’s also be smart about the effective use of them. In my mind, that’s part of learning engineering. And I’m by no means accusing the approach that started this discussion of getting it wrong, this is my own editorial soapbox ;).  There’s much we can and should be doing, and new tools are welcome. But let’s also think about when they make sense.  So, does this make sense?

13 December 2016

Improving design processes

Clark @ 8:02 am

Recently, I had a chance to catch up with a client who’d used my services to review their design processes. Per my approach, I’d generated a report with a large number of suggestions. What happened, not surprisingly, is that a subset of them actually were implemented.  Still, it was gratifying to hear of the changes, and I think they’re worth exploring as an opportunity to show how small changes can yield big improvements.

One of the major areas was to work on how they use SMEs. Instead of just having a contracted expert, they’re moving to work with folks who have ‘on the ground’ expertise to couple with their domain expertise.  I argue for triangulating on the real objectives with several perspectives, and this approaches that ideal.

A second thing was that there was a bit of a ‘waterfall model’, and instead I suggest having collaboration at critical points along the way. In this case, they have moved to more collaboration at an important juncture point, with more roles involved to be more innovative on the possibilities.

Also, in their particular case, they have to abide by certain constraints, such as amount of time.  As a result of looking at the opportunities, they are moving to meeting their requirements by adding more practice, not more content.  This is a big win for the learners, and the learning.

They are also looking at applying their refined understanding of the nuances of learning. This is embryonic, I was told, but they are moving to looking to redesigning their content to better align with what is known about learning science. At this point it may not be instituted in their processes, but it’s already affecting the mindset they bring to the task.

There are a couple of other things they’re beginning, but I reckon these are some big wins for their audience and the outcomes. With only small changes in what they are doing, they are increasing the likelihood of effective learning.  And that’s the point: for any design process, there are inflection points where better outcomes can be achieved with minimal impact on the overall processes. That is, the processes may change, but they’re different, not hugely larger.

There’s much we can and should be doing to improve our learning design processes, for our learners, and the learning. We know what the opportunities are, now we just need to marshal the will to make changes.  Are you ready?

6 December 2016

Aligning Learning

Clark @ 8:09 am

Online Educa logoLast week, at Online Educa in Berlin, I gave a tutorial on deeper elearning as a pre-conference event. In it, I talked about getting more meaningful objectives, writing practice that actually develops meaningful outcomes, and content (concepts & examples) aligned to support effective practice. I also talked about emotional engagement and social learning, before talking about revising design processes to incorporate these deeper elements in an effective and not-too-different approach.  In short, I was talking about aligning our designs, and our design processes, to how we think and learn.

This is something that most organizations should be thinking about.  I was pleasantly surprised that the audience included folks from universities, not-for-profits, and government agencies as well as businesses.  The challenges are different in some respects, but there are shared elements. Education tends to be about long term learning relationships (typically at least a half year to several years), versus the short-term relationships (e.g. an hour to several days) in organizational learning.  Yet the need to respect how our brains work is a continuum. Our brains learn in particular ways that are unaffected by the curricular needs.  Learning solutions for performance and for education both still need to respect our neural and cognitive architecture.

And too little of what we do reflects what we know.  As a recent commenter noted, there’s a conflict between the de-facto practices and what research says.  As she also noted, our tools are also focused on supporting wrong approaches.  It’s not that tools prevent doing meaningful learning, it’s just that you have to get your design right first and then make the tool conform (as opposed to the alternative). And our limitations as designers flow from the same source, our brain, as our limitations as learners.  Thus we need to be as aware of cognition in our designing as in our design.

I’ll be talking about the problems this engenders in a special webinar tomorrow. There’re still a few slots left. If you’re committed to trying to improve your learning design, and you have the resources to do so, this is an opportunity to get started.  There isn’t a lot of pressure yet, but it’s time to be proactive before people start asking questions about the business impact of what we’re doing. What do you think?

Next Page »

Powered by WordPress