Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

31 July 2012

Levels of eLearning Quality

Clark @ 5:34 AM

Of late, I’ve been both reviewing eLearning, and designing processes & templates. As I’ve said before, the nuances between well-designed and well produced eLearning are subtle, but important. Reading a forthcoming book that outlines the future but recounts the past, it occurs to me that it may be worthwhile to look at a continuum of possibilities.

For the sake of argument, let’s assume that the work is well-produced, and explore some levels of differentiation in quality of the learning design. So let’s talk about a lack of worthwhile objectives, lack of models, insufficient examples, insufficient practice, and lack of emotional connection.  These combine into several levels of quality.

The first level is where there aren’t any, or aren’t good learning objectives. Here we’re talking about waffly objectives like ‘understand’, ‘know’, etc. Look, I’m not a behaviorist, but I think *when* you have formal learning goals (and that’s not as often as we deliver), you bloody well ought to have some pretty meaningful description around it.  Instead what we see is the all-to-frequently observed knowledge dump and knowledge test.

Which, by the way, is a colossal waste of time and money.  Seriously, you are, er, throwing away money if that’s your learning solution. Rote knowledge dump and test reliably lead to no meaningful behavior change.  We even have a label for it in cognitive science: “inert knowledge”.

So let’s go beyond meaningless objectives, and say we are focused on outcomes that will make a difference. We’re ok from here, right? Er, no.  Turns out there are several different ways we can go wrong.  The first is to focus on rote procedures. You may want execution, but increasingly the situation is such that the decisions are too complex to trust a completely prescribed response. If it’s totally predictable, you automate it!

Otherwise, you have two options; you provide sufficient practice, as they do with airline plots and heart surgeons. If lives aren’t on the line and failure isn’t as expensive as training, you should focus on providing model-based instruction where you develop the performer’s understanding of what’s underlying the decisions of how to respond.  That latter gives you a basis for reconstructing an appropriate response even if you forget the rote approach.   I recommend this in general, of course.

Which brings up another way learning designs go wrong.  Sufficient practice as mentioned above would suggest repeating until you can’t get it wrong.  What we tend to see, however, is practice until you get it right. And that isn’t sufficient.  Of course, I’m talking real practice, not knowledge test ala multiple choice questions. Learners need to perform!

We don’t see sufficient examples, either. While we don’t want to overwhelm our learners, we do need sufficient contexts to abstract across. And it does not have to occur in just one day, indeed, it shouldn’t!  We need to space the learning out for anything more than the most trivial of learning. Yet the ‘event’ model of learning crammed into one session is much of what we see.

The final way many designs fails is to ignore the emotional side of the equation.  This manifests itself in several ways, including introductions, examples, and practice.  Too often, introductions let you know what you’re about to endure, without considering why you should care.  If you’re not communicating the value to the learner, why should they care? I reckon that if you don’t convey the WIIFM, you better not expect any meaningful outcomes.  There are more nuances here (e.g. activating relevant knowledge, etc), but this is the most egregious.

In examples and practice, too, the learner should see the relevance of what is being covered to what they know is important and they care about.  These are two important and separate things.  What they see should be real situations where the knowledge being addressed plays a real role. Then they should also care about the examples personally.

It’s hard to be able to address all the elements, but aligning them is critical to achieving well-designed, not just well-produced learning. Are you really making the necessary distinctions?

27 July 2012

More slides please…

Clark @ 5:45 AM

Really?  Yes.  Let me explain:

I’ve been reviewing some content for a government agency. This is exciting stuff, evaluating whether contract changes are valid.  Ok, it’s not exciting to me, but to the audience it’s important.  And there’s a reliable pattern to the slide deck that the instructor is supposed to use: it’s large amounts of text.

Again, exciting stuff, right from the regulations.  But that’s important to this audience; I actually don’t have a problem with it. The problem is that it’s all crammed on one screen!  Why is this a problem?

It’s not a problem for printing.  You wouldn’t want to waste paper, and trees, printing it out. So being dense in this way isn’t bad. No, it’s bad when it’s presented.

When it’s presented, there is some highlighting of the important things. But if you were to hear someone go over the three wordy bullet points on one screen, you’d be hard pressed to follow.  However, if you spaced the same screen out three times, one for each bullet point, , you’d support cognitive load more appropriately.  You’re using more screens, but covering the same material in the same time, you’re just switching between screens emphasizing the separate points.  And you don’t have to put each bullet point on a separate screen; to help maintain context you could have the same text but only the relevant one clear and the others greyed out or blurred.

Hey, screens are cheap. In fact, they’re essentially free!  Using more screens when presenting doesn’t cost any more.  Really!  You can address each point clearly, maintaining context but helping focus attention.  It’ll help the instructor too, not just the students.

Ok, so there is one cost.  Maintaining a separate deck for printing and projecting could be some extra management overhead.  But for one, who’s better at policies and procedures than the government?  More seriously, I often will have a slide in my deck that’s a prose version of something I convey graphically, e.g. the five slides I use to present Brent Schlenker’s five-ables of social media (findable, feedable, linkable, taggable, editable).  In the presentation I have a slide with an image for each. For print, I hide those five and show the one text one.  It’s not that hard.  The same principle could be used here, the full slide for printing, the three equivalents for presenting.

There are times when you want more slides. They’re simpler, more focused, and better support maintaining context and focus. Don’t scrimp on the slides.  It’s better to have slides with not so much text, but if you must, space it out.

26 July 2012

You know you’re mobile when…

Clark @ 5:22 AM

I was thinking about the different ways you can be mobile, and I think it’s broader than most people think.  So I tried to capture it in a diagram.  For once, I’m not particularly happy with it, but in the spirit of ‘thinking out loud’…

When are you mobile?

The notion is there that you’re mobile when you’re not at your desk with your desktop or even laptop. Now, sometimes you have a laptop with you, but increasingly I think it’ll be tablet or just a pocketable device (and see my earlier distinctions around those, particularly that laptops don’t typically count).  When you’re at your desk, you’re clearly using your desktop or laptop for work, and you’re not mobile.

With the caveat that if the organization is blocking access to some sites (e.g. any search term like ‘game’ or social media site like Facebook and Twitter), you’re highly likely to use your  mobile device to get around this. Rightly so, I must say. Increasingly your network is part of your brain and your solution set, and anyone who’d block it is keeping you from being as effective as possible. If they’re worried about, or you really aren’t using it for work purposes, the problem is not the network.

Now, you can be out of your particular workspace but still in either your own office, a satellite office, or even in someone else’s office (e.g. client or partner’s office), but you’re in an office. You may be having meetings, making a site visit, whatever.  I reckon attending a conference or a workshop is similar.  There you are mobile, unless you’ve lugged your desktop with you (umm, no).  Again, increasingly it’ll be a tablet or a pocketable.

And there’s the particular situation of being ‘on the go’, when you’re actually in motion, in a way station (in a shop, restaurant, coffeeshop, or pub), or even some place where there’s no real seating (factory floor, for example).  There you’re far more likely to be using your pocketable device in opposition to the laptop or tablet.

You’ll still be accessing your social network, too.  More so; you’ll not only getting answers and assistance, but updating people as well.

There are a couple of unique situations.  One is attending a virtual meeting. At your desktop, you’d use it.  When in another context, you can use your laptop or your tablet.  It’s not quite as feasible with a pocketable device (though that will change).  Your mobile, but your part of an out-of-context or virtual context event, so it’s conceptually distinct, though practically it may not be.

The other is context-specificity. If the device is doing something unique because of where or when you are, it’s really a different situation than accessing just any content or capability you need.  Particularly if the interaction is context-specific.  And capturing  your context, with media, really is a different category.

The point I’m trying to make is that, particularly in the middle category, mobile is more ubiquitous than you think. You know you’re mobile when you’re not at your desk.  And that’s an increasing amount of the time for most people.  Which is healthier anyway.

18 July 2012

mLearning 3.0

Clark @ 7:24 AM

Robert Scoble has written about Qualcomm’s announcement of a new level of mobile device awareness. He characterizes the phone transitions from voice (mobile 1.0) to tapping (2.0) to the device knowing what to do (3.0).  While I’d characterize it differently, he’s spot on about the importance of this new capability.

I’ve written before about how the missed opportunity is context awareness, specifically  not just location but time.  What Qualcomm has created is a system that combines location awareness, time awareness, and the ability to build and leverage a rich user profile. Supposedly, according to Robert, it’s also tapped into the accelerometer, altimeter, whatever sensors there are.  It’ll be able to know in pretty fine detail a lot more about where you are and doing.

Gimbal is mostly focused on marketing (of course, sigh), but imagine what we could do for learning and performance support!

We can now know who you are and what you’re doing, so:

  • a sales team member visiting a client would get specialized information different than what a field service tech would get at the same location.
  •  a student of history would get different information at a particular location such as Boston than an architecture student would
  • a person learning how to manage meetings more efficiently would get different support than a person working on making better presentations

I’m sure you can see where this is going.  It may well be that we can coopt the Gimbal platform for learning as well.  We’ve had the capability before, but now it may be much easier by having an SDK available.  Writing rules to take advantage of all the sensors is going to be a big chore, ultimately, but if they do the hard yards for their needs, we may be able to ride on the their coattails for ours.  It may be an instance when marketing does our work for us!

Mobile really is a game changer, and this is just another facet taking it much further along the digital human augmentation that’s making us much more effective in the moment, and ultimately more capable over time.  Maybe even wiser. Think about that.

11 July 2012

A game? Who says?

Clark @ 5:01 AM

I just reviewed a paper submitted to a journal (one way to stay in touch with the latest developments), and all along they were doing research on the cognitive and motivational relationships in the game. They claimed it was a game, and proceeded on that assumption.  And then the truth came out.

When designing and evaluating learning experiences, you really want to go beyond whether it’s effective or easy to use, and decide whether it’s engaging.  Yes, you absolutely need to test usability first (if there’s a problem with the learning outcomes, is it the pedagogy or the interaction?), and then learning effectiveness. But ultimately, if you want it optimally tuned for success, pitched at the optimal learning level using meaningful activities, it should feel like a game.  The business case is that the effectiveness will be optimized, and the tuning process to get there is less than you think (if you’re doing it right).  And the only real way to test it is subjectively: do the players think it’s a game.

If you create a learning experience and call it game, but your learners don’t think it is, you undermine their motivation and your credibility.  It can be relative (e.g. better than regular learning) as you might not have the resources to compete with commercial games, but it ought to be better than having to sit through a page turner, or you’ve failed.

There are systematic ways to design games that achieve both meaningful engagement and effective education practice. Heck, I wrote a whole book on the topic.  It’s not magic, and while it requires tuning, it’s doable. And, as I’ve stated before: you can’t say it’s a game, only your players can tell you that.

So here were these folks doing research on a ‘game’. The punchline: “students, who started playing the game with high enthusiasm, started complaining after a short while, ‘this is not a game’, and stopped gameplay”.  Fail.

Seriously, if you’re going to make a game, make it demonstrably fun. Or it’s not a game, whether you say so or not.

10 July 2012

Emergent & Semantic Learning

Clark @ 7:34 AM

The last of the thoughts still percolating in my brain from #mlearncon finally emerged when I sat down to create a diagram to capture my thinking (one way I try to understand things is to write about them, but I also frequently diagram them to help me map the emerging conceptual relationships into spatial relationships).

Semantic and Emergent rules for contentWhat I was thinking about was how to distinguish between emergent opportunities for driving learning experiences, and semantic ones.  When we built the Intellectricity ™ system, we had a batch of rules that guided how we were sequencing the content, based upon research on learning (rather than hardwiring paths, which is what we mostly do now).  We didn’t prescribe, we recommended, so learners could choose something else, e.g. the next best, or browse to what they wanted.  As a consequence, we also could have a machine learning component that would troll the outcomes, and improve the system over time.

And that’s the principle here, where mainstream systems are now capable of doing similar things.  What you see here are semantic rules (made up ones), explicitly making recommendations, ideally grounded in what’s empirically demonstrated in research.  In places where research doesn’t stipulate, you could also make principled recommendations based upon the best theory.  These would recommend objects to be pulled from a pool or cloud of available content.

However, as you track outcomes, e.g. success on practice, and start looking at the results by doing data analytics, you can start trolling for emergent patterns (again, made up).  Here we might find confirmation (or the converse!) of the empirical rules, as well as potentially  new patterns that we may be able to label semantically, and even perhaps some that would be new.  Which helps explain the growing interest in analytics.  And, if you’re doing this across massive populations of learners, as is possible across institutions, or with really big organizations, you’re talking the ‘big data’ phenomena that will provide the necessary quantities to start generating lots of these outcomes.

Another possibility is to specifically set up situations where you randomly trial a couple alternatives that are known research questions, and use this data opportunity to conduct your experiments. This way we can advance our learning more quickly using our own hypotheses, while we look for emergent information as well.

Until the new patterns emerge, I recommend adapting on the basis of what we know, but simultaneously you should be trolling for opportunities to answer questions that emerge as you design, and look for emergent patterns as well.  We have the capability (ok, so we had it over a decade ago, but now the capability is on tap in mainstream solutions, not just bespoke systems), so now we need the will.  This is the benefit of thinking about content as systems – models and architectures – not just as unitary files.  Are you ready?


3 July 2012

Piecing together collaboration and cooperation: The Coherent Organization

Clark @ 5:48 AM

In an insightful piece, Harold Jarche puts together how collaboration and cooperation are needed to make organizations work ‘smarter’, integrating workgroups with the broader social network by using communities of practice as the intermediary.  This makes a lot of sense to me, and I was inspired to take a look at the practices within those categories.  (Jay Cross has explored different facets of the implications of this way of thinking and talks about how we are building on this.)

Working Collaboratively and cooperativelyIn this depiction,we see behaviors of effective collaboration within work groups, such as coaching each other, using good practices for brainstorming, the elements of a learning organization, being willing to admit to problems, and being willing to lose if you don’t lose the lesson.

At the next level, communities of practice need to continue to evolve their practices, sharing issues and working together to resolve them.  Within these communities, sharing pointers as well as deeper thoughts are mechanisms for ‘stealth mentoring‘ and explicit mentoring is valuable as well.

At the outermost level, social networks are about tracking what’s happening and who knows what, looking for developments in related fields as mechanisms for improving designs, and sharing practice is a way to give back to the community.

At the intersections, you need practices of both sharing outward and bringing inward, always looking for fresh inspiration and valuable feedback. The transparency provides real value in developing trust among the constituencies.

I put reflection underpinning all of these, as a core practice.  Reflection is absolutely critical to continual improvement in every area.

Note that the firewall tends to cross the middle of the diagram, and by blocking access you’re effectively cutting off a portion of the corporate brain!

This should not by any means be considered definitive, as it’s my first draft, but I think it helps (me, at least) think about what practices could accelerate an organization to be both effective and efficient, able to move nimbly to deliver ongoing customer delight by continual innovation while executing as well. We’re thinking about this as the ‘Coherent Organization’, aligning the flows of information, and aligning the work with the organizational goals.   As always, I welcome your feedback: what should be added, removed, modified, etc.

2 July 2012

Stealth mentoring

Clark @ 6:10 AM

I was looking for any previous post I’d made about stealth mentoring, so I could refer to it in a post I was writing, and I couldn’t find it. It’s a concept I refer to often (and have to give credit to my colleague Jay Cross who inspired the thought), so here’s my obligatory place holder.

When someone is thinking and learning ‘out loud’, e.g. putting their deeper reflections on line via, say, a blog (er, like this one, recursively), they’re allowing you to look at where and how their thinking is going.  When they also are leaving a trail of what they think is interesting (e.g. by pointing to things on Twitter or leaving bookmarks at a social bookmarking site), you can put together what’s interesting to them and what their resulting thoughts are, and start seeing the trajectory of their thinking and learning.

In formal learning, we can think of modeling behavior and cognitive annotation, the processes covered in Cognitive Apprenticeship as a development process. In a more informal sense, if you had a leader who shared discussions of their thinking with you, you’d consider that  mentoring.

Similarly, here, with a difference.  If they’re blogging and tweeting, or otherwise leaving tracks of their thinking, they can be mentoring you and not even know it. You’re being a stealth mentee!  So, if you can find interesting people who blog and tweet a lot, and you follow their blogs and tweets, they can be mentors to you!

I strongly recommend this path to self-development. One of the ways to accelerate your own growth, part of your personal knowledge management path, is to mentor folks who represent the type of thinking you believe is interesting and important.  By the way, don’t just consume, interact.  If they say something you don’t understand or disagree with, engage: either you’ll learn, or they will.

And, as an associated caveat, I strongly recommend that you also similarly share your thinking.  You can be not only stealth mentored, but folks who read and comment become actual real mentors for you, shaping your thinking. The feedback I’ve gotten through comments on my blog has been extremely beneficial to improving my own thinking, and I’m very grateful.

I really do think this is an important opportunity for personal self-development, and it’s a benefit of the increasing use of social media. I hope you are practicing learning out loud and leaving traces of what’s interesting you as you wander hither and yon. I think it’s something an app like Tappestry could provide as well, leveraging the Tin Can API, where you might more explicitly see a richer picture of what someone’s doing.  But I’m getting into the weeds here, so I’ll simply point out that there’s an opportunity here. You owe it to others to think and learn out loud, and then can take advantage of others who do so with a clear conscience.

Powered by WordPress