Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Archives for July 2012

Levels of eLearning Quality

31 July 2012 by Clark 8 Comments

Of late, I’ve been both reviewing eLearning, and designing processes & templates. As I’ve said before, the nuances between well-designed and well produced eLearning  are subtle, but important. Reading a forthcoming book that outlines the future but recounts the past, it occurs to me that it may be worthwhile to look at a continuum of possibilities.

For the sake of argument, let’s assume that the work is well-produced, and explore some levels of differentiation in quality of the learning design. So let’s talk about a lack of worthwhile objectives, lack of models, insufficient examples, insufficient practice, and lack of emotional connection.  These combine into several levels of quality.

The first level is where there aren’t any, or aren’t good learning objectives. Here we’re talking about waffly objectives like ‘understand’, ‘know’, etc. Look, I’m not a behaviorist, but I think *when* you have formal learning goals (and that’s not as often as we deliver), you bloody well ought to have some pretty meaningful description around it.  Instead what we see is the all-to-frequently observed knowledge dump and knowledge test.

Which, by the way, is  a colossal waste of time and money.  Seriously, you are, er, throwing away money if that’s your learning solution. Rote knowledge dump and test reliably lead to no meaningful behavior change.  We even have a label for it in cognitive science: “inert knowledge”.

So let’s go beyond meaningless objectives, and say we are focused on outcomes that will make a difference. We’re ok from here, right? Er, no.  Turns out there are several different ways we can go wrong.  The first is to focus on rote procedures. You may want execution, but increasingly the situation is such that the decisions are too complex to trust a completely prescribed response. If it’s totally predictable, you automate it!

Otherwise, you have two options; you provide sufficient practice, as they do with airline plots and heart surgeons. If lives aren’t on the line and failure isn’t as expensive as training, you should focus on providing model-based instruction where you develop the performer’s understanding of what’s underlying the decisions of how to respond.  That latter gives you a basis for reconstructing an appropriate response even if you forget the rote approach.   I recommend this in general, of course.

Which brings up another way learning designs go wrong.  Sufficient practice as mentioned above would suggest repeating until you can’t get it wrong.  What we tend to see, however, is practice until you get it right. And that isn’t sufficient.  Of course, I’m talking real practice, not knowledge test ala multiple choice questions. Learners need to perform!

We don’t see sufficient examples, either. While we don’t want to overwhelm our learners, we do need sufficient contexts to abstract across. And it does not have to occur in just one day, indeed, it shouldn’t!  We need to space the learning out for anything more than the most trivial of learning. Yet the ‘event’ model of learning crammed into one session is much of what we see.

The final way many designs fails is to ignore the emotional side of the equation.  This manifests itself in several ways, including introductions, examples, and practice.  Too often, introductions let you know what you’re about to endure, without considering why you should care.  If you’re not communicating the value to the learner, why should they care? I reckon that if you don’t convey the WIIFM, you better not expect any meaningful outcomes.  There are more nuances here (e.g. activating relevant knowledge, etc), but this is the most egregious.

In examples and practice, too, the learner should see the relevance of what is being covered to what they know is important and they care about.  These are two important and separate things.  What they see should be real situations where the knowledge being addressed plays a real role. Then they should also care about the examples personally.

It’s hard to be able to address all the elements, but aligning them is critical to achieving well-designed, not just well-produced learning. Are you really making the necessary distinctions?

More slides please…

27 July 2012 by Clark 1 Comment

Really?  Yes.  Let me explain:

I’ve been reviewing some content for a government agency. This is exciting stuff, evaluating whether contract changes are valid.  Ok, it’s not exciting to me, but to the audience it’s important.  And there’s a reliable pattern to the slide deck that the instructor is supposed to use: it’s  large  amounts of text.

Again, exciting stuff, right from the regulations.  But that’s important to this audience; I actually don’t have a problem with it. The problem is that it’s all crammed on one screen!  Why is this a problem?

It’s  not a problem for printing.  You wouldn’t want to waste paper, and trees, printing it out. So being dense in this way isn’t bad. No, it’s bad when it’s presented.

When it’s presented, there is some highlighting of the important things. But if you were to hear someone go over the three wordy bullet points on one screen, you’d be hard pressed to follow.  However, if you spaced the same screen out three times, one for each bullet point, , you’d support cognitive load more appropriately.  You’re using more screens, but covering the same material in the same time, you’re just switching between screens emphasizing the separate points.  And you don’t have to put each bullet point on a separate screen; to help maintain context you  could have the same text but only the relevant one clear and the others greyed out or blurred.

Hey, screens are cheap. In fact, they’re essentially  free!  Using more screens when presenting doesn’t cost any more.  Really!  You can address each point clearly, maintaining context but helping focus attention.  It’ll help the instructor too, not just the students.

Ok, so there is  one cost.  Maintaining a separate deck for printing and projecting could be some extra management overhead.  But for one, who’s better at policies and procedures than the government?  More seriously, I often will have a slide in my deck that’s a prose version of something I convey graphically, e.g. the five slides I use to present Brent Schlenker’s five-ables of social media (findable, feedable, linkable, taggable, editable).  In the presentation I have a slide with an image for each. For print, I hide those five and show the one text one.  It’s not that hard.  The same principle could be used here, the full slide for printing, the three equivalents for presenting.

There are times when you want more slides. They’re simpler, more focused, and better support maintaining context and focus. Don’t scrimp on the slides.  It’s better to have slides with not so much text, but if you must, space it out.

You know you’re mobile when…

26 July 2012 by Clark 3 Comments

I was thinking about the different ways you can be mobile, and I think it’s broader than most people think.  So I tried to capture it in a diagram.  For once, I’m not particularly happy with it, but in the spirit of ‘thinking out loud’…

When are you mobile?

The notion is there that you’re mobile when you’re not at your desk with your desktop or even laptop. Now, sometimes you have a laptop with you, but increasingly I think it’ll be tablet or just a pocketable device (and see my earlier distinctions around those, particularly that laptops don’t typically count).  When you’re at your desk, you’re clearly using your desktop or laptop for work, and you’re not mobile.

With the caveat that if the organization is blocking access to some sites (e.g. any search term like ‘game’ or social media site like Facebook and Twitter), you’re highly likely to use your  mobile device to get around this. Rightly so, I must say. Increasingly your network is part of your brain and your solution set, and anyone who’d block it is keeping you from being as effective as possible. If they’re worried about, or you really aren’t using it for work purposes, the problem is  not the network.

Now, you can be out of your particular workspace but still in either your own office, a satellite office, or even in someone else’s office (e.g. client or partner’s office), but you’re in an office. You may be having meetings, making a site visit, whatever.  I reckon attending a conference or a workshop is similar.  There you are mobile, unless you’ve lugged your desktop with you (umm, no).  Again, increasingly it’ll be a tablet or a pocketable.

And there’s the particular situation of being ‘on the go’, when you’re actually in motion, in a way station (in a shop, restaurant, coffeeshop, or pub), or even some place where there’s no real seating (factory floor, for example).  There you’re far more likely to be using your pocketable device in opposition to the laptop or tablet.

You’ll still be accessing your social network, too.  More so; you’ll not only getting answers and assistance, but updating people as well.

There are a couple of unique situations.  One is attending a virtual meeting. At your desktop, you’d use it.  When in another context, you can use your laptop or your tablet.  It’s not quite as feasible with a pocketable device (though that will change).  Your mobile, but your part of an out-of-context or virtual context event, so it’s conceptually distinct, though practically it may not be.

The other is context-specificity. If the device is doing something unique because of where or when you are, it’s really a different situation than accessing just any content or capability you need.  Particularly if the interaction is context-specific.  And capturing  your context, with media, really is a different category.

The point I’m trying to make is that, particularly in the middle category, mobile is more ubiquitous than you think. You know you’re mobile when you’re  not at your desk.  And that’s an increasing amount of the time for most people.  Which is healthier anyway.

mLearning 3.0

18 July 2012 by Clark 2 Comments

Robert Scoble has written about Qualcomm’s announcement of a new level of mobile device awareness. He characterizes the phone transitions from voice (mobile 1.0) to tapping (2.0) to the device knowing what to do (3.0).  While I’d characterize it differently, he’s spot on about the importance of this new capability.

I’ve written before about how the missed opportunity is context awareness, specifically  not just location but time.  What Qualcomm has created is a system that combines location awareness, time awareness, and the ability to build and leverage a rich user profile. Supposedly, according to Robert, it’s also tapped into the accelerometer, altimeter, whatever sensors there are.  It’ll be able to know in pretty fine detail a lot more about where you are and doing.

Gimbal is  mostly  focused on marketing  (of course, sigh), but imagine what we could do for learning and performance support!

We can now know who you are and what you’re doing, so:

  • a sales team member visiting a client would get specialized information different than what a field service tech would get at the same location.
  •  a student of history would get different information at a particular location such as Boston than an architecture student would
  • a person learning how to manage meetings more efficiently would get different support than a person working on making better presentations

I’m sure you can see where this is going.  It may well be that we can coopt the Gimbal platform for learning as well.  We’ve had the capability before, but now it may be much easier by having an SDK available.  Writing rules to take advantage of all the sensors is going to be a big chore, ultimately, but if they do the hard yards for their needs, we may be able to ride on the their coattails for ours.  It may be an instance when marketing does our work for us!

Mobile really is a game changer, and this is just another facet taking it much further along the digital human augmentation that’s making us much more effective in the moment, and ultimately more capable over time.  Maybe even wiser. Think about that.

A game? Who says?

11 July 2012 by Clark 1 Comment

I just reviewed a paper submitted to a journal (one way to stay in touch with the latest developments), and all along they were doing research on the cognitive and motivational relationships in the game. They claimed it was a game, and proceeded on that assumption.  And then the truth came out.

When designing and evaluating learning experiences, you really want to go beyond whether it’s effective or easy to use, and decide whether it’s engaging.  Yes, you absolutely need to test usability first (if there’s a problem with the learning outcomes, is it the pedagogy or the interaction?), and then learning effectiveness. But ultimately, if you want it optimally tuned for success, pitched at the optimal learning level using meaningful activities, it should feel like a game.  The business case is that the effectiveness will be optimized, and the tuning process to get there is less than you think (if you’re doing it right).  And the only real way to test it is subjectively: do the players think it’s a game.

If you create a learning experience and call it game, but your learners don’t think it is, you undermine their motivation and your credibility.  It can be relative (e.g. better than regular learning) as you might not have the resources to compete with commercial games, but it ought to be better than having to sit through a page turner, or you’ve failed.

There are systematic ways to design games that achieve both meaningful engagement and effective education practice. Heck, I wrote a whole  book  on the topic.  It’s not magic, and while it requires tuning, it’s doable. And, as  I’ve stated before:  you can’t say it’s a game, only your players can tell you that.

So here were these folks doing research on a ‘game’. The punchline: “students, who started playing the game with high enthusiasm, started complaining after a short while, ‘this is not a game’, and stopped gameplay”.  Fail.

Seriously, if you’re going to make a game, make it demonstrably fun. Or it’s not a game, whether you say so or not.

Emergent & Semantic Learning

10 July 2012 by Clark 2 Comments

The last of the thoughts still percolating in my brain from #mlearncon finally emerged when I sat down to create a diagram to capture my thinking (one way I try to understand things is to write about them, but I also frequently diagram them to help me map the emerging conceptual relationships into spatial relationships).

Semantic and Emergent rules for contentWhat I was thinking about was how to distinguish between emergent opportunities for driving learning experiences, and semantic ones.  When we built the Intellectricity© system, we had a batch of rules that guided how we were sequencing the content, based upon research on learning (rather than hardwiring paths, which is what we mostly do now).  We didn’t prescribe, we recommended, so learners could choose something else, e.g. the next best, or browse to what they wanted.  As a consequence, we also could have a machine learning component that would troll the outcomes, and improve the system over time.

And that’s the principle here, where mainstream systems are now capable of doing similar things.  What you see here are semantic rules (made up ones), explicitly making recommendations, ideally grounded in what’s empirically demonstrated in research.  In places where research doesn’t stipulate, you could also make principled recommendations based upon the best theory.  These would recommend objects to be pulled from a pool or cloud of available content.

However, as you track outcomes, e.g. success on practice, and start looking at the results by doing data analytics, you can start trolling for emergent patterns (again, made up).  Here we might find confirmation (or the converse!) of the empirical rules, as well as potentially  new patterns that we may be able to label semantically, and even perhaps some that would be new.  Which helps explain the growing interest in analytics.  And, if you’re doing this across massive populations of learners, as is possible across institutions, or with really big organizations, you’re talking the ‘big data’ phenomena that will provide the necessary quantities to start generating lots of these outcomes.

Another possibility is to specifically set up situations where you randomly trial a couple alternatives that are known research questions, and use this data opportunity to conduct your experiments. This way we can advance our learning more quickly using our own hypotheses, while we look for emergent information as well.

Until the new patterns emerge, I recommend adapting on the basis of what we know, but simultaneously you should be trolling for opportunities to answer questions that emerge as you design, and look for emergent patterns as well.  We have the capability (ok, so we had it over a decade ago, but now the capability is on tap in mainstream solutions, not just bespoke systems), so now we need the will.  This is the benefit of thinking about content as systems – models and architectures – not just as unitary files.  Are you ready?

 

Piecing together collaboration and cooperation: The Coherent Organization

3 July 2012 by Clark 8 Comments

In an insightful piece, Harold Jarche puts together how collaboration and cooperation are needed to make organizations work ‘smarter’, integrating workgroups with the broader social network by using communities of practice as the intermediary.  This makes a lot of sense to me, and I was inspired to take a look at the practices within those categories.  (Jay Cross has explored different facets of the implications of this way of thinking and talks about how we are building on this.)

Working Collaboratively and cooperativelyIn this depiction,we see behaviors of effective collaboration within work groups, such as coaching each other, using good practices for brainstorming, the elements of a learning organization, being willing to admit to problems, and being willing to lose if you don’t lose the lesson.

At the next level, communities of practice need to continue to evolve their practices, sharing issues and working together to resolve them.  Within these communities, sharing pointers as well as deeper thoughts are mechanisms for ‘stealth mentoring‘ and explicit mentoring is valuable as well.

At the outermost level, social networks are about tracking what’s happening and who knows what, looking for developments in related fields as mechanisms for improving designs, and sharing practice is a way to give back to the community.

At the intersections, you need practices of both sharing outward and bringing inward, always looking for fresh inspiration and valuable feedback. The transparency provides real value in developing trust among the constituencies.

I put reflection underpinning all of these, as a core practice.  Reflection is absolutely critical to continual improvement in every area.

Note that the firewall tends to cross the middle of the diagram, and by blocking access you’re effectively cutting off a portion of the corporate brain!

This should not by any means be considered definitive, as it’s my first draft, but I think it helps (me, at least) think about what practices could accelerate an organization to be both effective and efficient, able to move nimbly to deliver ongoing customer delight by continual innovation while executing as well. We’re thinking about this as the ‘Coherent Organization’, aligning the flows of information, and aligning the work with the organizational goals.   As always, I welcome your feedback: what should be added, removed, modified, etc.

Stealth mentoring

2 July 2012 by Clark 3 Comments

I was looking for any previous post I’d made about stealth mentoring, so I could refer to it in a post I was writing, and I couldn’t find it. It’s a concept I refer to often (and have to give credit to my colleague Jay Cross who inspired the thought), so here’s my obligatory place holder.

When someone is thinking and learning ‘out loud’, e.g. putting their deeper reflections on line via, say, a blog (er, like this one, recursively), they’re allowing you to look at where and how their thinking is going.  When they also are leaving a trail of what they think is interesting (e.g. by pointing to things on Twitter or leaving bookmarks at a social bookmarking site), you can put together what’s interesting to them and what their resulting thoughts are, and start seeing the trajectory of their thinking and learning.

In formal learning, we can think of modeling behavior and cognitive annotation, the processes covered in Cognitive Apprenticeship as a development process. In a more informal sense, if you had a leader who shared discussions of their thinking with you, you’d consider that  mentoring.

Similarly, here, with a difference.  If they’re blogging and tweeting, or otherwise leaving tracks of their thinking, they can be mentoring you and not even know it. You’re being a stealth mentee!  So, if you can find interesting people who blog and tweet a lot, and you follow their blogs and tweets, they can be mentors to you!

I strongly recommend this path to self-development. One of the ways to accelerate your own growth, part of your personal knowledge management path, is to mentor folks who represent the type of thinking you believe is interesting and important.  By the way, don’t just consume, interact.  If they say something you don’t understand or disagree with, engage: either you’ll learn, or they will.

And, as an associated caveat, I strongly recommend that you also similarly share your thinking.  You can be not only stealth mentored, but folks who read and comment become actual real mentors for you, shaping your thinking. The feedback I’ve gotten through comments on my blog has been extremely beneficial to improving my own thinking, and I’m very grateful.

I really do think this is an important opportunity for personal self-development, and it’s a benefit of the increasing use of social media. I hope you are practicing learning out loud and leaving traces of what’s interesting you as you wander hither and yon. I think it’s something an app like Tappestry could provide as well, leveraging the Tin Can API, where you might more explicitly see a richer picture of what someone’s doing.  But I’m getting into the weeds here, so I’ll simply point out that there’s an opportunity here. You owe it to others to think and learn out loud, and then can take advantage of others who do so with a clear conscience.

Clark Quinn

The Company

Search

Feedblitz (email) signup

Never miss a post
Your email address:*
Please wait...
Please enter all required fields Click to hide
Correct invalid entries Click to hide

Pages

  • About Learnlets and Quinnovation

The Serious eLearning Manifesto

Manifesto badge

Categories

  • design
  • games
  • meta-learning
  • mindmap
  • mobile
  • social
  • strategy
  • technology
  • Uncategorized
  • virtual worlds

License

Previous Posts

  • June 2025
  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2008
  • September 2008
  • August 2008
  • July 2008
  • June 2008
  • May 2008
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • May 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006

Amazon Affiliate

Required to announce that, as an Amazon Associate, I earn from qualifying purchases. Mostly book links. Full disclosure.

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.Ok