Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

23 June 2015

The Learning Styles Zombie

Clark @ 7:37 am

It’s June, and June is Learning Styles month for the Debunker’s Club.  Now, I’ve gone off on Learning Styles before (here, here, here, and here), but  it’s been a while, and they refuse to die. They’re like zombies, coming to eat your brain!

Let’s be clear, it’s patently obvious learners differ.  They differ in how they work, what they pay attention to, how they like to interact, and more. Surely, it make sense to adapt the learning to their style, so that we’re optimizing their outcome, right?

Er, no.  There is no consistent evidence that adapting to learning styles works.  Hal Pashler and colleagues, on a study commissioned by Science in the Public Interest (read: a non-partisan, unbiased, truly independent work) found (PDF) that there was no evidence that adapting to learning styles worked. They did a meta-analysis of the research out there, and concluded this with statistical rigor.  That is, some studies showed positive effects, and some showed negative, but across the body of studies suitably rigorous to be worth evaluating, there was no evidence that trying to adapt learning to learner characteristics had a definitive impact.

At least part of the problem is that the instruments people use to characterize learning styles are flawed.  Surely, if learners differ, we can identify how?  Not with psychometric validity (that means tests that stand up to statistical analysis). A commissioned study in the UK (like the one above, independent, etc) led by Coffield evaluated a representative sample of instruments (including the ubiquitous MBTI, Kolb, and more), and found (PDF) only one that met all four standards of psychometric validity. And that one was a simple one of one dimensions.

So, what’s a learning designer to do?  Several things: first, design for what is being learned. Use the best learning design to accomplish the goal. Then, if the learner has trouble with that approach, provide help.  Second, do use a variety of ways of supporting comprehension.  The variety is good, even if the evidence to do so based upon learning style isn’t.  (So, for example, 4MAT isn’t bad, it’s just not based upon sound science, and why you’d want to pay to use a heuristic approach when you can do that for free is beyond me.)

Learners do differ, and we want them to succeed. The best way to do that is good learning experience design. We do have evidence that problem-based and emotionally aware learning design helps.  We know we need to start with meaningful objectives, create deep practice, ground in good models, and support with rich examples, while addressing motivation, confidence, and anxiety.  And using different media maintains attention and increases the likelihood of comprehension.  Do good learning design, and please don’t feed the zombie.


17 June 2015

Embrace Plan B

Clark @ 7:56 am

The past two weeks, I’ve been on the road (hence the paucity of posts).  And they’ve been great opportunities to engage around interesting topics, but also have provided some learning opportunities (ahem).  The title of this post, by the way, came from m’lady, who was quoting what a senior Girl Scout said was the biggest lesson she learned from her leader, “to embrace Plan B” ;).

So two weeks ago I was visiting a client working on upping their learning game. This is a challenge in a production environment, but as I discussed many times in posts over the second half of 2014 and some this year, I think there are some serious actions that can be taken.  What is needed are better ways to work with SMEs, better constraints around what makes useful content, and perhaps most importantly what makes meaningful interaction and practice.  I firmly believe that there are practical ways to get serious elearning going without radical change, though some initial hiccups will be experienced.

This past week I spoke twice. First on a broad spectrum of learning directions to a group that was doing distance learning and wanted to take a step back and review what they’d been doing and look for improvement opportunities. I covered deeper learning, social learning, meta-learning, and more. Then I went beyond and talked about 70:20:10, measurement, games and simulations, mlearning, the performance ecosystem, and more.  I then moved on to a separate (and delightful) event in Vancouver to promote the Revolution.

It was the transition between the two events last week that threw me. So, Plan A was to fly back home on Tuesday, and then fly on to Vancouver on Wed morning.   But, well, life happened.  All my flights were delayed (thanks, American) on my flight there and back to the first engagement, and both of the first flights such that I missed the connection. On the way out I just got in later than I expected (leading to 4.5 hours sleep before the long and detailed presentation).  But on the way back, I missed the last connecting flight home.  And this had several consequences.

So, instead of spending Tuesday night in my own bed, and repacking for the next day, I spent the night in the Dallas/Fort Worth airport.  Since they blamed it on weather (tho’ if the incoming flight had been on time, it might’ve gotten out in time to avoid the storm), they didn’t have any obligation to provide accommodation, but there were cots and blankets available. I tried to pull into a dark and quiet place, but most of the good ones were taken already. I found a boarding gate that was out of the way, but it was bright and loud.  I gave up after an hour or so and headed off to another area, where I found a lounge where I could pull together a couple of armchairs and managed to doze for 2.5 or so hours, before getting up and on the hunt for some breakfast.  Lesson: if something’s not working, change!

I caught a flight back home in just enough time to catch the next one up to Vancouver. The problem was, I wasn’t able to swap out my clothes, so I was desperately in need of some laundry.  Upon arriving, I threw one of the shirts, socks, etc into a sink and gave them a wash and hung them up. (I also took a shower, which was not only a necessity after a rough night but a great way to gather myself and feel a bit more human).  The next morning, as I went to put on the shirt, I found a stain!  I couldn’t get up in front of all those people with a stained shirt. Plan B was out the door. Also, the other shirt had acquired one too!  Plan C on the dust heap. Now what?  Fortunately, my presentation was in the afternoon, but I needed to do something.

So I went downstairs and found a souvenir shop in the hotel, but the shirts were all a wee bit too loud.  I didn’t really want to pander to the crowd quite so egregiously. I asked at the hotel desk if there was a place I could buy a shirt within walking distance, and indeed there was.  I was well and truly on Plan D by this time. So I hiked on out to a store and fortunately found another shirt I could throw on.  Lesson: keep changing!

I actually made the story part of my presentation.  I made the point that just like in my case, organizations need not only optimal execution of the plans, but then also the ability to innovate if the plan isn’t working.  And L&D can (and should) play a role in this.  So, help your people be prepared to create and embrace Plan B (and C and…however many adaptations they need to have).

And one other lesson for me: be better prepared for tight connections to go awry!

2 June 2015

Model responses

Clark @ 8:12 am

I was thinking about how to make meaningful practice, and I had a thought that was tied to some previous work that I may not have shared here.  So allow me to do that now.

Ideally, our practice has us performing in ways that are like the ways we perform in the real world.  While it is possible to make alternatives available that represent different decisions, sometimes there are nuances that require us to respond in richer ways. I’m talking about things like writing up an RFP, or a response letter, or creating a presentation, or responding to a live query. And while these are desirable things, they’re hard to evaluate.

The problem is that our technology to evaluate freeform text is difficult, let alone anything more complex.  While there are tools like latent semantic analysis that can be developed to read text, it’s complex to develop and  it won’t work on spoken responses , let alone spreadsheets or slide decks (common forms of business communication).  Ideally, people would evaluate them, but that’s not a very scalable solution if you’re talking about mentors, and even peer review can be challenging for asynchronous learning.

An alternative is to have the learner evaluate themselves.  We did this in a course on speaking, where learners ultimately dialed into an answering machine, listened to a question, and then spoke their responses.  What they then could do was listen to a model response as well as their response.  Further, we could provide a guide, an evaluation rubric, to guide the learner in evaluating their response in respect to the model response (e.g. “did you remember to include a statement and examples”?).

This would work with more complex items, too.  “Here’s a model spreadsheet (or slide deck, or document); how does it compare to yours?”  This is very similar to the types of social processing you’d get in a group, where you see how someone else responded to the assignment, and then evaluate.

This isn’t something you’d likely do straight off; you’d probably scaffold the learning with simple tasks first.  For instance, in the example I’m talking about we first had them recognize well- and poorly-structured responses, then create them from components, and finally create them in text before having them call into the answering machine. Even then, they first responded to questions they knew they were going to get before tasks where they didn’t know the questions.  But this approach serves as an enriching practice on the way to live performance.

There is another benefit besides allowing the learner to practice in richer ways and still get feedback. In the process of evaluating the model response and using an evaluation rubric, the learner internalizes the criteria and the process of evaluation, becoming a self-evaluator and consequently a self-improving learner.  That is, they use a rubric to evaluate their response and the model response. As they go forward, that rubric can serve to continue to guide as they move out into a performance situation.

There are times where this may be problematic, but increasingly we can and should mix media and use technology to help us close the gap between the learning practice and the performance context. We can prompt, record learner answers, and then play back theirs and the model response with an evaluation guide.  Or we can give them a document template and criteria, take their response, and ask them to evaluate theirs and another, again with a rubric.  This is richer practice and helps shift the learning burden to the learner, helping them become self-learners.   I reckon it’s a good thing. I’ll suggest that you consider this as another tool in your repertoire of ways to create meaningful practice. What do you think?

27 May 2015

Attention to connections

Clark @ 8:16 am

A colleague was describing his journey, and attributed much of his success (rightly) to his core skills: including his creativity. I was resonating with his list until I got to ‘attention to detail’, and it got me to thinking.

Attention to detail is good, right?  We want people to sweat the nuances, and I certainly am inspired by folks who do that. But there are times when I don’t want to be responsible for the details. To be sure, these are times when it doesn’t make sense to have me do the details. For example, once I’ve helped a client work out a strategy, the implementation really largely should be on them, and I might take some spot reviews (far better than just helping them start and abandoning them).

So I wondered about what the alternative would be. Now the obvious thought is lack of attention to detail, which might initially be negative, but could there be a positive connotation?  What came to me was attention to connections. That is, seeing how what’s being considered might map to a particular conceptual model, or a related field. Seeing how it’s contextualized, and bringing together solutions.    Seeing the forest, not the trees.

I’m inclined to think that there are benefits to those who see connections, just as there is a need for those who can plug away at the details.  And it’s probably contextual; some folks will be one in one area and another in another.  For example, there are times I’m too detail oriented (e.g. fighting for conceptual clarity), and times where I’m missing connections (particularly in reading the politics of a situation).  And vice-versa, times when I’m not detail-0riented enough, and very good at seeing connections.

They’re probably not ends of a spectrum, either, as I’ve gone away from that in practical matters (hmm, wonder what that implies about the Big 5?). Take introvert and extrovert, from a learning perspective it’s about how well you learn on your own versus how well you learn with others, and you could be good or bad at each or both.  Similarly here, you could be able to do both (as in my colleague, he’s one of the smartest folks I know who is demonstrably innovative and connecting as well as being able to sweat the details whether writing code or composing music).

Or maybe this is all a post-hoc justification for wanting to play out at the conceptual frontier, but I’m not going to apologize for that.  It seems to work…

12 May 2015

A new ‘turn to your neighbor’

Clark @ 7:40 am

So, I was continuing the campaign for the Revolution, and wanted to expand the audience interaction. I could’ve used the tired ‘turn to your neighbor’ technique, but I had a thought (dangerous, that).  Could it be improved upon?

As I may have mentioned, there has been a backlash against ‘brainstorming’. For example, the New York Times had an article about how it didn’t work, saying that if you bring people into a room, and then give them a problem or topic, and then get them to discuss, it won’t work. And they’re right!  Because that is a broken model of brainstorming; it’s a straw man argument.

A real model of brainstorming has the individuals thinking about the problem individually beforehand, before you bring them together. When you have them not have a chance to think independently, the first person to speak colors the thoughts of the others, but if people can come up with their own ideas first, then share and improve, it works well.  The room is smarter than the smartest person in the room, as the quote has it, but the caveat is that you have to manage the process right.

So how does this relate to the ‘turn to your neighbor’?  It occurred to me that a clear implication was that if you thought to yourself first, before sharing, you’d get a better outcome. And so that’s what I did: I had them think for themselves on the question I presented, then share, and then stop.

Now, to be fair, I didn’t have time to ask for all the output, instead I asked who had come up with ‘formal’ for a question on what supports optimal execution, and who came up with facilitating the flow of information as a solution for supporting innovation. So we have practical limits on what we can do with a large audience and a small amount of time.  However, I did ask at the end of the first one whether they thought it worthwhile. And I asked again of a subset of the audience who attended the next day workshop (“Clark Quinn’s workshop on Strategic Elearning is awesome” was a comment, <fist pump>) what they thought.

Overall the feedback was that it was an improvement. Certainly the outputs should be better.  One was “energized”. The overall take of the large audience and the smaller one was very positive.  It doesn’t take much longer, because it’s easy to do the quick thinking bit (and it’s no easier to get them to stop sharing :), but it’s a lesson and an improved technique all in one!

So, now you know that if you see anyone doing just the ‘turn to your neighbor’, they’re not up on the latest research.  Wonder if we can get this to spread?  But continue exploration is a necessary element to improvement, and innovations happen through diligent work and refinement.  Please do try it out and let me know how it goes!  And, of course, even just your thoughts.

6 May 2015

Trojan Mice?

Clark @ 1:15 pm

One of the mantras of the Learning Organization is that there should be experimentation.  This has also become, of course, a mantra of the Revolution as well.  So the question becomes, what sort of experiments should we be considering?

First, for reasons both pragmatic and principled, these are more likely to be small experiments than large.  On principled reasons, even large changes are probably better off implemented as small steps. On pragmatic reasons, small changes can be built upon or abandoned as outcomes warrant.  These small changes have colloquially been labeled ‘trojan mice‘, a cute way to capture the notion of change via small incursions.

The open question, then, is what sort of trojan mice might be helpful in advancing the revolution?  We might think of them in each of the areas of change: formal, performance support, social, culture, etc.  What are some ideas?

In formal, we might, for one, push back on taking orders.  For instance, we might start asking about measures that any initiatives will be intended to address. We could also look to implementing some of the Serious eLearning Manifesto ideas. Small steps to better learning design.

For performance support, one of the first small steps might be to even do performance support, if you aren’t already. If you are, maybe look to broadening the media you use (experiment with a video, an annotated sequence of pictures, or an ebook).  Or maybe try creating a portal that is user-focused, not business-silo structured.

In the social area, you might first have to pilot an exterior social network if there isn’t one. If there is, you might start hosting activities within it.  A ‘share your learning lunch’ might be a fun way to talk about things, and bring out meta-learning.   Certainly, you could start instituting the use within L&D.

And with culture, you might start encouraging people to share how they work; what resources they use.  Maybe film the top performers in a group giving a minute or two talk on how they do what they do.  It’d be great if you could get some of the leadership to start sharing, and maybe do a survey of what your culture actually is.

The list goes on: in tech you might try some microlearning, a mobile experiment, or considering a content model (ok, not actually build one, that’s a big step ;).  In strategy, you might start gathering data about what the overall organization goals are, or what initiatives in infrastructure have been taken elsewhere in the org or are being contemplated.

The point is to start taking some small steps.  So, I’m curious, what small steps have you tried, or what ones might you think of and suggest?

30 April 2015

Activities for Integrating Learning

Clark @ 8:11 am

I’ve been working on a learning design that integrates developing social media skills with developing specific competencies, aligned with real work.  It’s an interesting integration, and I drafted a pedagogy that I believe accomplishes the task.  It draws heavily on the notion of activity-based learning.  For your consideration.

Activity ModelThe learning process is broken up into a series of activities. Each activity starts with giving the learning teams a deliverable they have to create, with a deadline an appropriate distance out.  There are criteria they have to meet, and the challenge is chosen such that it’s within their reach, but out of their grasp.  That is, they’ll have to learn some things to accomplish it.

As they work on the deliverable, they’re supported. They may have resources available to review, ideally curated (and, across the curricula, their responsibility for curating their own resources is developed as part of handing off the responsibility for learning to learn).  There may be people available for questions, and they’re also being actively watched and coached (less as they go on).

Now, ideally the goal would be a real deliverable that would achieve an impact on the organization.  That, however, takes a fair bit of support to make it a worthwhile investment. Depending on the ability of the learners, you may start with challenges that are like but not necessarily real challenges, such as evaluating a case study or working on a simulation.  The costs of mentoring go up as the consequences of the action, but so do the benefits, so it’s likely that the curriculum will similarly get closer to live tasks as it progresses.

At the deadline, the deliverables are shared for peer review, presumably with other teams. In this instance, there is a deliberate intention to have more than one team, as part of the development of the social capabilities. Reviewing others’ work, initially with evaluation heuristics, is part of internalizing the monitoring criteria, on the path to becoming a self-monitoring and self-improving learner. Similarly, the freedom to share work for evaluation is a valuable move on the path to a learning culture.  Expert review will follow, to finalize the learning outcomes.

The intent is also that the conversations and collaborations be happening in a social media platform. This is part of helping the teams (and the organization) acquire social media competencies.  Sharing, working together, accessing resources, etc. are being used in the platform just as they are used for work. At the end, at least, they are being used for work!

This has emerged as a design that develops both specific work competencies and social competencies in an integrated way.  Of course, the proof is when there’s a chance to run it, but in the spirit of working out loud…your thoughts welcome.

23 April 2015

Personal Mobile Mastery

Clark @ 8:29 am

A conversation with a colleague prompted a reflection.  The topic was personal learning, and in looking for my intersections (beyond my love of meta-learning), I looked at my books. The Revolution isn’t an obvious match, nor is games (though trust me, I could make them work ;), but a more obvious match was mlearning. So the question is, how do we do personal knowledge mastery with mobile?

Let’s get the obvious out of the way. Most of what you do on the desktop, particularly social networking, is doable on a mobile device.  And you can use search engines and reference tools just the same. You can find how to videos as well. Is there more?

First, of course, are all the things to make yourself more ‘effective’.  Using the four key original apps on the Palm Pilot for instance: your calendar to remind you of events or to check availability, using ToDo checklists to remember commitments to do something, using memos to take notes for reference, and using your contact list to reach people.  Which isn’t really learning, but it’s valuable to learn to be good at these.

Then we start doing things because of where you are.  Navigation to somewhere or finding what’s around you are the obvious choices. Those are things you won’t necessarily learn from, but they make you more effective.  But they can also help educate you. You can look where you are on a map and see what’s around you, or identify the thing on the map that’s in that direction (“oh, that’s the Quinnsitute” or “There’s Mount Clark” or whatever), and have a chance of identifying a seen prominence.

And you can use those social media tools as before, but you can also use them because of where or when you are. You can snap pictures of something and send it around and ask how it could help you. Of course, you can snap pictures or films for later recollection and reflection, and contribute them to a blog post for reflection.  And take notes by text or audio. Or even sketching or diagramming. The notes people take for themselves at conferences, for instance, get shared and are valuable not just for the sharer, but for all attendees.

Certainly searching things you don’t understand or, when there’s unknown language, seeing if you can get a translation, are also options.  You can learn what something means, and avoid making mistakes.

When you are, e.g. based upon what you’re doing, is a little less developed.  You’d have to have rich tagging around your calendar to signal what it is you’re doing for a system to be able to leverage that information, but I reckon we can get there if and when we want.

I’m not a big fan of ‘learning’ on a mobile device, maybe a tablet in transit or something, but not courses on a phone.  On the other hand, I am a big fan of self-learning on a phone, using your phone to make you smarter. These are embryonic thoughts, so I welcome feedback.   Being more contextually aware both in the moment and over time is a worthwhile opportunity, one we can and should look to advance.  I think there’s  much yet, though tools like ARIS are going to help change that. And that’ll be good.


21 April 2015

Why models matter

Clark @ 7:52 am

In the industrial age, you really didn’t need to understand why you were doing what you were doing, you were just supposed to do it.  At the management level, you supervised behavior, but you didn’t really set strategy. It was only at the top level where you used the basic principles of business to run your organization.  That was then, this is now.

Things are moving faster, competitors are able to counter your advances in months, there’s more information, and this isn’t decreasing.  You really need to be more agile to deal with uncertainty, and you need to continually innovate.   And I want to suggest that this advantage comes from having a conceptual understanding, a model of what’s happening.

There are responses we can train, specific ways of acting in context.  These aren’t what are most valuable any more.  Experts, with vast experience responding in different situations, abstract models that guide what they do, consciously or unconsciously (this latter is a problem, as it makes it harder to get at; experts can’t tell you 70% of what they actually do!).  Most people, however, are in the novice to practitioner range, and they’re not necessarily ready to adapt to changes, unless we prepare them.

What gives us the ability to react are having models that explain the underlying causal relations as we best understand them, and then support in applying those models in different contexts.  If we have models, and see how those models guide performance in context A, then B, and then we practice applying it in context C and D (with model-based feedback), we gradually develop a more flexible ability to respond. It’s not subconscious, like experts, but we can figure it out.

So, for instance, if we have the rationale behind a sales process, how it connects to the customer’s mental needs and the current status, we can adapt it to different customers.  If we understand the mechanisms of medical contamination, we can adapt to new vectors.  If we understand the structure of a cyber system, we can anticipate security threats. The point is that making inferences on models is a more powerful basis than trying to adapt a rote procedure without knowing the basis.

I recognize that I talk a lot in concepts, e.g. these blog posts and diagrams, but there’s a principled reason: I’m trying to give you a flexible basis, models, to apply to your own situation.  That’s what I do in my own thinking, and it’s what I apply in my consulting.  I am a collector of models, so that I have more tools to apply to solving my own or other’s problems.   (BTW, I use concept and model relatively interchangeably, if that helps clarify anything.)

It’s also a sound basis for innovation.  Two related models (ahem) of creativity say that new ideas are either the combination of two different models or an evolution of an existing one.  Our brains are pattern matchers, and the more we observe a pattern, the more likely it will remind us of something, a model. The more models we have to match, the more likely we are to find one that maps. Or one that activates another.

Consequently, it’s also one  of the things I push as a key improvement to learning design. In addition to meaningful practice, give the concept behind it, the why, in the form of a model. I encourage you to look for the models behind what you do, the models in what your presented, and the models in what your learners are asked to do.

It’s a good basis for design, for problem-solving, and for learning.  That, to me, is a big opportunity.

18 November 2014

L&D and working out loud #wolweek

Clark @ 6:15 am

This week is Working Out Loud week, and I can’t but come out in support of a principle that I think is going to be key to organizational success. And, I think, L&D has a key role to play.

The benefits from working out loud are many. Personally, documenting what you’re doing serves as a reminder to yourself and awareness for others. The real power comes, however, from taking that next level: documenting not just what you’re doing, but why. This helps you in reflecting on your own work, and being clear in your thinking. Moreover, sharing your thinking gives you a second benefit in getting others’ input which can really improve the outcome.

In addition, it gives others a couple of benefits. They get to know what you’re up to, so it’s easier to align, but if your thinking is any good, it gives them the chance to learn from how you think.

So what is the role of L&D here? I’ll suggest there are two major roles: facilitating the skills and enabling the culture.

First, don’t assume folks know what working out loud means. And even if they do, they may not be good at it in terms of knowing how to indicate the underlying thinking. And they likely will want feedback and encouragement. First, L&D needs to model it, practicing what they preach. They need to make sure the tools are easily available and awareness is shared. Execs need to be shown the benefit and encouraged to model the behavior too. And L&D will have to trumpet the benefits, accomplishments, and encourage the behavior.

None of this is really likely to succeed if you don’t have a supportive culture. In a Miranda organization, no one is going to share. Instead, you need the elements of a learning organization: the environment has to value diversity, be open to new ideas, provide time for reflection, and most of all be safe. And L&D has to understand the benefits and continue to promote them, identify problems, and work to resolve them.

Note that this is not something you manage or control. The attitude here has to be one of nourishing aka (seed, feed, and weed). You may track it, and you want to be looking for things to support or behaviors to improve, but the goal is to develop a vibrant community of sharing, not squelching anything that violates the hierarchy.

Working out loud benefits the individual and the organization in a healthy environment. Getting the environment right, and facilitating the practice, are valuable contributions, and ones that L&D can, and should, contribute to.


Next Page »

Powered by WordPress