Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

24 November 2015

CERTainly room for improvement

Clark @ 8:08 am

As mentioned before, I’ve become a member of my local Community Emergency Response Team (CERT), as in the case of disaster, the official first-responders (police, fire, and paramedics) will be overwhelmed.  And it’s a good group, with a lot of excellent efforts in processes and tools as well as drills.  Still, of course, there’s  room for improvement.  I encountered one such at our last meeting, and I think it’s an interesting case study.

So one of the things you’re supposed to do in conducting search and rescue is to go from building to building assessing damage and looking for people to help.  And one of the useful things to do is to mark the status of the search and the outcomes, so no one wastes effort on an already explored building. While the marking is covered in training and there’re support tools to help you remember,  ideally it’d be memorable, so that you  can regenerate the information and don’t have to look it up.

The design for the marking is pretty clear: you first make a diagonal slash when you start investigating a building, and then you make a crossing slash when you’ve made your assessment. And specific information is to be recorded in each quarter of the resulting X: left, right, top, and bottom.  (Note that the US standard set by FEMA doesn’t correspond to the international standard from the International Search & Rescue Advisory Group, interestingly).

However, when we brought it up in a recent meeting (and they’re very good about revisiting things that quickly fade from memory), it was obvious that most people couldn’t recall what goes where. And when I heard what the standard was, I realized it didn’t have a memorable structure.  So, here are the four things to record:

  • the group who goes in
  • when the group completes
  • what hazards may exist
  • and how many people and what condition they’re in*

So how would you map these to the quadrants?  And in one sense it doesn’t matter if there’s a sensible rationale behind them. One sign that there’s not?  You can’t remember what goes where.

Our local team leader was able to recall that the order is: left – group, top – completion, right – hazards, and bottom – people.  However, this seems to me to be less than  memorable, so let me explain.

To me, wherever you put the in, left or top, the coming out ought to be opposite. And given our natural flow, group going in makes sense to the left, and coming out ought to go on the right.  In – out.  Then, it’s relatively arbitrary where hazards and people go.  I’d make a case that top-of-mind should be the hazards found to warn others, but that the people are the bottom line (see what I did there?).  I could easily make a case for the reverse, but either would be a mnemonic to support remembering.  Instead, as far as I can tell, it’s completely arbitrary. Now, if it’s not arbitrary and there is a rationale,  it’d help to share that!

The point being, to help people remember things that are in some sense arbitrary, make a story that makes it memorable. Sure, I can look it up, assuming that the lookup book they handed out stays in the pocket in my special backpack.  (And I’m likely to remember now, because of all this additional processing, but that’s not what happens in the training.)  However, making it regenerable from some structure gives you a much better chance of having it to hand. Either a model or a story is better than arbitrary, and one’s possible with a rewrite, but as it is, there’s neither.

So there’s a lesson in design to be had, I reckon, and I hope you’ll put it to use.

* (black or dead, red or needing immediate treatment for life-threatening issues, yellow or needing non-urgent treatment, and green or ok)

13 November 2015

Learning and frameworks

Clark @ 8:13 am

There’s recently been a spate of attacks on 70:20:10 and moving beyond courses, and I have to admit I just don’t get it.  So I thought it’s time to set out why I think these approaches make sense.

Let’s start with what we know about how we learn. Learning is action and reflection.  Instruction (education, training) is designed action and guided reflection.  That’s why, by the way, that information dump and knowledge test isn’t a learning solution.   People need to actively apply the information.

And it can’t follow an ‘event’ model, as learning is spaced out over time. Our brains can only accommodate so much (read: very little) learning at any one time.  There needs to be ongoing facilitation after a formal learning experience – coaching over time and stretch assignments – to help cement and accelerate the learning experience.

Now, this can be something L&D does formally, but at some point formal has to let go (not least for pragmatics) and it becomes the responsibility of the individual and the community. It shifts from formal coaching to informal mentoring, personal exploration, and feedback from colleagues and fellow practitioners.  It’s impractical for L&D to take on this full responsibility, and instead becomes a role in facilitation of mentoring, communication, and collaboration.

That’s where the 70:20:10 framework comes in.  Leaving that mentoring and collaboration to chance is a mistake, because it’s demonstrably the case that people don’t necessarily have good self-learning skills.  And if we foster self-learning skills, we can accelerate the learning outcomes for the organization. Addressing the skills and culture for learning, personally and collectively, is a valuable contribution that L&D should seize. And it’s not about controlling it all, but making an environment that’s conducive, and facilitating the component skills.

Further, some people  seem to get their knickers in a twist about the numbers, and I’m not sure why that is.  People seem comfortable with the Pareto Principle, for instance (aka the 80/20 rule), and it’s the same. In both cases it’s not the exact numbers that matter, but the concept. For the Pareto Rule it’s recognizing that some large fraction of outcomes comes from a small fraction of inputs.  For the 70:20:10 framework, it’s recognizing that much of what you apply as your expertise comes from things other than courses.  And tired old cliches about “wouldn’t want a doctor who didn’t have training” don’t reflect that you’d also not want a doctor who didn’t continue learning through internships and practice.  It’s not denying the 10, it’s augmenting it.

And this is really what Modern Workplace Learning is about: looking beyond the course.  The course is one important, but ultimately small, piece of being a practitioner, and organizations can no longer afford to ignore the rest of the learning picture.  Of course, there’s also the whole innovation side and performance support when learning doesn’t have to happen as well, which is something L&D also should facilitate (cue the L&D Revolution), but getting the learning right by looking at the bigger picture of how we really learn is critical.

I welcome debate on this, but pragmatically if you think about how you learned what you do, you should recognize that much of it came from other than courses. Beyond Education, the other two E’s have been characterized as Exposure and Experience. Doing the task in the company of others, socially learning, and by the outcomes of actually applying the knowledge in context, and making mistakes.  That’s real learning, and the recognition that it should not be left to chance is how these frameworks help raise awareness and provide an opportunity for L&D to become more relevant to the organization.  And that, I strongly believe, is a valuable outcome. So, what do you think?

11 November 2015

Levels of Design

Clark @ 8:03 am

In a recent conversation, we were talking about the Kirkpatrick model, and a colleague had an interesting perspective that hadn’t really struck me overtly. Kirkpatrick is widely (not widely enough, and wrongly) used as an evaluation tool, but he talked about using it as a design tool, and that perspective made clear for me a problem with our approaches.

So, there’s a lot of debate about the Kirkpatrick model, whether it helps or hinders the movement towards good learning. I think it’s misrepresented (including by its own progenitors, though they’re working on that ;), and while I’m open to new tools I think it does a nice job of framing a fairly simple but important idea. The goal is to start with the end in mind.

And the evidence is that it’s not being used well. The largest implementation of the model is level 1, which isn’t of use (correlation between learner reaction and actual impact is .09, essentially zero with a rounding error). Level 2 drops to a third of orgs, and it drops from there. And this is broken.

The point, and this is emphasized by the ‘design’ perspective, is that you are supposed to start with level 4, and work back. What’s the measurable indicator in the organization that isn’t up to snuff, and what behavior (level 3) would likely impact that? And how do we change that behavior (Level 2)? And here’s where it can go beyond training: that intervention might be a job aid, or access to a network (which hasn’t been much in the promotion of the model).

To be fair, the proponents do argue you should be starting at Level 4, but with the numbering (which Don admits he might have got wrong) and the emphasis on evaluation, it doesn’t hit you up front. Using it as a design tool, however, would emphasize the point.

So here’s to thinking of learning design as working backwards from a problem, not forwards from a request. And, of course, to better learning design overall.

10 November 2015

Under the ‘Content’ Cover

Clark @ 8:06 am

Too often I see instructional design training and tools, in addition to talking about ‘objectives’ and ‘assessment’ (which I tend to call ‘practice’, for hopefully obvious reasons), talking about ‘content’. And I think that simplification is a path to bad learning design. It misses emphasizing the nuances, and that’s a bad thing.

What should be the elements of content are an introduction to the learning experience, a presentation of the concept(s), examples that illustrate applying the concept to contexts, and a closing of the experience. Each of these have component parts that, when addressed, contribute to the likelihood of a good learning outcome. Ignoring them, however, is likely to lead to a lack of impact.

The problem is that our cognitive architecture is prone to mistakes in execution. We’re bad at remembering bits and pieces, and we naturally can skip steps. That’s why we create external tools like checklists and templates to support good design. So if we’re not scaffolding here, we run the risk of creating content that may be well-written, but isn’t well-designed.

And we see this all too often: eLearning that’s content-heavy and learning light. It may have good production values, with a consistent look-and-feel, elegant prose, and great images, but it also tends to have too much rote information, little enough concepts, sparse and un-illuminating examples, and no real emotional ‘hook’.

Instead, we could be using checklists or templates to ensure we get the right elements. We could have support for designing introductions, concept, examples, and closing, (and better support for good practice too ;). It doesn’t have to be built into an authoring tool, but certainly should be manifest in the development tools for interim representations.

There are other reasons to be a bit more granular, such as flexible content that supports repurposing for delivery in the moment, and adaptive learning, but overall the real reason is for good design. It doesn’t have to be granular, but it does have to explicitly consider the elements that contribute to learning and get those right. Right?

4 November 2015

A Competent Competency Process

Clark @ 8:10 am

In the process of looking at ways to improve the design of courses, the starting point is good objectives. And as a consequence, I’ve been enthused about the notion of competencies, as a way to put the focus on what people do, not what they know. So how do we do this, systematically, reliably, and repeatably?

Let’s be clear, there are times we need knowledge level objectives. In medicine or any other field where responses need to be quick and accurate, we need a very constrained vocabulary. SO drilling in the exact meanings of words is valuable, as an example. Though ideally, that’s coupled with using that language to set context or make decisions. So “we know it’s the right medial collateral ligament, prep for the surgery” could serve as a context, or we could have a choice to operate on the left or right atrial ventricle as a decision point. As Van Merriënboer’s 4 Component Instructional Design talks about, we need to separate out the knowledge from the complex problems we apply it to. Still, I suggest that what’s likely to make a difference to individuals and organizations is the ability to make better decisions, not recite rote knowledge.

So how do we get competencies when we want them? The problem, as I’ve talked about before, is that SMEs don’t have access to 70% of what they actually do, it’s compiled away. We then need good processes, so I’ve talked to a couple of educational institutions doing competencies, to see what could be learned. And it’s clear that while there’s no turnkey approach, what’s emerging is a process with some specific elements.

One thing is that if you’re trying to cover a whole college level course, you’ve got to break it up. Break down the top level into a handful of competencies. Then you continue to take each of those apart, and perhaps another level, ‘til you have a reasonable scope. This is heuristic, of course, but with a focus on ‘do’, you have a good likelihood to get here.

One of the things I’ve heard across various entities trying to get meaningful objectives is working with more than one SME. If you can get several, you have a better chance of triangulating on the right outcomes and objectives. They may well disagree about the knowledge, but if you manage the process right (emphasize ‘do’, lather, rinse, repeat), you should be able to get them to converge. It may take some education, and you may have to let them get the

Not just any SMEs will do. Two things are really valuable: on the ground experience to know what needs to be done (and doesn’t), and the ability to identify and articulate the models that guide the performance. Some instructors, for instance, can teach to a text but really aren’t truly masters of the content nor are experienced practitioners. Multiple helps, but the better the SME, the better the outcome.

I believe you want to ensure that you’re getting both the right things, and all the things. I’ve recommended to a client about triangulating not just with SMEs, but with practitioners (or, rather, the managers of the roles the learners will be engaged in), and any other reliable stakeholders. The point is to get input from the practice as well as the theory, identifying the models that support proper behavior, and the misconceptions that underpin where they go wrong.

Once you have a clear idea of the things people need to be able to do, you can then identify the language for the competencies. I’m not a fan of Bloom’s (unwieldy, hard to reliably apply), but I am a fan of Mager-style definitions (action, context, metric).

After this is done, you can identify the knowledge needed, and perhaps created objectives for that, but to me the focus is on the ‘do’, the competencies. This is very much aligned with an activity-based learning model, whereby you immediately design the activities that align with the competencies before you decide the content.

So, this is what I’m inferring. There would be good tools and templates you could design to go with this, identifying competencies, misconceptions, and at the same time also getting stories and motivations. (An exercise left for the reader. ;) The overall goal, however, of getting meaningful objectives is key to getting good learning design. Any nuances I’m missing?

3 November 2015

Ho Mee Yin #LearnTech Keynote Mindmap

Clark @ 6:53 pm

LTC Ho Mee Yin told the mstory of rethinking the learning design for the Singapore Armed Forces.  She talked about some new frameworks that helped move to a more enlightened learning design that was more activity-centric, and a performance support tool for instructors.

Nuancing Engagement

Clark @ 8:14 am

I’ve talked in the past about the importance of engaging emotionally before beginning learning. And I’ve talked about the importance of understanding what makes a topic intrinsically interesting. But I haven’t really separated them out, as became clear to me in a client meeting. So let me remedy that here.

I’ve argued, and believe, that we should open up learners emotionally before we address them cognitively. Before we tell them what they’ll learn, before we show them objectives, we should create a visceral reaction, a wry recognition of “oh, yes, I do need to know this”. It can be a dramatic or humorous exaggeration of the positive consequences of having the knowledge or the negative consequences of not. I call this a ‘motivating example’ different than the actual reference examples used to illustrate the model in context. In previous content we’ve used comics to point out the problems of not knowing, and similarly Michael Allen had a fabulous video that dramatized the same. Of course, you also have a graphic novel introduction of someone saving the day with this knowledge. It of course depends on your audience and what will work for them.

Another story I tell is when a colleague found out I did games, and asked if I wanted to assist him and his team. The task was, to me and many, not necessarily a source of great intrinsic interest, but he pointed out that he’d discovered that to practitioners, it was like playing detective. Which of course gave him a theme, and a overarching hook. And this is the second element of engagement we can and should lever.

Once we’ve hooked them into why this learning is important, we then want to help maintain interest through the learning experience. If we can find out what makes this particular element interesting, we should have it represented in the examples and practice tasks. This will help illuminate the rationale and develop learner abilities by integrating the inherent nature of the task into the learning experience.

Often SMEs are challenging, particularly to get real decisions out of, but here’s where they’re extremely valuable. In addition to stories illustrating great wins and losses that can serve as examples (and the motivating example I mentioned above), they can help you understand why this is intrinsically interesting to them. They’ve spent the time to become experts in this, we want to unpack why this was worth such effort. You may have to drill a bit below “make the world a better place”, but you could and should be able to.

By hooking them in initially by making them aware of the role of this knowledge, and then maintaining interest through the learning experience, you have a better chance of your learning sticking. And that’s what we want to achieve, right?

Gary Stager #LearnTech2015 Keynote Mindmap

Clark @ 1:39 am

Gary presented a passionate and compelling argument for the value of using the maker movement as a vehicle for education reform.

2 November 2015

Roger Schank #learntech2015 Keynote Mindmap

Clark @ 7:29 pm

Roger gave his passioned, opinionated, irreverent, and spot-on talk to kick off LearnTechAsia. He covered the promise (or not) of AI, learning, stories, and the implications for education.

28 October 2015

Non-invasive Brain Surgery

Clark @ 8:03 am

Changing behavior is hard. The brain is arguably the most complex thing in the known universe. Simplistic approaches aren’t likely to work. To rewire it, one approach is to try surgery. This is problematic for a several reasons: it’s dangerous, it’s messy, and we really don’t understand enough about it. What’s a person to do?

Well, we do know that the brain can rewire itself, if we do it right. This is called learning. And if we design learning, e.g. instruction, we can potentially change the brain without surgery. However, (and yes, this is my point) treating it as anything less than brain surgery (or rocket science), isn’t doing justice to what’s known and what’s to be done.

The number of ways to get it wrong is long. Information dump instead of skills practice. Massed practice instead of spaced. Rote knowledge assessment. Lack of emotional engagement. The list goes on. (Cue the Serious eLearning Manifesto.) In short, if you don’t know what you’re doing, you’re likely doing it wrong and are not going to have an effect. Sure, you’re not likely to kill anyone (unless you’re doing this where it matters), but you’ll waste money and time. Scandalous.

Again, the brain is complex, and consequently so is learning design. So why, in the name of sense and money, do we treat it as trivial? Why would anyone buy a story that we can achieve anything meaningful by taking content and adding a quiz (read: rapid eLearning)? As if a quiz is somehow going to make people do better. Who would believe that just anyone can present material and learning will occur? (Do you know the circumstances when that will work?) And really, throwing fuzzy objects around the room and ice-breakers will somehow make a difference? Please. If you can afford to throw money down the drain (ok, if you insist, throw it here ;), and don’t care if any meaningful change happens, I pity you, but I can’t condone it.

Let’s get real. Let’s be honest. There’s a lot (a lot) of things being done in the name of learning that are just nonsensical. I could laugh, if I didn’t care so much. But I care about learning. And we know what leads to learning. It’s not easy. It’s not even cheap. But it will work. It requires good analysis, and some creativity, and attention to detail, and even some testing and refinement, but we know how to do this.

So let’s stop pretending. Let’s stop paying lip-service. Let’s treat learning design as the true blend of art and science that it is. It’s not the last refuge of the untalented, it’s one of the most challenging, and rewarding, things a person can do. When it’s done right. So let’s do it right! We’re performing brain surgery, non-invasively, and we should be willing to do the hard yards to actually achieve success, and then reap the accolades.

OK, that’s my rant, trying to stop what’s being perpetrated and provide frameworks that might help change the game. What’s your take?

Next Page »

Powered by WordPress