Search Results for: engagement

Last week, at Online Educa in Berlin, I gave a tutorial on deeper elearning as a pre-conference event. In it, I talked about getting more meaningful objectives, writing practice that actually develops meaningful outcomes, and content (concepts & examples) aligned to support effective practice. I also talked about emotional engagement and social learning, before talking about revising design processes to incorporate these deeper elements in an effective and not-too-different approach.  In short, I was talking about aligning our designs, and our design processes, to how we think and learn.

This is something that most organizations should be thinking about.  I was pleasantly surprised that the audience included folks from universities, not-for-profits, and government agencies as well as businesses.  The challenges are different in some respects, but there are shared elements.  Education tends to be about long term learning relationships (typically at least a half year to several years), versus the short-term relationships (e.g. an hour to several days) in organizational learning.  Yet the need to respect how our brains work is a continuum. Our brains learn in particular ways that are unaffected by the curricular needs.  Learning solutions for performance and for education both still need to respect our neural and cognitive architecture.

And too little of what  we do  reflects what we know.  As a recent commenter noted, there’s a conflict between the de-facto practices and what research says.  As she also noted, our tools are also focused on supporting wrong approaches.  It’s not that tools prevent doing meaningful learning, it’s just that you have to get your design right first and then make the tool conform (as opposed to the alternative). And our limitations as designers flow from the same source, our brain,  as our limitations as learners.  Thus we need to be as aware of cognition in our designing as in our design.

I’ll be talking about the problems this engenders in a special  webinar  tomorrow. There’re still a few slots left. If you’re committed to trying to improve  your learning design, and you have the resources to do so, this is an opportunity to get started.  There isn’t a lot of pressure  yet, but it’s time to be proactive  before people start asking questions about the business impact of what we’re doing. What do you think?

It’s a continual bane of a consultant’s existence that there are people who want to ‘pick your brain’.  It’s really asking for free consulting, and as such, it’s insulting. If you google the phrase, you’ll see how many people have indicated their problems with this! However, there are quite legitimate ways to  pick  my brain  and I thought I’d mention a couple.  In both cases,  I think were great engagements on both sides, high value for a reasonable investment.

Both in this case were for folks who develop content. In one case a not-for-profit, the other in the higher-ed space.  One  had heard me speak about learning design, and one had heard about a workshop I’d given, but both contacted me. It is clear they realized that there’s value to them for having a scrutable learning design.

Content Review

So for the first one, they wanted some feedback on their design, and we arranged that I’d investigate a representative sample and provide feedback.  I went through systematically, taking notes, and compiled my observations into a report I sent them.  This didn’t take any investment in travel, but of course this feedback  only points out what’s wrong, and doesn’t really provide mechanisms to improve.

I think they were surprised at the outcome, as the feedback was fairly robust.  They had a good design, largely, under the constraints, but there were some systematic design problems.  There were also  some places where they’d managed to have some errors that had passed editorial (and this was only a small sample of a replicated model across a broad curriculum). To be fair, some of my complaints came from situations that were appropriate given some aspect of their  context that I hadn’t known, but there were still a set of specific improvements I could recommend:

“We found his comments insightful, and  we look forward to implementing his expert suggestions to further improve of our product…“

Learning Design Workshop

In this case, they’d heard about a workshop that I’d run on behalf of a client, and were interested in getting a similar experience. They had been designing content and had a great ability to track the results of their design and tweak, but really wanted a grounding in the underlying learning science.  I did review some sample content, but I also traveled to their site for a day and presented learning science details and workshopped the implications to their design process.

I went through details such as:

• the importance and format for  objectives,
• SME limitations and tips how to work with  them,
• what makes effective practice,
• the role and characteristics of concepts,
• the details behind examples,
• introduction and the role of emotions in the learning experience,
• and more.

We went through examples of their content, and workshopped how they could adjust their design processes in pragmatic ways to instill the important details into their approach.  We also talked about ways to followup to not lose the momentum, but it was clear that this first visit was viewed favorable:

“…a walking encyclopedia of learning science… was able to respond to our inquiries with one well-researched perspective after another”.

So, there are ways to pick my brain  that provide high value with mutual benefit on each side.  Sure, you can read my blog or books, but sometimes you may want assistance in contextualizing it to your situation.  I encourage you to think of making an investment in quality.  These are about learning design, but I have some examples in strategy that I intend to share soon.  And more.  Stay tuned for more    ‘adventures in  consulting’ tales that talk about ways in which a variety of needs are met.  Maybe one will resonate with you.  Of course, they’ll be mixed in with the regular reflections you’ve come to expect.

Designing learning is a probability game.  To paraphrase Dorothy Parker, you can lead a learner to learning, but you can’t make them think.  What I mean is that the likelihood that the learning actually sticks is a result of a myriad of design decisions, and many elements contribute to that likelihood.  It will vary by learner, despite your endeavors, but you increase the probability that the desired outcome is achieved by following what’s know about how people learn.

This is the point of learning engineering, applying learning science to the design of learning experiences.  You need to align elements like:

• determining  learning objectives that will impact the desired outcome
• designing  sufficient contextualized practice
• appropriately presenting a conceptual model that guides performance
• providing a sufficient and elaborated suite of examples to   illustrate the concept in context
• developing emotional engagement

and so on.

And to the extent that you’re not fully delivering on the nuances of these elements, you’re decreasing the likelihood of having any meaningful impact. It’s pretty simple:

If you don’t have the right objectives (e.g. if you just take an order for a course), what’s the likelihood that your learning will achieve anything?

If you don’t have sufficient practice, what’s the likelihood that the learning will still be there when needed?

If you have abstract practice, what’s the likelihood that your learners will transfer that practice to appropriate situations?

If you don’t guide performance with a model, what’s the likelihood that learners will be able to adapt their performance to different situations?

If you don’t provide examples, what’s the likelihood that learners will understand the full range of situations and appropriate adaptations for each?

And if you don’t emotionally engage them, what’s the likelihood that any of this will be appropriately processed?

Now, let’s tie that back to the dollars it costs you to develop this learning.  There’s the SME time, and the designer time, and development time, and the time of the learners away from their revenue-generating activity. At the end of the day, it’s a fair chunk of change.  And if you’re slipping in the details of any of this (and I’m just skating the surface, there’re nuances around all of these), you’re diminishing the value of your investment, potentially all the way to zero. In short, you could be throwing your money away!

This isn’t to make you throw up your hands and say “we can’t do all that”.  Most design processes have the potential to do the necessary job, but you have to comprehend the nuances, and ensure that the i’s are dotted and t’s crossed on development.  Just because  you have an authoring tool doesn’t mean what comes out is actually achieving anything.

However, it’s possible to tune up the design process to acknowledge the necessary details. When you  provide support at just the right places, and put in place the subtle tweaks on  things like working with SMEs, you can develop and deliver learning that has a high likelihood of having the desired impact, and therefore have a process that’s justifiable for the investment.

And that’s really the goal, isn’t it?  Being able to allocate resources to impact the business in meaningful ways is what we’re  supposed to be doing. Too frequently we see the gaps continue (hence the call for Serious eLearning), and  we can only do it if we’re acting like the professionals we need to be.    It’s time for a tuneup in organizational learning.  It’s not too onerous, and it’s needed.  So, are you ready?

So, a few weeks ago I ran a survey asking about elearning processes*, and it’s time to look at the results (I’ve closed it).  eLearning process is  something I’m suggesting is ripe for change, and I thought it appropriate to see what people thoughts.  Some caveats: it’s self-selected, it’s limited (23 respondents), and it’s arguably readers of this blog or the other folks who pointed to it, so it’s a select group.  With those caveats, what did we see?

The first question was looking at how we align our efforts with business needs. The alternatives were ‘providing what’s asked for’ (e.g. taking orders), ‘getting from  SMEs’, and ‘using a process’.  These are clearly in ascending order of appropriateness. Order taking doesn’t allow for seeing if a course is needed and SMEs can’t tell you what they actually do. Creating a process to ensure a course is the best solution (as opposed to a job aid or going to the network), and then getting the real performance needs (by triangulating), is optimal.  What we see, however, is that only a bit more than 20% are actually getting this right from the get-go, and almost 80% are failing at one of the two points along the way.

The second question was asking about how the assessments were aligned with the need. The options ranged from ‘developing from good sources’, thru ‘we test knowledge’ and ‘they have to get it right’ to ‘sufficient spaced contextualized practice’, e.g. ’til they can’t get it wrong.  The clear need, if we’re bothering to develop learning, is to ensure that they can do it at the end.  Doing it ‘until they get it right’ isn’t sufficient to develop a new ability to do.  And, we see more than 40% are focusing on using the existing content! Now, the alternatives were not totally orthogonal (e.g. you could have the first response and any of the others), so interpreting this is somewhat problematic.  I assumed  people would know to choose the lowest option in the list if they could, and I don’t know that (flaw in the survey design).  Still it’s pleasing to see that almost 30% are doing sufficient practice, but that’s only a wee bit ahead of those who say they’re just testing knowledge!  So it’s still a concern.

The third question was looking at the feedback provided. The options included ‘right or wrong’, ‘provides the right answer’, and ‘indication for each wrong answer’.  I’ve been railing against one piece of feedback for all the wrong answers for years now, and it’s important. The alternatives to the wrong answer shouldn’t be random, but instead should represent the ways learners typically get it wrong (based upon misconceptions).  It’s nice (and I admit somewhat surprising) that almost 40% are actually providing feedback that addresses each wrong answer. That’s a very positive outcome.  However, that it’s not even half is still kind of concerning.

The fourth question digs into the issue of examples.  There are nuances of details about examples, and here I was picking up on a few of these. The options  ranged from ‘having’, thru ‘coming from SMEs’ and ‘illustrate the concept and context’, to ‘showing the underlying thinking’.  Again, obviously the latter is the best.  It turns out that experts don’t typically show the underlying cognition, and yet it’s really valuable for the learning. We see that we are getting the link of concept to context clear, and together with showing thinking we’re nabbing roughly 70% of the examples, so that’s a positive sign.

The fifth question asks about concepts.  Concepts are (or should be) the models that guide performance in the contexts seen across examples and practice (and the basis for the aforementioned feedback). The alternatives ranged from ‘using good content’ and ‘working with SMEs’ to ‘determining the underlying model’.  It’s the latter that is indicated as the basis for making better decisions, going forward.  (I suggest that what will helps orgs is not the ability to receive knowledge, but to make better decisions.)  And we see over 30% going to those models, but still a high percentage still taking the presentations from the SMEs. Which isn’t totally inappropriate, as they  do have access to what they learned. I’m somewhat concerned overall that much of ID seems to talk about practice and ‘content’, lumping intros and concepts and examples and closing all together into the latter (without suitable differentiation), so this was better than expected.

The sixth question tapped into the emotional side of learning, engagement. The options were ‘giving learners what they need’, ‘a good look’, ‘gamification’, and ‘tapping into intrinsic motivation’.  I’ve been a big proponent of intrinsic motivation (heck, I effectively wrote a book on it ;), and not gamification. I think an appealing visual design, but just ‘giving them what they need’ isn’t sufficient for novices: they need the emotional component too. For practitioners, of course, not so much.  I’m pleased that no one talked about gamification (yet the success of companies that sell ‘tart up’ templates suggests that this isn’t the norm). Still, more than a third are going to the intrinsic motivation, which is heartening. There’s a ways to go, but some folks are hearing the message.

The last question gets into measurement.  We should be evaluating what we do. Ideally, we start from a business metric we need to address and work backward. That’s typically not seen. The questions basically covered the Kirkpatrick model, working from ‘smile sheets’, through’ testing after the learning experience’ and ‘checking changes in workplace behavior’ to ‘tuning until impacting org  metrics’.  I was pleasantly surprised to see over a third doing the latter, and my results don’t parallel what I’ve seen elsewhere. I’m dismayed, of course, that over 20% are still just asking learners, which we know in general isn’t of particular use.

This was a set of questions deliberately digging into areas where I think elearning falls down, and (at least with this group of respondents), it’s not good as I’d hope, but not as bad as I feared.  Still, I’d suggest there’s room for improvement, given the constraints above about who the likely respondents are.  It’s not a representative sample, I’d suspect.

Clearly, there are ways to do well, but it’s not trivial. I’m arguing that we can do good elearning without breaking the bank, but it requires an understanding of the inflection points of the design process where small changes can yield important results. And it requires an understanding of the deeper elements to develop the necessary tools and support. I have been working with several organizations to make these improvements, but it’s well past time to get serious about learning, and start having a real impact.

So over to you: do you see this as a realistic assessment of where we are? And do you take the overall results as indicating a healthy industry, or an industry that needs to go beyond haphazard approaches and start practicing Learning Engineering?

*And, let me say, thanks  very much to those respondents who bothered to take the time to respond.  It was quick, but still, the effort was completely appreciated.