Archives for March 2010

Well, it turns out I was wrong.   I like to believe it doesn’t happen very often, but I do have to acknowledge it when I am. Let me start from the worst, and then qualify it all over the place ;).

In the latest Scientific American Mind, there is an article on The Pluses of Getting It Wrong (first couple paragraphs available here). In short, people remember better if they first try to access knowledge that they don’t have, before they are presented with the to-be-learned knowledge.   That argues that pre-tests, which I previously claimed are learner-abusive, may have real learning benefits.   This result is new, but apparently real.   You empirically have better recall for knowledge if you tried to access it, even though you know you don’t have it.   My cognitive science-based explanation is that the search in some ways exercises appropriate associations that make the subsequent knowledge stick better.

Now, I could try to argue against the relevance of the phenomenon, as it’s focused on knowledge recovery which is not applied, and may still lead to ‘inert knowledge’ (where you may ‘know it’, but you don’t activate it in relevant situations).   However, it is plausible that this is true for application as well.   Roger Schank has argued that you have to fail before you can learn. (Certainly I reckon that’s true with overconfident learners ;). That is, if you try to solve a problem that you aren’t prepared for, the learning outcome may be better than if you don’t.   Yet I don’t think it’s useful to deny this result, and instead I want to think about what it might mean for still creating a non-aversive learner experience.

I still believe that giving learners a test they know they can’t pass at best seems to waste their time, and at worst may actually cause some negative affect like lack of self-esteem.   Obviously, we could and should let them know that we are doing this for the larger picture learning outcome.   But can we make the experience more ‘positive’ and engaging?

I think we can do more. I think we can put the mental ‘reach’ in the form of problem-based learning (this may explain the effectiveness of PBL), and ask learners to solve the problem. That is, put the ‘task’ in a context where the learner can both recognize the relevance of the problem and is interested in it.   Once learners recognize they can’t solve the problem, they’re motivated to learn the material.   And they should be better prepared mentally for the learning, according to this result. While it *is*, in a sense, a pre-test, it’s one that is connected to the world, is applied, and consequently is less aversive.   And, yes, you should still ensure that it is known that this is done to achieve a better outcome.

Now, I can’t guarantee that the results found for knowledge generalize to application, but I do know that, by and large, rote knowledge is not going to be the competitive edge for organizations.   So I’d rather err on the side of caution and have the learners do the mental ‘reach’ for the answer, but I do want it to be as close as possible to the reach they’ll do when they really are facing a problem.   If there is (and please, do ensure there really is, don’t just take the client’s or SME’s word for it), then you may want to take this approach for that knowledge too, but I’m (still) pushing for knowledge application, even in our pre-tests.

So, I think there’s a revision to the type of introduction you use to the content,   presenting the problem or type of problem they’ll be asked to solve later and encouraged to have an initial go at it before the concept, examples, etc are presented.   It’s a pre-test, but of a more meaningful and engaging kind.   Love to see any experimental investigation of this, by the way.

At the eLearning Guild’s Learning Solutions conference this week, Jean Marripodi convinced Steve Acheson and myself to host a debate on the viability of ADDIE in her ID Zone.   While both of us can see both sides of ADDIE, Steve uses it, so I was left to take the contrary (aligning well to my ‘genial malcontent’ nature).

This was not a serious debate, in the model of the Oxford Debating Society or anything, but instead we’d agreed that we were going to go for controversy and fun in equal measures.   This was about making an entertaining and informative event, not a scientific exploration.   And in that, I think we succeeded (you can review the tweet stream from attendees and some subsequent conversation).   Rather than recap the debate (Gina Minks has a short piece in her overall summary of the day), I’ll recap the points:

The pros:

• ADDIE provides structured guidance for design
• ADDIE includes a focus on implementation and evaluation
• ADDIE serves as a valuable checklist to complement our idiosyncratic design habits

The cons:

• ADDIE is inherently a waterfall model, and needs patching to accommodate iterative development and rapid prototyping
• People use ADDIE too much as a crutch for design without taking responsibility for using it appropriately
• It assumes courses

The pragmatics:

Steve showed how he does take responsibility, putting evaluation in the middle and using it more flexibly. He uses Dick & Carey’s model to start with, ensuring that a course is the right solution.   The fact that the initial ‘course, job aid, other problem’ analysis is not included, however, is a concern.

It also came out that having a process is a powerful argument against those who might try to press unreasonable production constraints on you.   If a VP wants it done in an unreasonable time frame, or doesn’t want to allow you to question the analysis that a course is needed, you have a push back (“it’s in our process”), particularly in a process organization.   You do want a process.

The Alternatives:

The obvious question came up about what would be used in place of ADDIE.   I believe that ADDIE as a checklist would be a nice accompaniment to both a more encompassing   and a more learning-centric approach.   For the former, I showed the HPT model as a representation of a design approach considering courses as part of a larger picture.   For the latter, I suggested that a focus on learning experience design would be appropriate.

Using an HPT-like approach first, to ensure that a course is the right solution, is necessary.   Then, I’d focus on working backwards from the needed change (Michael Allen talked about using sketches as lightweight prototypes at the conference, and first drawing the last activity the user engaged in) thinking about creating a learning experience that develops the learner’s capability.   Finally, I’d be inclined to use ADDIE as a checklist to ensure all the important components are considered, once I’d drafted an initial design (or several).   ADDIE certainly may be useful in taking that design forward, through development, implementation and evaluation.

Summary

I think ADDIE falls apart most in the initial analysis, not being broad enough, and in the design process: e.g. most ID processes neglect the emotional side of the equation, despite the availability of Keller’s ARCS model (which wasn’t even in the TIP database!).   Good users, like Steve, take responsibility for reframing it practically, but I’m not confident that even a majority of ADDIE use is so enabled.   Consequently, I worry that ADDIE is more detrimental than good.   It ensures the minimum, but it essentially prevents inspiration.

I’m willing to be wrong, but I’ve been looking at the debate on both sides for a long time.   While I know that PowerPoint doesn’t kill people, people kill people, and the same is true of ADDIE, the continued reliance on it is problematic.   We probably need a replacement, one that starts with a broader analysis, and then provides guidance across job aid development, course development and more, that has at core iterative and situated design, informed by the recognition of the emotional nature of human use.   Anyone have one to hand?   Thoughts on the above?