Will, thanks for weighing in. And I agree, it is complex. What seems simple up-top unpacks down into considerable nuances. That’s why you get paid the big bucks! :). Seriously, I do think impact and efficiency are important, concepts, as well as effectiveness. Sometime we need to sit down around a white board and beat this into submission. Maybe Orlando in March?

I’m still learning about learning evaluation — it is an immensely complex area of endeavor — but I’m beginning to think about learning evaluation with a different frame. One I’ve missed before, or maybe one that hasn’t arisen before. I’m thinking that evaluation is done to help us make decisions. Indeed, there are different players who must make different decisions. In short, and I will elaborate on this in the future, but in short we should evaluate to help us make decisions. So, to determine what to evaluate, we have to determine what our decisions are.

Labels like “efficiency” and “impact” I don’t think are that helpful. Anyway, I know I’m on my own here, and so maybe I’m just suffering from early-onset of dementia, but I’ll try to keep working on this to see if I get any traction.

Guy, thanks for weighing in, and I think you know what I mean (heck, your Lean ISD is a guide here), but…. I’ll say impact 1st, effectiveness 2nd, and efficiency 3rd. Why? Because we could be effective at a learning objective that doesn’t do anything for the org, and be efficient at it, too. And it’s still not what we should be doing. And I’d extend: design for, and then measure/evaluate. Make sense?