I continue to get asked about social learning metrics. Until we get around to a whitepaper or something on metrics, here’re some thoughts:
Frankly, the problem with Kirkpatrick (sort of like with LMS’ and ADDIE, *drink*) is not in the concept, but in the execution. As he would say, stopping at level 1 or 2 is worthless. You need to start with Level 4, and work back. This is true whether you’re talking about formal learning, informal learning, or whatever. Then, I’m not feeling like you have to be anal about levels 1-3, it’s level 4 that matters, but there’s plausibility that making the link makes your case stronger. And I also like what I heard added at a client meeting: level 0, are they even taking the course/accessing the system? But I digress…
So, let’s say you are interested in seeing what social media can do for your organization: what are you not seeing but need to? If you’re putting in a social media system into a call center, maybe you want reduced time to problem solution, fewer customer return calls on the same problem, etc. If you’re into an operations group, maybe you want more service or product ideas. What is it you’re trying to achieve? What would indicate the innovation that you’re looking to spark?
Then, you need to find ways to measure those outcomes. You have three basic decisions to make in terms of a strategic initiative:
- it’s working, yay, let’s keep it.
- hmm, it’s kinda working, but we need to tweak it
- oh oh, this is bleeding money, let’s kill it
You should set parameters before you launch the initiative that you think indicate the thresholds you are talking about. The keep and kill thresholds likely have to do with the costs versus the benefits. You may change those parameters on inspection of the results at any time, but at least you are doing it consciously. And gradually your patience will or should fade. Eventually you end up with either a leave or kill decision.
Frankly, even activity is a metric. A vendor of a social media system uses that as a metric for billing (though I don’t think that two touches a month constitutes a meaningful interaction by a user), and if people are talking productively and getting value, you’ve got at least an argument that intangible benefits are being generated. You could couple that with subjective evaluation of value, but overall I would like to argue for more meaningful outcomes.
And don’t think that you have to have only one. Depending on the size of the initiative and the different silos that are being integrated, you might have more. You might check not only key business metrics, but look for impacts on retention and morale as well, if the benefits of improving work environments are to believed (and I do). And, of course, there’s more than the installation and measuring: the tweaking for instance could involve messaging, culture, interface design, or more.
Metrics for informal learning aren’t rocket science, but instead mapping of best principles into specific contexts. Your organization needs to find ways to facilitate social learning, as the innovation outcomes are the key differentiator going forward, as so many say. You should be experimenting, but with impacts you’d like to have, not just on faith.