Peeragogy Handbook V1.0/Following the money

Summary

The metrics for learning in corporations are business metrics based on financial data. Managers want to know: "Has the learning experience enhanced the workers' productivity?"

Follow the money

When people ask about the ROI of informal learning, ask them how they measure the ROI of formal learning. Test scores, grades, self-evaluations, attendance, and certifications prove nothing. The ROI of any form of learning is the value of changes in behavior divided by the cost of inducing the change. Like the tree falling over in the forest with no one to hear it, if there’s no change in behavior over the long haul, no learning took place. ROI is in the mind of the beholder, in this case, the sponsor of the learning who is going to decide whether or not to continue investing. Because the figure involves judgment, it’s never going to be accurate to the first decimal place. Fortunately, it doesn’t have to be. Ballpark numbers are solid enough for making decisions.

Assessing Workplace Learning from Jay Cross on Vimeo.

The process begins before the investment is made. What degree of change will the sponsor accept as worthy of reinvestment? How are we going to measure that? What’s an adequate level of change? What’s so low we’ll have to adopt a different approach? How much of the change can we attribute to learning? You need to gain agreement on these things beforehand. Monday morning quarterbacking is not credible. It’s crazy to assess learning immediately after it occurs. You can see if people are taking part or if they’re complaining about getting lost, but you cannot assess what sticks until the forgetting curve has ravaged the learners’ memories for a few months. Without reinforcement, people forget most of what they learn in short order. It’s beguiling to try to correlate the impact of learning with existing financial metrics like increased revenues or better customer service scores. Done on its own, this approach rarely works because learning is but one of many factors that influence results. Was today's success due to learning or the ad campaign or weak competition or the sales contest or something else? The way to assess how people learn is to ask them. How did you figure out how to do this? Who did you learn this from? How did that change your behavior? How can we make it better? Too time consuming? Not if you interview a representative sample. For example, interviewing less than 100 people out of 2000 yields an answer within 10% nineteen times out of twenty, a higher confidence level than most estimates in business. Interviewing 150 people will give you the right estimate 99% of the time.