One of the questions I hear most often around the use of learning tools such as mobile and social is, “How do we measure?” There is always reluctance to deploy a new technology initiative if there is no concrete way to measure its success. When people ask me how to measure these new ways of learning, I always answer with a question of my own (even though my father told me not to do that): How are you measuring learning now?
It may seem an abrasive way to answer the question, but in reality this is where you need to start. There is no reason you cannot measure use of a social learning technology or a mobile deployment the same way you are measuring learning right now — unless, of course, you are doing a terrible job at measuring learning right now. In that case, you have bigger fish to fry.
Look at it this way: let’s simplify a learning process or initiative into the act of using a screwdriver to screw a cover onto a wooden box. That’s the activity in its entirety. How does Wooden Box, Inc. measure the process of screwing the cover onto the box? Chances are they are looking at the following:
- How long it takes to screw on the cover
- The tightness of the cover once it is screwed on
- The alignment of the cover to the box
- How many covers are screwed on in a day/month/year
One day the production manager comes to the floor and announces that everyone in the Box Cover Attachment division is getting a brand new cordless electric screwdriver. Once the employees start using this new tool, is the company suddenly going to stop measuring the items they had previously measured? Of course not. Those same metrics are precisely what you need to see if the new tool is working well or not. In fact, they will likely add a few metrics, such as:
- Battery life of the screwdriver
- Cost to charge the battery
- Cost to replace the battery
- Reduction in wrist injuries
Whether your organization uses the Kirkpatrick Model or some other measurement framework, the outcomes you are looking for in traditional learning modalities are essentially the same for new media. Level 1 is huge here, especially early on, to get a sense of whether or not you are delivering in a way that is resonating with your learners. According to the preliminary data from Brandon Hall Group’s Social and Collaborative Learning Survey, learner satisfaction is the number one metric for social learning tools, followed closely by demonstration of knowledge acquisition. Not exactly unheard-of metrics. But now there are additional metrics specific to the tool. For mobile, it could be user adoption or device preferences. For social media, you can measure items such as the number of contributions a person makes to discussion threads, or how many questions they answer in forums.
Don’t let your new media initiatives get derailed by naysayers who demand measurement. The measurement is already there. The question is not if there will be anything to measure, but rather selecting just what new metrics you want to include. These new technologies generate more than enough data for you to sift through.
–David Wentworth, Senior Learning Analyst
Brandon Hall Group