The relationship between big data and KPIs may not be immediately obvious. KPIs are what you use to measure performance, while big data is what Netflix uses to decide which series to recommend to you next, right? That’s true, but big data, KPIs and performance measurement are inextricably linked. And, as we’ll see in this article, big data has drastically altered how companies measure performance.
Connecting the dots between big data and KPIs
The increasing datafication of our world has changed what we can measure, and how. What do I mean by datafication? I mean the huge volumes of data that we are generating every single day – every time we go online, every time we pay for something with a credit card, every time we touch in with an Oyster card on the London Underground, every time my seven-year-old asks Alexa to play their favourite song for the hundredth time that day…
This exponential increase in data means that companies have more options for measuring performance than ever before. They have more data at their disposal (and, importantly, more real-time data). And they have more options for analysing and reporting that data. As a result, there are many new types of indicators for tracking performance.
Positive ways big data has impacted KPIs
Big data has helped to overcome two major limitations of traditional KPIs: firstly, that too many KPIs are standard proxies (such as generic surveys) and, secondly, that many are lagging indicators, telling us what’s already happened. The value of proxy tools or lagging indicators in today’s fast-paced business world is increasingly limited.
Now, instead of using a standard survey to monitor staff engagement, you can monitor actual employee behaviour. While surveys are useful as a starting point or for benchmarking purposes, they have their weaknesses. For starters, people often lie on surveys – whether it’s because they’re wary of reprisals if they tell the truth, they feel they should tell the company what it wants to hear, or it’s simply not in the individual’s interests to tell the truth.
Big data technology means it’s possible to track actual behaviour. Or, to put it another way, we can understand what people really do, not what they say they do.
Importantly, these KPIs are becoming increasingly rooted in real time. Instead of measuring past performance (last week, last month or last quarter, for instance), big data means it’s now possible to track what’s happening right now, in real time, or even predict future performance more accurately.
Exciting new ways of measuring performance
Here are just a few of the ways companies are using big data-related technology to improve their performance measurement.
What about the disadvantages?
Of course, for any kind of people-related data, there are privacy and ethical issues to consider. So it’s important you gather only the critical data you need to improve business performance, rather than gather every little bit of data just for the sake of it. It’s also important to be transparent with customers and employees about what data you’re collecting, why, and how it benefits them.
But, setting aside potential privacy pitfalls, perhaps the biggest disadvantage big data has brought to KPIs is information overload. The biggest challenge used to be finding the right data; now, the biggest challenge is narrowing down the seemingly unlimited options!
With so many options for measuring performance, companies can split into two camps: those who get overwhelmed and bury their head in the sand, ignoring the overwhelming options (thereby passing up some juicy insights that could really help boost performance); or those who jump in feet first, and capture data on anything and everything, just because they can, and regardless of whether or not it benefits the business.
Staying focused in a big data-driven world
It’s never a good idea to start collecting masses of data that you don’t really need, just because you can. Big data’s ability to transform KPIs for the better isn’t about collecting impressive amounts of data or slicing and dicing that data in lots of different ways. Rather, the power of big data lies in how you use it; and, in the case of KPIs and performance management, that means gathering the insights that will help people in the company make better decisions and improve performance.
Therefore, the best way to overcome the information overload problem is to stay focused on what it is you need to know. Focus on your business challenges and identify the questions you need to answer to overcome those challenges and meet your strategic goals. Then, once you know what questions you need to answer, you can design the most useful KPIs and gather the best data to help you answer them.
Where to go from here
If you would like to know more about KPIs and performance management, check out my articles on:
Bernard Marr is a bestselling author, keynote speaker, and advisor to companies and governments. He has worked with and advised many of the world's best-known organisations. LinkedIn has recently ranked Bernard as one of the top 10 Business Influencers in the world (in fact, No 5 - just behind Bill Gates and Richard Branson). He writes on the topics of intelligent business performance for various publications including Forbes, HuffPost, and LinkedIn Pulse. His blogs and SlideShare presentation have millions of readers.