The enormous amount of data being generated every day is changing our world, and has given rise to a term most of us are now familiar with: big data. The sheer volume of data available is just one of the many challenges in working with big data. But perhaps the biggest data threat is one that many businesses don’t see coming.

Big data brings big advantages

Thanks to big data, organisations of all shapes and sizes are gaining unprecedented insights into customer behaviour, market preferences and business performance. This knowledge is enabling businesses to make smarter decisions, better understand their customers, improve business processes, increase revenue, and more.

One great example comes from German retail company Otto. Data showed that Otto’s customers were less likely to return items when they arrived within two days, and when the items arrived all together at once, as opposed to separate shipments. This makes sense, but it posed a major challenge for Otto, which, like Amazon, sells products from many different suppliers. Keeping every item in stock so that goods can be shipped together is a tall order. Thanks to big data – to be precise, the data from 3 billion past transactions – the company was able to build a predictive model that could anticipate, with 90% accuracy, what customers would want to buy in the next 30 days. This means the right products can be ordered ahead of time. As a result, product returns have reduced by over 2 million items a year.

In another example, PepsiCo used data to drive sales of a new product: Quaker Overnight Oats. The company was able to identify the most valuable shoppers to market the product to – pinpointing 24 million priority households from a dataset of 110 million households. After a carefully targeted launch campaign, these priority customers drove 80% of the product’s sales in the first 12 weeks after launch.

The darker side of big data

There are many challenges associated with big data. When I work with business leaders to help them develop their data strategy, often the first pitfalls or challenges they think of are around technology or skills. In other words, without the technical infrastructure, in-house knowledge or vast budgets of companies like Amazon or Facebook, many business leaders think the advantages of big data are beyond their grasp.

But thanks to big-data-as-a-service (BDaaS), technology and skills pose less of a problem than you might think. BDaaS essentially refers to the delivery of big data tools and technology – and sometimes even the data itself – through software-as-a-service platforms, meaning companies can access big data tools without having to invest in expensive infrastructure or new hires. As these off-the-peg data and analytics tools become more advanced, my hope is that technology, infrastructure, and skills will become less daunting barriers to working with data.

Data security is another challenge that business leaders cite, and with good reason. As data becomes an increasingly critical business asset, the need to protect that data (particularly personal data) becomes even more vital. That’s why every business needs a robust data security policy in place, and must take steps to educate employees on the potential threats. Most businesses are all too aware of this need, though. What are they less aware of?

One pitfall that doesn’t get the recognition it deserves

In my experience, there’s one threat that too many businesses downplay. And that threat is data privacy. For many years, there’s been something of a Wild West culture around big data, with companies playing fast and loose with privacy. Think of the news in 2019 that human analysts listen to people’s private Alexa conversations. Amazon maintained the practise is vital for improving Alexa’s abilities, and I’m sure that’s true, but the fact remains that most people didn’t know their Alexa recordings might be heard by others – because that not-so-insignificant detail was buried in lengthy terms and conditions that few people read. Consumer outrage ensued, and Amazon wisely offered an opt-out option.

This is just one of many examples of companies taking liberties with people’s data. Apps that demand access to all your phone contacts. Social media sites using facial recognition technology on you and your friends without you realising. Companies reserving the right to share your personal details with “interested third parties”. The list goes on.

The introduction of the European Union’s GDPR shows that data privacy laws are beginning to tighten up, and will continue to do so – but there’s no doubt that regulators are playing catchup on this issue. I believe it’s up to companies to forge an ethical, responsible approach to data privacy. An approach that recognises and safeguards individuals’ right to privacy. An approach that’s far more transparent than what we’voe seen in the past.

Because the truth is, most of us are happy to part with our precious data when we’re getting something useful in return. Plenty of Alexa users wouldn’t mind their recordings being used to improve the Alexa service, so long as they know that’s what they’re signing up for, and that they can opt-out at any point. When customers can’t make an informed choice, that’s when problems can occur.

Bad news has always travelled fast. But in this digital age, a company’s reputation can be tarnished in seconds. Companies who don’t recognise the importance of data privacy risk not only legal blowback and hefty fines, but also serious reputational damage.

A more responsible approach to data

Data is clearly powerful stuff. But as any Spider-Man fan knows, with great power comes great responsibility! Here are some key tips for wielding the power of big data responsibly, so that you don’t get blindsided by the data privacy threat:

• Be upfront with your customers and stakeholders about what data you’re collecting and why.
• Don’t bury this information in lengthy service or product terms and conditions that you know nobody will read.
• Offer an opt-out wherever possible.
• Anonymise customer data wherever possible so that individuals cannot be identified.
• Make sure customers know what they’re getting in return for giving up their data.