Comment: Dealing with the data deluge - Three steps to help you get the most from a vital resource

Our society creates vast quantities of data – 2.5 quintillion bytes every single day, or 18 zeros for the curious. From businesses to individuals, information is created and stored at almost every turn – something likely to grow further with the onset of the Internet of Things. All this data can be used to quantify everything and analyse even more – or at least that’s the idea. The truth, however, is that most of the time this vast resource isn’t used to its fullest. Many larger organisations have the theory right and are collecting vast reservoirs of information. Yet they don’t have the processes and systems in place to turn it all into actionable insight. With this in mind, we will look at the three key issues businesses must get right in order to glean first class insight from their data.

You get out what you put in

In order to get the most from your investment, the first step needs to be ensuring that the data is useful to begin with. The adage of ‘getting out what you put in’ certainly rings true here. Gathering data is a good first step, but if the quality of that data is bad, the insight will be too. At its most basic, it needs to be stored in the same formats, and in the same manner across the business to be truly useful.

Healthy data is vital

We experienced the challenge that less than ideal data quality can cause when we recently tried to analyse possible links between socioeconomic factors and increases in prescriptions for asthma drugs and antibiotics. The NHS has vast swathes of Open Data from all of the surgeries and other locations under its control. Yet that data is saved in many different ways and formats, making it harder to analyse. This highlights the obstacle that poor quality data can pose – before businesses embark on a journey to improving insight, they must get their data ‘house’ in order first: otherwise they can face the same issues as the vast, sprawling and decentralised NHS.

I want ALL the data

Poor data quality isn’t the only stumbling block for organisations. One of the other challenges is not knowing what is available. In order to analyse everything successfully it’s vital the business has a complete view based on all relevant information, but in many cases,  they don’t actually know what they have to begin with. This is as a result of data being saved in disparate locations by each department, without a central repository. This creates data silos that are hard to access and that have little visibility outside of the team that created them. Therefore, the second step to successfully utilising company data is to take stock of what the organisation has available and make sure that central storage is put in place.

Bringing it all together with expertise

When an organisation knows what data is available; where it is stored; and that it is in good condition, the final step is to ensure that the right people are in place. Without expertise to analyse the information, it’s unlikely that an organisation will be able to learn from its data. In the past these might have been data scientists, but now it’s time for technology to enable a broader group to lead the data revolution. Now organisations need to find people from within who can ask the right questions of their data. This is important, as those just looking at information and drawing conclusions will quickly find that the ‘insight’ they are generating is, at best, of little use to the business.

Essentially businesses will be creating citizen data scientists. Rather than needing extensive data management qualifications they can be upskilled with a little training to build on their existing business instincts. What is most important is that they are in touch with business goals – both long term and short term – and know what needs to be achieved. This way, they can ask the right questions of the data to glean the most valuable insight to help towards business outcomes. 

Unlocking insight

There might be a temptation to put in systems and platforms to help create insight from data as quickly as possible. However, in order to make the most of all this data, the business must first put the right processes in place and train the right people. Only when data is in good condition; accessible; and with the right employees trained to use it, can organisations make every decision an informed decision.

Mark Hinds has been Global CEO at business intelligence and analytics firm Polymatica for just over a year, and has over twenty year's experience in the data analytics industry.