Big Data offers the prospect of reaching the ‘Holy Grail’ of customer insight – being able to analyse and track consumer behaviour in real-time and at a large enough scale to confidently make decisions. Not only that, but it’s real, actual behaviour – not reported data from surveys. It ticks all the boxes for a CEO.
However, making the most of this promise will prove tricky to achieve; requiring new skills, new techniques and big budgets. To make Big Data really work we need to augment it with the data we currently use - reported data, perceptions and qualitative insight. This is how we will make the most out of the nirvana of Big Data.
What is ‘Big Data’ and why have we started talking about it? The reason is that technology now captures data – our clicks, actions, responses – very easily. And the Internet makes this possible from remote sources, meaning clicks on one machine can be recorded, stored and tracked on a remote server. Yes, Big Data is also a bit ‘Big Brother’! If you click a link in an email, the sender could track that and plot it against your name on a database – recording your behaviour.
In the last 5 years we have seen a massive rise in data being collected off the back of the increasing web access, smartphones that also use location data and Customer Relationship Management systems that centralise purchase history. Social media use has also skyrocketed, meaning what we say online is also now recordable and trackable.
As the popularity and use of these has increased, the data wave has fast turned into a tidal wave. Companies have wised-up to the potential of data and started joining the data sources together to provide greater analysis potential. It’s all real-time, it’s all real actions and it’s all ‘free’ to collect.
That’s not the end of the story. What we are going to witness in the next decade is a tsunami of data. The internet is now embedded into daily appliances and devices – the Internet of Things (IoT), as it has come to be known. The potential for embedding the internet into devices is far greater than at first glance. A desktop, mobile, tablet, radio, games console are all fairly obvious. But the amount of places it could be embedded easily dwarfs these.
Some of these (fridges for example) may seem unrealistic, unnecessary and futuristic, but I think they will all come with time. This means that far more of our daily lives and consumption habits will become recordable and trackable.
When you think about this and reflect that ‘only’ three billion of the world’s population are connected online and that our use of mobile and social media is still only in its infancy, you realise that the data being collected is set to expand exponentially.
The tidal wave of data will become a tsunami. A fantastic opportunity for those that can access it, harness it and make it yield to their needs.
Big Data brings some big challenges for companies. The first is that it is will take big budgets to make the most of all of this data in the system integrations that will be needed. Secondly, it will require specialist analytical software to be able to make sense of the volumes of data. Many of these systems will need to run automated algorithms, tailored to make sense of the different types of data.
Third, Big Data analysis will require new specialist skills – and likely full time jobs – to run sophisticated analysis and interpret what it all means. And here lies the hidden challenge of Big Data: getting to grips with the meaning.
While there is a new ‘promised land’ of data, it isn’t the panacea that it first appears. This new vast data source is incredibly powerful – and just the type of data businesses need and want. But there is also a familiar and old-style issue with it. Data on its own means nothing. It is the interpretation that brings the meaning and application.
This isn’t a new challenge at all – it has always been there. It’s just that it actually matters more than ever, because there is higher quality data available. The value lies in the interpretation of the trends, the shifts in behaviour and their significance.
What Big Data has really done is change the game in how businesses collect customer data. In the past, our only source was from primary data collection methods. The market research industry was the main supplier, specialising in providing high quality and rigorous collection processes. That rug, so to speak, has been firmly pulled from under researchers. Technology has shifted the game of information gathering considerably towards automated, real-time and event-based data.
The market research industry has to focus on providing ‘added value’ to the data, rather than the rigour of data collection. Market researchers are now in direct competition with ‘in house’ business analysts and the management consultancy sector.
The advantage that market researchers have is the importance that ‘Little Data’ will have in the future. ‘Little Data’ is all of the other market research that will still be necessary – in fact, not just necessary, but imperative. We will still need customer satisfaction and perception feedback. This will be collected differently now – in smaller bite-sized chunks, be more event-driven and more engaging for respondents.
But the real added value is that which qualitative research brings – as William Mills nicely summarised for AQR recently, it is qual research that adds meaning and emotion. This will be vital in providing the understanding of why the Big Data trends are happening and will make action possible – explaining how and why to implement a change.
Businesses love quantitative data. When making a decision, it is imperative to understand the scale of something. It adds confidence to a decision and enables finance directors to allocate budget to strategies. But businesses are in danger of forgetting the individual.
Customers are not conglomerated data sets – they are individuals, with emotions, irrational behaviour and relationships; they operate in fluid environments, contexts that change with time, place and function. None of these can be captured effectively in the Big Data world, so all need to be augmented for effective decisions to be taken.
This is where market research – and the future of FlexMR – lies. We are positioned at the intersection of Big Data and Little Data. Our role is unique in that we are bring meaning to Big Data and combine it with Little Data through the use of our customer panels.
Our unique software platform enables us to action short, engaging polls in real-time, augment this with Big Data profiles and initiate in-depth targeted qualitative insights ‘on the fly’ to bring meaning and emotion into the boardroom.