Big data is a buzzword which has become a focal point of discussion. To demonstrate scale of the hype, upon writing this blog, my Google search for the term big data brought back over 400 million search results and I’m sure that will be more by now.
Commonly, big data and market research are being pushed into the same pool. Some have even gone as far as to say that big data will eventually lead to the eradication of market research… personally, I certainly hope not, but I also strongly believe not too. In my opinion, they should be handled and treated as separate commodities. That’s not to say they can’t be used in conjunction with each other; if you have the budget then I’d certainly recommend they should be used together to get the most out of your insights. But both are valuable in their own right and here I’d like to prove their value as individual entities by highlighting some of the key differences. Let’s explore them…
Size Does Matter!
The most obvious difference comes in the size of the data we are dealing with. Big data is just that… BIG! So big that’s it’s actually quite scary how much big data knows about you!
It is large amounts of structured, unstructured and complex data which, as a result of the ever expanding internet, is just getting bigger. It is data about our behaviours which is collected very easily through and driven by the use of technology – what we purchase, what we share or say on social media, which links we click on in emails we receive… the list goes on. It is real time and real actions.
 |
Tweet This |
|---|---|
| "Big Data is made up of large. unstructured and complex data sets gathered in real time." |
In market research studies, you may collect large amounts of data by using multiple methods or large sample sizes, but never on the same scale, or even close to it, as big data. Market research is the little data driven by an understanding of psychology.
Dealing With Your Data
Due to the wealth of information that big data provides, it is impossible to use traditional market research techniques to attempt to interpret and understand the data; one of the biggest challenges big data faces is processing it – it would certainly be too time consuming to sift through the endless amounts of data to be able to make sense of it.
Instead, specialist software and techniques are needed to make the best use of the data you have gathered from your customers; the technology and software for big data aims to manage the volume, variety and velocity of information and allow you to condense them into manageable insights. To get the most value from your data, you also need to be careful to avoid confirmation bias, in other words, don’t just look for the information that supports your point of view and ensure you have the right team of scientists or analysts who can confidently navigate their way around the data set and analyse with ease.
Indeed, market research faces its own challenges in how we handle the data, but on a much different scale – quite literally. One of the biggest challenges market research faces is the ability to scale the results to represent the entirety of the market. Achieving representative samples and good response rates is essential in market research to be able to provide the voice of the customer (VoC). Without this, data simply has no value.
Careful planning when choosing your sample in research will pay dividends: use a sampling calculator to ensure your sample is large enough to be representative of your customers and screen participants carefully to guarantee that the data collected represents the views of the majority.
Making Sense Of It All
Lars Perner points out that consumer behaviour is defined as “The study of individuals, groups, or organizations and the processes they use to select, secure, use, and dispose of products, services, experiences, or ideas to satisfy needs and the impacts that these processes have on the consumer and society.”
As a business, you aim to predict consumer behaviour so that you can tailor your products or services to your customers, but actually understanding the behaviour is possibly one of the biggest challenges a business will face. Both big data and market research will face up to this challenge, but in different ways. Deciding whether you use big data or market research ultimately comes down to what you want to find out.
|
Tweet This |
|---|---|
| "Understanding behaviour is one of the biggest challenges a business will ever face." |
Big data can provide businesses with trends to show purchasing patterns, for example, over long periods and offers the ability to analyse and track behaviours in real-time and at a large enough scale to allow you to confidently make decisions and predict future behaviours.
However, with big data it can be easy to get lost in the volumes of data and forget about the customer as an individual. As human beings, our behaviours are not always rational. What big data lacks is the ability to understand why the customer buys something in particular and identify why an oddity in the trend behaviour occurs.
You may know from your big data analytics that John Smith gets his groceries delivered every Friday between 9am and 11am and has done so for the last 2 years, so you can predict that’s what he’ll do in the future. You can even use the information about what’s on his shopping list to target him with particular offers based on products he buys on a regular basis. What big data can’t tell you is why that customer shops this way. Individuals and their contexts change over time and place, meaning the predictions we make from using big data does not account for oddities in behaviours. Using John Smith’s example again, if one week he did his shopping on the Saturday afternoon instead of the Friday morning, you would never know the reason why. That’s where market research comes in.
Typically, market research seeks to ask participants to recall or predict their behaviour in a given scenario – their opinions. Market research is the pursuit of meaning and add emotions and reasoning. It provides understanding for why big data trends occur, telling us the how and why to help implement change. Market research would set out to understand how John Smith does his shopping and his motivations behind it. With this, predictions can then be made about his future behaviour based on his opinions and attitudes.
In short, market research is the intention and big data is the resulting behaviour.
In Summary
Most evidently, the size of the data dictates how we handle it, the way we analyse it and the insights it creates. In my opinion, these are the key differences to consider between big data and market research. Deciding which data you use comes down to what you want to know about your customers’ behaviour – if you want to know the ‘what’, big data analytics will tell you this; if you want to know the ‘how’ and ‘why’, you need market research.
Big data and market research should only ever be used to complement each other, not to replace one another. They both have their own purpose, benefits and even challenges. But if used correctly, they can be extremely effective in understanding customer behaviour.
I’d be interested to hear other peoples’ take on the issue; whether you can think of any other differences I’ve not outlined here or your thinking on big data and market research in general – leave your comments below to share your thoughts.
