Every time we interact with the online domain, we contribute to an ever-growing stream of information about our usage, behaviour and interactions with whatever sites or pages we use. A quick search suggests that of data are generated daily and by searching for that information I have contributed to that amount.
A staggering amount of data is generated passively through our online behaviour and makes up part of what is referred to as Big Data, providing valuable opportunities for market researchers as well as some pretty large challenges. Let’s take a look at the developing relationship between market research and Big Data, how they are evolving, and how they intersect.
Market Research and Big Data
Big Data is collected constantly, providing dynamic, real-time information on what we are doing at a scale, which makes most traditional market research sample sizes look minute. It is a fantastic resource for user experience (UX) researchers who want to uncover the behaviour of users on their sites and services to drive positive outcomes – companies like Netflix, Facebook and Amazon have become synonymous with Big Data, helping to produce recommendations on future purchases, viewing suggestions, and relevant advertisements.
Big Data provides the ‘what’ and the ‘how’ for researchers, but falls short of providing the all-important ‘why’ – you’ll still see opportunities for you to leave feedback on whether a recommendation was suitable for you, or why you no longer want to see a certain advert. These are the points at which we start to move more into traditional market research, and look at the intention behind our behaviour.
For market researchers, Big Data can be a blessing or a curse, depending on your point of view. There have been some in the industry who feel that, as Big Data develops, it will replace the need for market research all together. A much more optimistic viewpoint is to see Big Data as one of the most important tools researchers have at their disposal.
 |
Tweet This |
|---|---|
| The intersection of Big Data and market research represents an opportunity to scale up market research beyond what would be feasible normally - how can we make this happen? |
Traditional market research can be expensive (recruiting participants, designing research methodologies and fieldwork all cost money) but it provides a depth of information that is hard to acquire by other methods. Conversely, Big Data provides a vast amount of information on what users are doing, but can’t always provide the depth needed to provide actionable insight, and the volume and structure of Big Data provide additional challenges for analysis.
By bringing together market research and Big Data, researchers can use Big Data to evidence the finding of market research projects, relating insights to stakeholders with scale which is difficult to ignore, and analysts of Big Data can apply market research to dig further into observed behaviour to uncover the reasons behind changes or characteristics.
The intersection of Big Data and market research represents an opportunity to scale up market research beyond what would be feasible normally, while bringing depth to the insights developed through Big Data analysis to better utilise the wealth of information available.
What Comes Next?
For Big Data to be utilised successfully, it needs to be analysed and reported properly – sounds obvious, but this can be more complex than you might expect. Due to the sheer volume of Big Data, as well as the variety of data structures and its dynamic nature, analysis through some of the more traditional methods isn’t always feasible.
Technologies including Apache Spark, Power BI and Alteryx have all grown in popularity due to their ability to make sense of the vast amounts of data that can be utilised, as well as for communicating findings to stakeholders. For market researchers to make use of Big Data effectively, knowledge about how to use similar tools will likely become necessary. We have already started to see changes in the way we visualise and present insights due in part to Big Data; the dynamic and constant stream of fresh data that Big Data can deliver has meant real-time analysis has become more important, with tools such as real-time dashboards becoming more popular.
One of Big Data’s biggest draws for market research is it constantly adapts to current trends and behaviours, but this means that we need to find ways of communicating this effectively to stakeholders continuously. As Big Data becomes more integrated into market research, they ways in which we communicate findings will evolve to best suit new methodologies.
Artificial Intelligence, or AI, is likely to play a part in the evolution of Big Data and market research. It’s likely that you will have heard of or interacted with some kind of AI over the last few months. Tools such as have introduced AI to a variety of sectors and prompting many of us to think about how we can adapt and enhance our working practices with AI. Large volumes of data are essential to machine learning and AI, so it is of no surprise that Big Data is an area where AI can, and already is, having a major impact. Some of the largest obstacles to accessing insights from Big Data, such as the time it takes to properly structure and spot patterns in the data, are areas where AI is particularly efficient.
Introducing machine learning to our processes when dealing with Big Data improves efficiency when it comes to data preparation and pattern recognition, allowing better data-driven insights to be developed quicker, and at a scale larger than a single researcher could operate. Combined with domain knowledge and the depth offered by market research, AI has the potential to uncover actionable insights in Big Data that market research alone would have struggled to find.
|
Tweet This |
|---|---|
| There are many applications of Big Data in the market research space, and with the advancement of technology, there are new applications being discovered everyday. |
Any Downsides?
While Big Data can provide a vast wealth of insights on our behaviour, its collection also produces its own problems, including data discrimination, data security and data privacy. Through Big Data, we as researchers know much more about the behaviours of users, and it is therefore our responsibility to verify the validity of the data and ensure it is representative of the true population, provide insights that don’t discriminate against or overly favour different groups, and that we store and utilise Big Data ethically and securely.
Acting responsibility seems like an obvious part of our roles within market research, but, as Big Data evolves and becomes more intertwined with market research, it is important that our approaches to dealing with these issues evolve as well.
In the End
Big Data is here to stay, and represents an opportunity for market researchers to access huge amounts of valuable information. Considering how practices and applications of Big Data in market research will evolve over the coming years will help you and your organisations stay ahead of the curve, as well as avoiding any pitfalls that may await.
The applications of Big Data within the market research space are many, and with the advancement of technologies such as machine learning/AI and data analytics tools, it is becoming more accessible. I would encourage anyone interested to go take a look at what is happening!
