8 MIN READ

3 Challenges that Data Still has to Overcome

Emily James

How to Use Synthetic Data in Market Research

In recent years, the rapid advancement of technology and data analytics has revolutionised working p...

9 MIN READ

Dr Katharine Johnson


    In whatever shape or form, whether in our personal lives or professional, data is the driving force behind every decision we make. Quality, accurate data has the power to drive us towards goals, or keep us away from them. It is what makes our world turn.

    But data isn’t without its issues. There are a number of challenges that data still has to overcome, and here are three of the big ones:

    1. Consumer Intention-Action Perception Gap

    We have known for a good number of years about the Intention-Action perception gap. Behavioural science has enabled us to detect and better understand this phenomenon, but no matter how accurate we try to collect our data, we will still be at the mercy of this perception gap. We have not yet figured out a way to close the gap and create truly accurate and risk-free data.

    Tweet from FlexMR Tweet This
    Data has the power to drive us towards our goals or keep us away from them - but there are challenges that we must overcome before we can create truly accurate and risk-free data.

    The issue here is the warring states of logic vs emotion; what behavioural science experts call system 1 and system 2 thinking. System 1 makes 97% of all of our decisions and usually happens unconsciously (we have around 35,000 decisions to make every single day and don’t have time to think about them all individually); system 2 is the rational, logical side that takes longer to deliberate over the more important decisions and makes the right choice rather than the impulsive choice.

    When we conduct market research, how can we know which system of thinking that the participants are using to answer the questions? With our research being voluntary, we can assume that they would be using the more logical system 2, but with that system of thinking ruling the minority of the time, how can we consistently get the more realistic system 1 answers?

    Recent gamification techniques, while better known for increasing engagement, has also been employed to encourage participants into system 1 thinking for our observation. This technique can be enhanced with the technology of Virtual and Augmented Reality, but with the technology still not widely available this has become a secondary practice. Other theoretical practices such as Surowiecki’s Wisdom of Crowds, which puts forward the idea that large groups of people are collectively smarter than individual experts (but these crowds must be characterised by diversity of independent opinions), can be implemented to compensate for any biases that might otherwise occur in small samples, and can be more accurate through averaging of answers, but this remains a complex issue with no direct solution.

    2. Lack of Engagement

    We can produce all of the data in the world, and it would all be useless unless those who are in key positions of influence paid attention to and acted upon it. But this is one of the bigger issues that insight professionals have been fighting for decades, and we are only just getting to a point where we’re starting to address it. This challenge of getting key stakeholders and decision makers to pay attention to the data and insights generated is one that should have been tackled a long time ago.

    Graphs and data tables are great for providing a quick visual of the results, however, the reporting stage of research experiences hasn’t been innovated as much as the data collection or planning stages. This results in a disastrous monotony that has been perpetuated more and more as time has passed.

    At FlexMR we’re looking to kick-start a reporting revolution by exploring the most creative ways that we can report key data and insights; currently, our Consumer Postcard Project experiments with presenting insights as interpretable art-pieces, to encourage stakeholders to really study the insights and delve more into what the consumers might mean. This engagement with the research results will automatically mean that they stick in the mind when the time comes that decisions need to be made. There are many other ways of creatively reporting insights; we just need to take the time to discover them.

    3. Organisational Silos

    Lastly, this last issue I want to talk about links in well with the data engagement issue. The Silo Mentality is “a mind-set present in some companies when certain departments or sectors do not wish to share information with others in the same company.” Now this definition makes it sound like there is a purposeful retention of the silo, but the formation of silos can also be an unconscious action that needs to be rectified.

    However they’re formed, these silos can have detrimental consequences to the success and evolution of the organisation. From a reduced efficiency of the overall operations, to a reduction in morale and a tendency to ignore any data that does trickle through the cracks between department walls, these silos typically result in the degradation of company culture and organisational success. At a very basic level, walls prevent data and insights to flow between departments, and it prevents the production of right type of data that needs to be generated for each department to work towards the business goals as a whole.

    But for these silos to be broken down, an organisation-wide culture shift needs to happen. This is not an easy fix. Whether a business dives in at the deep end or changes things incrementally, this culture shift needs to work towards the organisation becoming data-driven, customer-centric, and transparent with all of it’s actions. In terms of recognising the value of data, and the opportunities to get involved with market research, transparency throughout the organisation is key. This transparency will enable all departments to understand what research is being conducted, what data there is already available to inform their decisions, and thus what research still needs to be conducted.

    Tweet from FlexMR Tweet This
    Challenges such as the Consumer Intention-Action Perception Gap, a lack of stakeholder engagement, and the perpetuation of organisational silos are only three of many that data faces.

    This isn’t just theory; after 144 years, the current CMO of Prudential rewired the organisation through digital transformation, that enabled consumer data to become the central driving force behind the success of the business. The Prudential case study proves that this challenge can be overcome, but it will require a herculean effort from all employees within the organisation.

    Overcoming Challenges

    Data by its very nature enables others to overcome challenges of their own; as such, it should be able to overcome its own. But often, these challenges aren’t anything to do with the data at all, just how it’s perceived by others. Challenges such as the three mentioned above prove that data faces challenges in every single stage of the research process and beyond, it’s up to us to clear the path and let data drive organisations around the world to success.

    You might also like...

    Blog Featured Image Header

    How to Use Synthetic Data in Market...

    In recent years, the rapid advancement of technology and data analytics has revolutionised working practices in the market research industry. One significant and growing innovation to watch is the pro...

    9 MIN READ
    Blog Featured Image Header

    How Delivery Platforms Reshaped the...

    Have you noticed how ordering food today feels like a whole new experience compared to a few years back? The restaurant and food delivery industry has been on quite a journey, and it’s amazing to see ...

    8 MIN READ
    Blog Featured Image Header

    Market Research Room 101: Round 2

    On Thursday 9th May 2024, Team Russell and Team Hudson duelled in a panel debate modelled off the popular TV show Room 101. This mock-gameshow-style panel, hosted by Keen as Mustard Marketing's Lucy D...

    8 MIN READ