Almost everyone believes they can be a market researcher, whether they have any formal training or not. Indeed, I’ve long believed they can – but with some training! I believe the core attributes of a researcher are inquisitiveness, an interest in people and having a reflexive mind. Beyond this, the skills required are taught through a combination of formal courses and experience in the field.
But as more people choose to conduct their own research, training can often be forgotten. With this in mind, I’ve written up the top ten most common mistakes that we need to stop making as researchers.
1. Poor Sampling. If you ask the wrong people to answer your questions, your whole analysis can be flawed. While a bad question stands out, it’s not as immediately obvious when you have poor sampling. That’s why this is the absolute most important mistake to avoid. Always be sure to define your sample at the start. Write out who you want to talk to and why. Define the rules which will, in turn, define who is included in the audience. Then decide whether your audience should be divided into sub-groups that may require further analysis or cross-comparison.
Be clear about the quality that you want from your sample. This is increasingly important as the rise of access panels has meant that it is often not questioned whether they have professional respondents. This applies to both qualitative & quantitative research, as well as specialist recruitment.
2. Ambiguous Questions. If your question isn’t precise, implies bias or has a vague meaning, then your analysis will be open to interpretation and, at best, your findings may be resisted in the organisation. The worst case scenario is that this data is used to make the wrong decisions.
‘How concerned with each of the following are you?’ This might seem like a reasonable at first glance, but ‘concern’ is a broad term that can stretch from worried emotions to simple awareness. For example, the phrase ‘I’m concerned about traffic’ may mean ‘I’m worried because it’s an issue that makes my journeys longer’ to one person but ‘I am interested in finding out more information about traffic’ to another.
Ambiguity can often be found in the answer blocks. It is very easy to write answer options from your own perspective yet miss some critical ones that respondents might think of. This creates ambiguity in your analysis and skews the results. Take time to ensure that ambiguity in the wording of answer options don’t lead to overlap between some of them and confuse your respondents. For example, ‘I chose it because the price was reasonable’ or ‘the total cost seemed fair to me’ seem like good choices to provide people and things people will associate with. However, when it comes to analysis there will be a debate about what prices are actually reasonable.
3. Cost Cutting Incentives. Incentives are best thought of as a compensation for someone giving up their time. People are busy and research asks them to give up some of their time in order to help us with our work. Yes, they are motivated by their interest in a topic or by helping us improve, but these shouldn’t be used as excuses not to value a person’s time. We are not trying to influence or skew the research with incentives either – so we don’t want them to be harshly low or overly high; it is about being fair and reciprocal. They give us time and we compensate them for that time.
| Tweet This | |
| "Poor compensation of participants is one of the biggest market research mistakes you can make." |
There is a lot of discussion at conferences about poor response rates and low engagement in research, but we often we over-complicate this issue. One of the core reasons a study receives low response rates is because it unfairly values a person’s time. Compensate fairly and you will receive better quality research as well as higher response rates.
4. Poor Participant Communication. This issue is also closely tied to low response rates. Communication is an art and one that we don’t spend enough time thinking about. Researchers worry about crafting the questions and analysis. Yet success often depends on how instructions and information is communicated to participants. Marketers analyse response rates far more than researchers, but getting the right subject line to an email, making sure your invitations have a clear call to action and ensuring the incentive/compensation for taking part is clear all improve response rates.
Make sure your invitations and reminders clearly set out what you want someone to do and why they should take part. Write in a friendly and relatable way – don’t be too formal or professional. In a qualitative exercise, tailor the communication to the individual as much as possible and try to avoid the temptation to programme automatic reminders, as these may lead to alienating your participants or worse. Good communication is the foundation and platform for successful research.
5. Heavy-Handed Moderation. Conducting outstanding qualitative research requires the right balance of questioning – be open and not too invasive or else risk stifling the discussion. Too many quick-fire short questions can shut people down before they open up. Practice active listening as much as possible, confirm and reassure that you have understood their point before asking a follow-up question. Ensure the questions are framed in language your participants understand and follow the areas they are interested in as much as possible.
The best moderators will get the answers required for the topic, but they may do this by following a different structure than the one intended at the beginning; simply because they listened, allowed the discussion to evolve and occasionally guided it back onto topic if it strayed too far from the path.
Moderation isn’t only about firing out the questions that stakeholders want to know. Qualitative research is not meant to be a survey with routing logic and verbatim answers. It is meant to be a discussion that seeks to understand and get behind the issues and answers.
6. Limited Online Imagination. Your customer spend so much of their lives online or on their mobiles and yet, still, researchers lag behind in this field. If this is where customers are happy, then we really need to be talking to them in this space (especially if we want to make research engaging, convenient and interesting for them). Even now, the extent of a lot of online research is limited to survey collection and the odd structured question board.
There is so much more we can do online to make it interesting whilst also gathering more insight from it. But the main mistake we make online is relying on question boards that are treated more like surveys. They are programmed with routing logic, seek individual verbatim responses and don’t generate group interaction or have enough personal moderation. If we applied the same approach to a face to face focus group then researchers would be aghast. It’s time we stopped making this mistake and treated online research with the respect and imagination that we apply to our traditional research skill set.
7. Complex Objectives. Too often, research is conducted without a clear understanding of why and what is hoped will be achieved. Everyone needs to be on the same page and ill-thought through objectives will always lead to misunderstandings and perceived failures at the end. These decisions, confusion and complexity will then be used to judge the success of the outcome.
Objectives are not questions renamed objectives. Frame short bullet points that outline the aims for the project and the type of knowledge and understanding that is sought. If it helps to write example questions, then make sure you write these separately or as sub-points to each objective.
They shouldn’t be vague or ambiguous aims. They need to define the purpose and reason for the work too, so before you write them make sure you set out the background to the project and the reasons why it is being conducted, what the business will do with the results and how those decisions will be made.
8. Survey Overkill. Surveys are over-used. It’s easy to see why. Nearly everyone in a business knows what a survey is, and combined with the perception that they are quick and easy to run, nearly every department takes advantage of them. But, they are also often too long, too complicated and too engineered.
| Tweet This | |
| "Surveys are the most common market research tool because they are the easiest to understand. not the best." |
Before writing a survey, ask yourself which method or structure will engage the participant most. Often non-survey based methods are more valuable, but are overlooked because surveys are just easier to run. If a survey is the best method in your situation, try to limit the length of it so that it doesn’t bore your participant. Participants don’t mind answering surveys – as long as they are simple and quick to do. Don’t over-engineer it either. The current trend to gamify surveys can be helpful but only in the right circumstances.
9 Vague Reports. Avoid vague reporting. Ensure all findings are backed-up with clear evidence. Try to make your points as succinctly as you can and avoid language such as ‘some’, ‘might’ and ‘may’. Be specific about who your finding relates to, what it means and what can be done. Precision is needed if action is to be taken. Don’t over-state your findings either. If you do, then this can lead to the vaguest report of all.
For example, don’t generalise a customer segment or stereotype behaviour when describing customers. Don’t associate behaviours with everyone of the same age or geography. And most importantly, don’t apply your findings to the whole business or make sweeping recommendations that go beyond your remit.
10. Graph-pocalypse. Finally, don’t pack your report out with so much data that it becomes hard to follow or understand. When people talk about story-telling or creating a narrative, this is often what they mean. It is the difference between 50 slides of charts and selecting 50 facts that follow on from each other and are ordered to build on each other. Use evidence but don’t kill the project by charting absolutely everything. If a client asks for it, then create a structure that uses a summary and an appendix.
What market research mistakes have you noticed throughout your career? Share them with us in the comments below, as well as your top tips on how to avoid them.