Good sampling doesn’t guarantee success, but a poor sample will guarantee failure.
Every new product or service that you see in the modern age is entirely guaranteed to have been subject to some form of market research. Within this process of research, there is one essential step that is arguably the most important: exposing your new idea to potential buyers/users to gain an insight into whether it is likely to be profitable or regrettable. The users that you allow to see your new idea early are referred to as your sample and ensuring that this sample is useful isn’t always simple but there are some steps that can reduce the risk of your research being flawed.
With everything new being a risk, every business is aware of one immutable fact, this is that the majority of new products, that begin with such hope and enthusiasm, end in failure (statistics differ but range from 60% to 95%). Everyone is aware of the famously bad failures, those so epic they make the news, Sinclair C5, Apple Newton and New Coke didn’t even require a Google and are seared into my memory. Outside of these famous failures, however, there are billions of new ideas each year that are squashed, most not even making it to a product launch, simply fading into ignominy.
Given this bleak outlook for almost every new idea, any step that can be taken to avoid this is essential and one of the first ports of call should always be doing your research. An obvious conclusion to anyone that is here, reading a market research blog, on a market research company’s website… but researching effectively has proven to be a difficult step to get right over and over again, leading to those failures we mentioned earlier.
| Tweet This | |
| Good sampling doesn't guarantee success - but a poor sample will guarantee failure. Here are 5 sampling issues to avoid to steer your research towards success. |
Doing your research is the default position for those wanting to avoid failure and embarrassment, but not all research is made equal. Poor quality research is not only incredibly common and harmful to success, but it often stems from one critical point: who you ask for feedback. Those that you ask are generally referred to as your sample and will be referred to as such from now on.
Unless you have thousands of friends that are willing to participate in your research (and happen also to be a broad demographic group) then you will likely be looking for a sample elsewhere and there’s a plethora of options out there that need to be approached with care. This sounds obvious but there is more to sampling than selecting from the appropriate demographic groups and sending them a questionnaire. Sampling needs to be planned and scrutinised with equal attention as you would any other research step.
First, you need to locate a sample source and this is likely to come from one or more of two main sources:
Once you have a sample or sample provider you must establish that the information held on your sample members is accurate, whether this is your own data or from a panel provider, you need to know that this is up to date. Much of the accuracy of the information depends upon when this was collected as old information obviously has a higher chance of being inaccurate.
Ask questions of your internal or external team regarding this, especially ‘When were they last profiled?’. Given the current economic climate and cost of living crisis, market demographics are changing rapidly and your sample group's spending priorities have likely changed drastically over the last 12 months.
Ideally, users should have been profiled in the last quarter to ensure that the data is up-to-date and accurate. If your internal data is older than 6 months or your selected panel provider can’t provide a date for the last profile then the data should be avoided.
Usually, your sample is based on the target demographic. Whilst targeting the broadest possible group sounds great, it isn’t really useful. Red Bull doesn’t really need to ask the over 70s about their new trial flavours, for example, it’s simply not their target demographic. Panel providers understand this and will help you to narrow down your sample to specific demographics in minute detail, albeit for a premium. To most, this this premium is worth paying to ensure that 100% of what you pay for is useful and we would always recommend defining your sample in as many characteristics as possible.
Where you are using internal data however you can reach as broad a demographic range as is available but it is important to still ensure that your actionable insights are taken from your most appropriate segments.
| Tweet This | |
| From weighing your sample to drawing insightful conclusions on your own - there are a number of sampling challenges that can derail your research project. |
Have your conclusions scrutinised (externally where possible). Knowing that you’ve done something right is incredibly satisfying, whether it’s nailing the exact ratio of ingredients in your morning brew or something more complex like a month-long multi-faceted research project, getting it right is a great feeling. The cherry on top of getting it right is having someone equally or even more qualified confirm to you that you’ve done it right. Regardless of your experience and influence level, having colleagues scrutinise the conclusions you have come to is an essential step to ensure that they are sound.
Should you work with a third-party research agency then these are perfectly placed to analyse your conclusions without any bias towards you or the company line. Whilst this will generally be a paid service, if there is room in the budget then a full review of your sampling, research and analysis is a step that pays for itself.
While we have focussed on how to ensure that your sample is relevant to you and your new idea, it is, however, easy to underuse a perfectly good sample and this is why a good sample doesn’t guarantee good results. One of the primary causes of ineffective sample use is hubris, over-confident marketers drawing conclusions from blinkered questioning has and still does cause some of the most severe and famous examples of market research failing. Understanding the issues detailed above will help you learn how to identify a good sample and use it well enough to gain the insights you need.