At the start of automation integration into research processes, there was a lot of scepticism in the insights industry - in fact, at the start, there was a lot of scepticism in all industries. With fears that researchers would be made redundant and that this was the gateway to AI takeover, it was a tough time to get anyone involved in the process at all.
Since then, research automation has come a long way in the insights industry. With initial fears being assuaged by the inhuman-based flaws in automated research processes and the need for researchers to check the accuracy of automation, we have come to embrace the benefits of automated aspects and focus on the tasks that really need our attention - humans and machines working together as one to boost efficiency and accuracy, and eliminating bias as much as possible.
Automating Efficiency in Recruitment
In the recruitment stage of market research, insight teams face a minefield of obstacles when attempting to recruit the right participants for their research project. Trustworthy, active, engaged panellists are a researchers dream, but they are so far and few between that it can be hard to find the right ones, never mind in the right timeframe who are actually interested in contributing their precious data for insight generation.
Normally, researchers come across repeat respondents who join many research projects because they have the time, professional respondents who are like repeat respondents but only really do it for the money and say what they think researchers want to hear rather than accurate data, and unresponsive participants who join up and then take part in either little or no research tasks, skewing the results.
When it comes to recruitment, most of the automation required in this area is dedicated to sourcing the right research participants, uploading those respondents onto a research platform or panel, and sending out mass communications and task invitations.
Within all of these tasks, there has been a revelation of automative innovation to help maximise efficiency, boost the accuracy of recruitment, so insight teams can concentrate more on building the research experience that these freshly recruited participants will go through.
This automation helps insight teams increase the quality of respondents recruited into the project, making sure they aren’t repeat or professional respondents (or at least making sure those that are are few and don’t skew the data), while also decreasing the cost of recruitment, whether that cost is paying a recruitment agency or acquiring the software to recruit the respondents themselves.
The machine learning algorithms that these automative softwares are built on have significantly ramped up the speed in which respondents can be found and recruited by searching global databases with specific demographic parameters and increasing the efficiency exponentially; but the more emphasis the developers of these automation algorithms put onto speed and efficiency, the less they focus on other areas of development - such as algorithmic bias.
In a perfect world, algorithms would be unbiased when selecting research respondent candidates, but these algorithms reflect the biases of their human programmers, meaning that they could dangerously skew the research sample. The more work that goes into building these algorithms, the more refined, efficient, and impactful they will become, so we still have a little way to go until recruitment automation is perfected.
Automating Better Fieldwork
Researchers have a lot on their plate when it comes to conducting research, from liaising with stakeholders and respondents to analysing and presenting data, with a lot more in-between. So in the heavy-duty data collection and analysis stage of research, insight teams already have a lot on their plate, and this is where automation comes in to help.
In the insights industry, we have developed a number of softwares to take some extraneous parts of the data collection, sorting, and analysis process from researchers so they can focus on making sure the data is relevant, accurate, and ready for actionable insight generation.
With platforms such as InsightHub, capable of mass communication, and automating both quantitative and qualitative analysis processes, researchers have more time to dedicate to quality control, moderation, and insight generation than ever before. Statistical analysis automation can come in the form of cleaning files, merging datasets, and identifying variables patterns for researchers to then take and work with.
Qualitative text analytics softwares are constantly improving, picking up more detail than before in text answers and spotting patterns, repetitions and more to save researchers time and effort. The task of the researcher is then to pick up on the nuances that the automation will have undoubtedly missed and generating insights from that data in time to deliver the most impact in stakeholder decision-making processes.
There are also technologies capable of video auto-transcription for easier qualitative text analysis, and image analytics in which the automation algorithm recognises specific shapes and patterns in the image, and then gathers quantitative data for further analysis.
With larger datasets (such as big data datasets) it would take researchers many hours to comb through it all to catch all the insights available. In cases such as this, automation algorithms are more than necessary to pick up most of that work and point out statistical patterns and correlations while researchers work to understand the deeper ‘why’ these patterns have occurred.
Automation and Advocating Action
Lastly, automation in insight reporting focussed mostly on data visualisation and communication. Automating data visualisation helps researchers not spend their time on the aesthetic of the report; with both branded and non-branded templates and automatically generated data graphs, charts and images, insight teams save a lot of time not making the little decisions - how should this be presented? What reporting method should I use? Should this be distributed to all stakeholders or just the few that need it now?
This last question could be used to argue that automated insight delivery channels be set up so stakeholders know exactly where to find the most up to date insights at all times. Insight or data warehouses, or simply an automated data dashboard could help streamline the reporting process and take away all the faffing with little decisions. Automated dashboards facilitate instant access to real-time data and insight generation, and some contain features that automatically searches through accessible databases to retrieve new data and integrate it into the dashboard stakeholders are using.
Tools like our own VideoMR are great for automatically storing video footage recorded from focus groups and surveys in an online video library for researchers to then sort through, analyse, and create video montages or clips to send to stakeholders. This raw footage creates a direct connection from stakeholders to respondents, and many insight professionals rave about the impact it has on stakeholders, getting them to act at a quicker pace than any business development team.
With the reporting and insight activation being the stage that insight teams have the least control over, automating their processes will leave them with more time to focus on how to make their insight delivery channels and reports reach and influence more people to act.
