The Challenges of Integrating AI into Online Research Software

Everything seems to come with a sprinkling of AI fairy dust these days. Whether it’s summaries of toaster reviews on Amazon, marketing collateral featuring customers with more than the usual number of digits, or respondent cohorts of synthetic personas, AI is creeping into many areas of our lives and industry.

For online research software, AI brings loads of opportunities for helping users to streamline tasks so that they can do more with less, particularly in these straightened times for the market research industry, plus being able to help you to focus on the key parts of your job. AI in market research software, well implemented, should act like an e-bike compared to a normal bike: you’ll be able to get a lot further a bit quicker with a lot less effort and possibly even have more fun along the way.

Selecting the Model & Honing Prompts

In dealing with AI, just like in writing research topic guides, 80% of the challenge is asking the right questions in the right way. Also, there’s a reason that “Prompt Engineer” is now a job title! Trying to ask the right questions of AI models to ensure a balance between efficiency and accuracy is quite an art. As with all software design, there’s always a balance to be had between offering lots of configuration options, ease of use and in this case asking the right questions of the models in the right way.

How to overcome this:

• Test a range of different models to see which is the best fit in terms of your use case in terms of price, output formatting, how recently training data sets were updated and your use case (such as whether you want to summarise data, handle images etc - different models work better for different scenarios).
• Experiment with asking for different numbers of characters of generative text (you may be surprised that some models provide fewer or more characters than you initially ask for for example).
• Make sure your prompts are specific and well-formed in terms of inputs and outputs. For example, do you want 3 bullet points in a summary or a short paragraph? Can you tell the AI model what the data is about to help guide it to the most useful output?

Data Privacy & Information Security Concerns

Many clients are naturally concerned that their customer data may be passed outside GDPR jurisdiction or used to train large language models without participants' consent. As well as information security ramifications, this could negatively impact how willing customers are to take part in our research.

How to overcome this:

• Work with clients to ensure that they can control when, how and whether their data is being handled.
• Introduce pattern matching and intelligent data cleaning to ensure that Personally Identifying Information (PII) is removed.
• Work with Information Security experts throughout development to ensure compliance.

Scalability & Performance

AI is often most useful and effective when used with big data sets. When conducting online research, you can amass an amazing and diverse amount of data over time. However, this brings its own challenges of scalability in terms of preparing large data sets to be submitted, timings required for processing, handling responses back to the platform and then integration of results/outputs with other tools.

How to overcome this:

• Test large datasets to ensure that response times are in line with client expectations.
• Ensure that measures are in place to handle tasks that take longer than expected, so as not to have an impact on the performance and stability of other parts of the system.

Managing Expectations

For years (even before we started developing InsightHub!) we have joked about having a ‘create report’ button at FlexMR - by which all your hard work from weeks of planning, designing, running and analysing a research project can be distilled into a bunch of insights at the touch of a button. Though there are companies who are starting to offer this feature, for many of us, it’s hard to know - or imagine - the bounds of AI-assisted research. 

Ultimately, while generative AI will totally create a report about your research, it’s likely going to be pretty generic and lack the organisational insight that comes from being within or working closely with a business. Where AI excels is in taking away the kind of tasks that humans did a bit better than software before but in a less arduous way (for example, scheduling focus groups; doing a first-pass analysis of qualitative data, etc). 

Therefore, when developing AI-based functionality, it’s important to be able to frame users’ expectations about what the aims and scope of the tools you’re providing.

How to overcome this:

• As for all software development, be clear about the problems we’re trying to solve and for whom.
• Ensure we’re asking the right questions of the data.
• Help users explore the full possibilities and best practices for using new tools.

The Future of Research and AI

For some insight professionals, AI can feel like a threat, with fears that insight functions could be replaced by bots to save money, but expectations of what AI implementations can currently offer may be too high. AI-assisted software is about taking the boring bits of research away and allowing you to more efficiently create actionable insights and work with stakeholders to embed these - and the changes they guide - in the organisation.

To work with AI and integrate it into online research tools, we need to negotiate to work around some familiar and some new challenges such as balancing usability performance, information security and a rich and useful set of features and how we explain those features to users.

As market and customer researchers, we are ultimately a link between different groups of humans (customers and stakeholders). To ensure communication between different groups, we are more like interpreters than translators, adding layers of meaning, significance and culture to help ensure that decision-makers within organisations can understand their customers. However good a machine translation might be, it would take a lot to replace that tacit human knowledge, and the same is true within the research industry.