Creative Production

Is AI really smart enough to interview itself?

Insight teams are being asked to deliver more and more research to tighter deadlines, and yet their budgets aren’t increasing. There’s a growing need for them to use technology to spend their money more wisely while also keeping up with increased demand. This is where AI is starting to prove essential, helping companies deliver findings that are on time and on budget.

How brands use technology in their research will vary with their needs. Most need a blended set of support options. Sometimes they may want an expert from an agency heavily involved to ensure they get advice and support throughout the process and avoid groupthink sneaking into their research.

Where it makes sense, from both a speed and financial perspective, AI is now there to help companies do the research for themselves. QuestDIY – Stagwell Marketing Cloud’s gen-AI tool to create, target, and deploy surveys at speed – democratises market research with AI technology so that you can just input your research topic and it will draft and script the survey automatically.

A company can then edit the final questions before the system invites the target audience from our many panels. The results can be delivered in hours.

The gains are obvious but the real questions are related to AI generated or synthetic data services that are being launched. Here AI doesn’t just help with formulating questions and understanding the responses, it will actually answer the questions itself.

This will be what I call the ‘third wave of AI’ in research and, while this will inevitably become more mainstream over time, I believe companies should be primarily focussing on near-term opportunities to leverage AI to speed up and reduce cost of their current methodologies.

The first and second AI waves

The first wave of AI in research has added value through efficiency and speed gains related to text. Questions can be formulated and the answers in open ended comments gathered for analysis, even if they are found in a video or audio recording of an interview.

The second wave, which we’re starting to see happen, is shaping up to be around the analysis of structured data. It’s here that we’re likely to see the biggest impact, beginning with discoverability. Lots of organisations carry out a significant amount of research but they struggle to take it all in and find the answers that are buried deep in a variety of studies. If you’re an airline, for example, you may be able to ask your AI research tool to reveal everything you know about seat comfort.

This natural language querying will interpret that and find research that has been done on the search term ‘seat comfort’, but also all related topics, for example leg room or seat spacing or lighting. It can tell these aspects are relevant, even if they weren’t specifically asked for. It can then bring together all these findings that otherwise may not have surfaced.

This discoverability will be really helpful but then so is the personalisation of findings. Executives will soon find they are not just getting a generic report or a set of data tables and findings from the company’s research. Instead, the results are built around that person and their role.

The most relevant answers from what we have learned about seat comfort would be different for a marketing manager looking to run a campaign than, say, an airline maintenance safety engineer. The ability to maximise the value of each study, by interpreting the results for different use cases, will enable research data to inform many more business decisions.

The third wave – synthetic data

The third wave, though, is going to see AI produce synthetic data by taking on the personas of people and answering survey questions. Instead of asking a group of people, you can brief Gen-AI to understand all of the different people you might want to run the survey with. So, if you’re in the UK, what is the makeup of the UK population in terms of age, gender, region, job, household income, and whatever other attributes you want for your audience. This will allow you to create thousands of different personas to answer the questions in a survey.

The claimed advantages are obvious, but so too are the potential risks. Synthetic data will provide very cost-effective, ultrafast research. It’s particularly well suited to audiences you might not be able to attract in great numbers, such as CEOs of the Fortune 500 companies, or when the topic of the research is controversial or has privacy concerns.

There’s also the application of carrying out pre work. Before running an expensive study with real respondents, the idea will be to run it a few times first with a virtual audience to make sure you’re asking the right questions. You may find the results show something you already know, so posing different questions might add greater value.

It is also likely to be better at knowing the state of broadly stable markets, for example perceptual maps of mature markets, where you are trying to understand consumers’ perceptions of brands. But generating trustworthy responses on something that’s not known, e.g. views and opinions on a new product innovation, is a much harder challenge.

Real people for real results

The single biggest drawback with synthetic data, though, is how easily it will create plausible looking findings that may or may not be representative of the intended population. You can go to any of these LLMs, tell them what you want to run the research on, and it’ll give you results. They’ll look plausible, but they’re not necessarily predictive of anything.

This is going to be exacerbated by how AI models are trained. LLMs primary training data comes from the internet which over-indexes on Western culture and English language content, and so research using these models are likely to inherit these same biases.

The major tech companies are training their AI to be very vanilla. They try to avoid controversy, and they will avoid certain topics, such as politics, gun control or religion, limiting the potential use of these techniques.

Like all new methodologies in market research, it takes time to develop and validate them.  I have no doubt that synthetic data will become part of a researchers tool kit and I am excited to see how it can deliver value – particularly in areas that are too difficult or expensive to currently investigate.

However, while these new methods are being developed, there is a wealth of opportunity for procurement teams to leverage AI to help create, interpret, analyse, compile and personalise research with real respondents. This is going to allow them to do so much more with their budgets and help insight teams keep up with the growing pressure on them to do more work within short time spans.