“It’s just a chatbot with a script!” Debunking this and six other misconceptions around AI-moderated pharma market research interviews

AI-moderated interviewing emerged as an extension of the shift to online qual accelerated by COVID. According to Greenbook’s GRIT tracking, COVID boosted online qual adoption to 85% when in-person interviews weren’t possible. That remote-first infrastructure is what AI moderation got layered onto once GenAI matured enough to run adaptive, real-time follow-up probing, with the first dedicated platforms appearing in 2022 and hitting the mainstream by 2024-25.

Today, AI-powered interviewing enables research to be conducted at scale and at pace, with every interview tailored to the individual with as much or as little automation as you wish. But, despite its advantages, there are still misconceptions around AI moderation, especially in pharma, with some questioning its rigor, depth of insight, respondent experience and transparency.

Chief Transformation Officer Dan Wasserman sat down with VP of Qualitative Research Marilisa Beatty to debunk some of the myths surrounding this style of research.

Myth number 1: AI-powered interviews offer a poor respondent experience

Marilisa: So, Dan, what’s a typical objection you come across from commercial pharma teams around using AI-powered interviews to gather primary market research insights?

Dan: I think one of the main beliefs is that respondents won’t like it and will disengage. Whereas the feedback we’ve had from respondents taking part in the studies we’ve conducted (and there have been many), is quite the opposite – around 75% consistently report liking the AI-moderated format. For HCPs (healthcare professionals) and caregivers, we’ve found this style of interview makes the whole experience much more convenient. HCPs can take part in interviews when they are not in the middle of a shift or a consultation, meaning they are more relaxed and willing to express themselves. Caregivers can fit the interview around patient needs at a time that suits them.

Myth Number 2: Respondents won’t open up

Marilisa: You mentioned that HCPs are more likely to open up, but can the same be said of patients, who might be uncomfortable talking to “a robot” so to speak?

Dan: First of all, you’re not exactly talking to a robot. I’ve seen other companies who do literally offer that. With Connection, KJT’s AI-powered solution, the person is simply answering a series of questions that appear on screen. But we’ve found that what happens with a lot of patients is quite the opposite. A quote that sticks in my mind came from a patient who said after their interview, “I can’t believe I’ve told you all this!” Often, it’s a very cathartic experience for patients, almost like therapy.

At KJT, one of our specialties is conducting market research for rare disease assets. Living with a rare disease can be a very isolating experience. Often the patient has had to live with symptoms for many years before being properly diagnosed. KJT’s Connection AI-powered interviews, which are tailored and adapted to every patient, allow them to talk freely and at length about their journey, their needs and challenges. In rare disease, pharma clients are aware that, because of the scarcity of patient numbers, every voice matters. And patients feel very strongly that they are helping progress treatments for their generation and, potentially helping to find cures for the generations that come after them. The moderator is the AI, but the insights come from real human voices.

Myth number 3: It’s just a chatbot with a script

Marilisa: Do you find clients don’t always understand how the methodology works?

Dan: Sometimes, yes. I think people aren’t aware of the capabilities that AI questioning offers. They think of it as “a chatbot with a script” – just a pre-defined questionnaire with a few varieties of routes to go down. In fact, the distinction is adaptive probing: the AI follows up on what a respondent says in real time, rather than branching through pre-set paths. This is usually one of the first things we try to clarify. With rare disease patients, we aren’t talking to a homogenous group of people. Each patient presents differently, has a different set of symptoms, could have been diagnosed and treated differently. They are also based in different countries and speak different languages. The ability to go down their unique path with them, no matter where in the world they are, is a very important capability that AI offers, without losing nuance, and without taking a lot of time to project-manage.

Myth number 4: You can’t cover sensitive topics

Marilisa: I was going to say, as a firm believer in the power of qualitative research and in-depth moderation, how does AI powered interviewing match up when it comes to covering sensitive or emotional topics?

Dan: There is definitely a misconception that AI moderation won’t be very good when it comes to handling sensitive topics. In fact, we’ve found that by removing any potential biases from traditional interviews, respondents can feel they can more easily talk about things that could be difficult to talk about with a stranger. It’s not always the case, but it can be true. Another one of KJT’s specialties is in metabolic diseases, particularly obesity. It’s a reality that overweight people can be subject to judgement and respondents can be wary of opening themselves up to that scenario. With an AI-moderated interview, they’re completely anonymous and free to say what they want. The landscape for GLP1s is changing rapidly. With AI-powered interviews you can get very quick feedback in a dynamic environment, with the insights able to help with segmentation of your market and differentiation of your asset.

Myth 5: The AI won’t ask the right questions

Marilisa: Is there a concern that the AI will ask inappropriate questions?

Dan: As mentioned before, the AI isn’t just a chatbot with a pre-determined set of questions and paths. The platform that Connection uses in partnership with Listen Labs is very sophisticated – it’s layered on top of leading foundational language models and it’s getting better all the time.  

Marilisa: I have to say, as someone who has been moderating for almost 20 years, I am personally impressed by the quality and level of probing the AI provides. The first time I saw it I was shocked.  In programming, we do give the AI guidance for where or how to probe. For any question that we want to have asked as-is, we program those in directly as well. Otherwise, we have the flexibility to have the AI probe on its own based on the objectives, and it does a fantastic job of doing that.

Myth 6: The richness and quality of insights isn’t as good as a qualitative interview  

Marilisa: Do you think an AI-powered interview offers the same benefits as a traditional in-person interview?

Dan: Typically, an AI-powered interview is shorter than a qualitative interview, say half the time. But that doesn’t impact on the quality and richness of insights gathered. The interview can be shorter simply because there’s less time needed for the typical moderator-respondent back-and-forth, or introductions and transitions, while still allowing the conversation to adapt to what the respondent says. Pre-programmed questions are customized in real-time based on respondent responses. AI captures and analyses the qualitative insights identifying key themes, trends and emotions and then our researchers review and validate findings for clarity and relevance. If you really want to go deep into perhaps more irrational beliefs or behaviors, an in-depth interview with a moderator may be the better option.

Myth 7: It’s trying to replace qualitative in-depth interviews

Marilisa: I was going to say, a lot of people think AI moderation is trying to replace human moderation and traditional qual.

Dan: Not at all, in fact, we find that because AI moderation runs at scale, people can assume it’s really a quant survey with open text boxes. We believe AI moderation does something structurally different from both qual and quant – it’s not a hybrid or a compromise between them. When combined with a quant survey, it allows you to dig into why something is happening, which means it can be really powerful – for example, when employed as an add-on to a regular brand tracking study. Or it can be used on its own in concept testing to evaluate the best creative or TPP (target product profile). We can also use it in combination with a MaxDiff to determine the optimum order and number of messages for an asset. In device testing, we can use Connection to explore optimal features and benefits. And in rare disease we have used it to understand barriers to use and switching behavior. There really are a lot of use cases and none of them is trying to replace the traditional in-depth qualitative interview.

In closing: Does AI-moderation ever fall short?

Marilisa: Are there any times you wouldn’t recommend an AI-moderated interview?

Dan: Possibly in certain areas of oncology, where there are a lot of acronyms, types of cancer and lines of therapy, the AI might struggle to interpret the situation correctly. If there was one watch out for AI moderation, I would say it’s about ensuring the quality of the transcripts. You need to be mindful of the AI not accurately picking up very thick accents or muffled voices. But we build that into our quality control procedures to ensure it doesn’t impact on the final output.

Another consideration is that, unlike a traditional qualitative interview, you can’t observe an AI-moderated interview live. Sometimes that live observation is an important part of the research experience, particularly when you’re trying to socialize the research across a larger team, build empathy for the people you’re speaking to, or give stakeholders direct exposure to how people think, feel and talk about an issue. In those cases, traditional qualitative research may be the better choice.

Marilisa: What if people still aren’t convinced?

Dan: If they have already have a business question in mind where they think this methodology would suit, they can contact us for a briefing call and a demonstration of the tool with their team. This methodology is fast, scalable and relatively inexpensive – it could be trialed as a low-risk low-cost way of testing its effectiveness for quick answers.

Authors

Dan Wasserman

Chief Transformation Officer and Head of AI Solutions

Dan Wasserman is KJT’s COO, leading the integration of generative AI into client solutions to drive innovation as part of Sparq Intelligence. With expertise in compliance, operations, and analytics, he transforms complex data into actionable insights. Dan values collaboration with KJT’s teams to help clients design better products and services.

Marilisa Beatty, MA

Qualitative Research, VP

Marilisa is a seasoned qualitative researcher with over 16 years in global healthcare research. She has shaped strategies for top pharmaceutical brands and emerging biotech firms. She is known for her problem-solving skills, innovative research methods, and personable rapport with healthcare professionals, patients, and caregivers.