Verifying qualitative research analysis falls into Andrej Karpathy’s category of “lagging” AI tasks, depending, as it does on “real-world knowledge…context, and common sense.” There’s no definitive “correct answer” to train a computer by. So what can we use AI for in product research?

We tested Google’s Gemini 2.5 Flash (used in NotebookLM) on qualitative analysis by running the same prompt multiple times.  It improved on GPT-4o and Claude Sonnet 3.5 at quote accuracy but still struggled with relevance and meaning.

NotebookLM is useful, but it’s still just LLMs deep down. There’s no magic technology that makes it immune to the problems of other AI systems.

Seen last week on social media: a market researcher trying to save their job after reporting a nonsense, LLM-generated analysis of survey data. It’s a common mistake, and there’s not much training available for researchers on how to account for differences between generative AI and traditional research tools.

The reality is that many researchers are now expected to use LLM-based tools in their work, despite their shortcomings. And it can be hard to push back on mandates from above, or competition from peers who don’t understand the risks. Are there ways we can integrate LLMs into research while still being responsible professionals about it?

When we use AI tools in product design, strategy, and user research, we need to keep something important in mind: the AI is not “understanding” our requests, and even with safeguards, it may not catch when it’s making mistakes.

Error-checking AI output takes time, so we need to be wary of claims of “time saved” using AI for product research. A thought-provoking quote by Emily M. bender from the March 31 episode of her Mystery AI Hype Theater 3000 podcast underscores this.