Using NotebookLM for user research: Useful, but ultimately just another LLM

NotebookLM is useful, but it’s still just LLMs deep down. There’s no magic technology that makes it immune to the problems of other AI systems.

Here’s another question from one of my past workshops: “Why not just use NotebookLM for user research analysis? It seems pretty much made to upload multiple transcripts and cite sources accurately.”

Here’s my answer:

In June, I did an exploration of how NotebookLM works under the hood.

NotebookLM is basically a fancy RAG LLM system, with some custom prompts, and an integration with DeepMind audio models (so it can turn text into speech). The LLM it’s currently using is Gemini 2.5 Flash, which is just another long-context model, like GPT 4o or Claude 3.5 Sonnet.

This means it’s subject to all the same issues:

  • 🤖 RAG can grab the wrong chunks of text in your documents, so it misses key points.

  • 🤖 The LLM can misquote its sources.

  • 🤖 There’s no guarantee that the themes the LLM generates will be correct or useful.

So there’s nothing magic about NotebookLM. It makes it a little easier to upload your documents, but it has all the same issues of any other RAG system.

And these problems are evident in NoteboookLM’s user reviews when you start to dig into them–there are reports of it misquoting sources and leaving out critical information, just like we’d expect based on its underlying technical architecture. And just like every other major LLM, it has a disclaimer in the footer: “NotebookLM can be inaccurate; please double check its responses.”

And while I may not recommend blindly trusting any LLM, I do think it’s safe to trust that disclaimer.

🧩

If you have questions about AI and user research, join me at one of my upcoming workshops:

  • If you’re trying to use AI research tools more effectively →

👩‍🔬 AI for UX Researchers (October, Rosenfeld Media - register here)

  • If you’re building AI products →

👩‍💻 User research for AI-enabled products (November, email me for info!)

I'd love to see you there.

AI, Methods & toolsLlewyn Paine