We need to be wary of claims of “time saved” using AI for product research

Error-checking AI output takes time, so we need to be wary of claims of “time saved” using AI for product research.

A thought-provoking quote by Emily M. bender from the March 31 episode of her Mystery AI Hype Theater 3000 podcast, in the context of healthcare:

“This sounds to me like they implemented RAG…over something somewhere in their pipeline. And AI-powered document search and synthesis is terrifying when you’re talking about medical records and medical decisions because if that sped things up, that means nobody was checking. And that means patients had random nonsense entered into the system, maybe even in their clinical records.”

This is a concern any time AI is used in the context of data synthesis.

In my Rosenfeld “AI for UX Researchers” workshop, I help researchers learn how to assess the risks of AI synthesis through rigorous testing.

Attendees learn how LLMs “synthesize” documents at a technical level, as well as a structured process to evaluate new AI models and prompts for synthesis tasks. It’s an invaluable skill as new models like GPT-5 continue to be introduced, and product practitioners are left to determine for themselves where they’re safe (or not!) to use.

It’s a bestselling Rosenfeld workshop, and we have one more run scheduled this year in October. I’d love to see you there!