Conventional user research will fail in emerging tech innovation: Here’s how I handled it
Bing Image Generator
I started Llewyn Paine Consulting, LLC because traditional market and user research techniques fall short when it comes to creating new markets for emerging technology. I developed a repeatable, systematic approach to technology-customer matchmaking that can be effectively applied by ordinary entrepreneurs, product managers, designers, and researchers in the real world.
Several years into a successful UX career, working for a top tech company and with multiple product launches under my belt, I was blindsided the day that my user research training failed me.
I was on a call with a startup that was building a solution for hand tracking. The technology was impressive. They were part of one of the country’s hottest startup ecosystems. But they had completely fallen flat at finding the right customer. They came to me as part of a last-ditch effort to save the company.
Up until then, I’d only worked on projects that followed the golden rule of user-centered design (UCD): “Focus on the user, and all else will follow.” 1 And until then it had been easy, because identifying the user was someone else’s job. Once the user was defined, I was a powerhouse at identifying their needs, coming up with valuable product ideas, and refining them into delightful experiences. But it always had to start somewhere (or rather, with someone).
When faced with a high-stakes project with zero sense of who the user might be, I was absolutely flummoxed–and, to be honest, a little heartbroken, because it meant telling this startup I couldn’t help them.
As I strove to make sense of what had gone wrong, I initially assumed that the problem was with me. There had to be some other design technique to answer this question that I simply wasn’t aware of. But as I surveyed the literature and reached out to colleagues, I realized that it wasn’t my fault–it’s a fundamental problem with user-centered design.
User-centered design struggles with transformational innovation
The reason was clear: user-centered design is designed to eliminate wasted effort by building only what the user needs. (In this way it’s almost identical to the lean startup process, which has the exact same problem.) But if you don’t have a user, there’s no starting point. Any attempt to start from the technical “solution” is working backwards. It defies the central premise of user-centered design.
As a result, user-centered design works great in cases where you’re designing for known users, which covers the majority of development work at established companies, but it falls short when you don’t know who the user is.
Business theorists categorize innovation work into three buckets: core, adjacent, and transformational.2 Core innovation involves updating existing products and features for your existing users (a.k.a. your “market”). Adjacent innovation involves expanding into products or markets that are “new to the business” but that are already established. So even if you’re designing for a new (to you) market, you can understand who your users will be by looking at who uses competitors’ existing products.
Where user-centered design falls short is in cases of transformational innovation. Transformational innovation involves creating entirely new markets where there are no direct competitors. This means all hypotheses about who the user is will originate with you, and will need to be validated by you.
If you’re working on unestablished emerging technologies–such as, in 2023, hand tracking, AI, spatial computing, or augmented reality–you’re probably doing transformational innovation.
This type of innovation is harder and riskier, but it also has the greatest potential upside if you’re successful. However, the lack of a predefined user creates problems for innovation teams following user-centered design or lean startup processes.
The more I paid attention, the more I saw my hand tracking startup scenario play out again and again among emerging technology teams:
Entrepreneurs or in-house product managers were deciding early on potential markets using inappropriate methodologies like TAM (total addressable market), surveys of their professional networks, combing through white papers, or copying what another startup was doing. In nearly every case, these techniques failed and left them scrambling to find a better market too late in the game.
Designers and researchers were being excluded from this process (either deliberately or inadvertently) because they lacked the tools to contribute before the user was defined.
These entrepreneurs, product managers, designers, and researchers were people I respected. They weren’t inexperienced. They certainly weren’t uninformed or lazy. It seemed clear to me that the problem wasn’t the people–it was the process.
There had to be a better way, and I dedicated myself to figuring out what it was.
Seeking answers beyond user-centered design
I tried a lot of things.
I found that traditional market research tended to be too large-scale and too expensive to make sense for early innovation work.
I learned about market sizing techniques used in product management, like TAM, SAM (serviceable addressable market), and SOM (serviceable obtainable market). But they all worked better for existing markets than for creating new ones.
I joined a team of business designers and adopted many of their frameworks and tools–especially the importance of razors. But they were more limited in terms of their customer research capabilities.
The closest I got to finding a real solution was when I dug into the discipline of technology commercialization. Commercialization is the process of finding applications for newly developed technologies and bringing them to market. You’ll often find commercialization offices in engineering departments at universities.
What was great about commercialization approaches was that finally, practitioners were talking about technology-first innovation and how to find a market for something that already exists.
What was disappointing was that the actual processes prescribed by practitioners–and even academics (at least the ones I encountered)–largely depended on luck and the strength of the innovator’s professional network. In effect, their advice for innovators was to talk to as many people in as many different industries as possible.
This isn’t bad advice, but it’s also not incredibly useful for most people. If you are an outgoing Harvard startup founder with your professors’ networks at your disposal, then it’s not hard to gain access to business leaders across a variety of industries, one of whom may know the perfect industry application for your product.
But if you’re like most people, your network is probably not so diverse. It’s probably dominated by people similar to you, from your own industry, with similar levels of seniority and authority, in the same cities where you’ve lived. This means that even if you talked to someone new from your network every day for a year, you might never hear about a perfect market in a vastly different industry.
“Even if you talked to someone new from your network every day for a year, you might never hear about a perfect market in a vastly different industry.”
I wanted to create a more repeatable, systematic approach to technology-customer matchmaking–one that could be effectively applied by ordinary entrepreneurs, product managers, designers, and researchers in the real world.
So that’s what I did.
Building a lean customer research framework for transformational innovation
The existing user research toolkit already covered 90% of the design process.3,4 I just needed to build out a framework for what happens in the critical window before you’ve found your user.
Bringing together my experience with experimental psychology, user-centered design, business design, product management, and 10+ years of my own innovation projects at the world’s top companies, I assembled a “best-of” toolkit for early-stage innovation research. And I added steps to help innovation teams consider new markets and industries that might not be familiar to people who work in tech, so that there would be fewer blind spots regardless of the strength of their own professional networks.
Years after my devastating experience with the hand tracking startup, I got the opportunity to advise a different emerging technology team with a similar problem. They had developed an innovative spatial computing technology, but the potential applications were so broad they couldn’t figure out where the market opportunity was. They were being pressured by their stakeholders to focus on customers that were familiar to them, rather than right for the technology.
Applying the framework, we systematically evaluated diverse industries and use cases. And what we learned was that if they had followed their stakeholders’ advice, virtually none of the target customers would have been able to allocate significant budget toward purchasing the team’s product.
But by using the updated toolkit, we were able to identify a new market where customers were eager to invest in the technology at rates more than 20 times greater than the familiar market. With that kind of evidence, it was easy for the team to make the case for pivoting to the more profitable market.
Creating profit and clarity
Since then, I’ve helped numerous product leaders and their teams solve challenging innovation problems by introducing this new research framework. Every team is different, and there’s no one-size-fits-all approach in terms of the tools themselves. But by incorporating customer research into all stages of the innovation process, I’ve been able to consistently save enterprise teams up to 8 figures of development costs, and to give smaller teams a reliable path forward in the face of uncertainty.
👋 I’m Llewyn Paine, Ph.D. I love helping product leaders solve hard problems. If you’re having questions about customers in your innovation process, let’s chat.
--
This is Google’s articulation of the principle, which I think makes the causal relationship most clear, but there are plenty of alternative expressions of this same idea.
The terms and figure used here are adapted from Nagji and Tuff’s Innovation Ambition Matrix from their 2012 article “Managing your innovation portfolio” in Harvard Business Review.
The actual value is not important, so much as to acknowledge that most businesses don’t spend a majority of their resources on transformational innovation. 90% is a nod to the recommended 70-20-10 balance discussed in Nagji and Tuff’s HBR article (cited previously), but this does not necessarily reflect real-world practice, nor is it necessarily the most conceptually appropriate estimate.
I find it fascinating that the existing user research toolkit covers 90% (if not more!) of the design process, but the missing 10% can render the entire effort wasted if you omit it or approach it haphazardly.