Build, borrow, or simulate: 5 thought starters to help your team test innovative experiences with less effort
Low-cost, “lean” prototypes are the most efficient way to test user satisfaction with innovative experiences. But many product teams either overengineer their prototypes or omit testing with users entirely until it’s too late. In this post, I share 5 thought starters for teams looking to think outside the box about simulating experiences for user testing.
When you’re building something new, you’ll have times when you need to judge whether the experience is subjectively “good enough” to satisfy your users.
Design thinking tells us that user testing wireframes or design prototypes can be an effective way to answer this question for layouts and basic interactions.
But what do you do when the experience in question goes beyond what standard design prototypes can simulate? Particularly when that experience is dependent upon emerging technologies or technical experimentation?
Here are some real-world examples of experiences that are hard to simulate using standard tools:
Is a particular field of view good enough for augmented reality headsets?
Is an AI agent good enough at remembering what users have already told it and keeping that context throughout a conversation?
Is a social robot’s facial expression authentic enough, without being creepy?
Is it noticeable enough when speech-to-text makes an error, so that the user can correct it?
Are gestures for spatial computing interfaces intuitive enough? Memorable enough? Low enough effort?
And a non-emerging technology example that inspired this post: Is it noticeable enough if you autocorrect users’ spelling as they type a query in a search bar?
In these cases, building a complete enough experience that it can be tested with naïve users would be time-consuming and costly. Building a version that only your engineering peers could understand might be faster, but this would mean missing out on critical input from actual users.
Either option is non-ideal and introduces unnecessary risk into the design process.
But it doesn’t have to be this way. Many experiences (even for emerging technologies) can be user-tested effectively without building a fully working prototype. However, doing so requires getting out of your comfort zone, collaborating with peers, and considering different tools.
Why innovation teams default to costly technical prototypes
Most innovation teams I’ve worked with turn straight to technology for testing user experience questions without considering other options. But starting with hardware and/or code means they have to postpone testing with representative users until something is built, which may be days or weeks into development.
If it’s so costly, why is it so common?
I believe there are three key reasons:
Familiar tools feel safer: Hardware and code are the tools technologists are most accustomed to, which makes them the automatic and safe choice.
No standard process: Most teams don’t have a standard process for thinking about how to simulate experiences through means other than hardware, code, or design prototypes. This means that any lean prototyping relies on the creativity of the person doing the work. As a result, it provides inconsistent results, making it harder to get buy-in.
No one wants to look silly: Lean prototyping–especially for immersive technologies like spatial computing–can require getting out of your comfort zone. Real-world examples involve constructing physical prototypes out of tape and cardboard (which, by the way, is standard practice in many non-tech industries), or “play-acting” a voice interface. None of this is currently part of mainstream tech culture, which means it can feel a little “fringe,” and therefore uncomfortable.
In other words, people avoid lean prototyping because it feels unfamiliar, ambiguous, and unorthodox.
If any of this sounds familiar, you could probably benefit from a framework. A testing-oriented prototyping framework will help you systematically explore your lean prototyping options, and to do it as a team in a way that is structured, safe, and productive.
The “Minimum Testable Experience” prototyping framework
In my own work, I’ve started using a “Minimum Testable Experience” framework. I designed this framework to help teams reap the benefits of lean prototyping, while removing some of the ambiguity. (Note: While you can use the framework solo, it is designed around group ideation concepts and will be most effective with a group.)
The framework has five steps, which are outlined in the chart below.
In this post I’m going to focus on Step 3, because this is where many teams get tripped up.
Steps 1 and 2 are all about defining your question and refining precisely what it is about the experience you want to test. I refer to this as your “Minimum Testable Experience” (MTE). The goal of Step 3, then, is to generate as many ways as possible to simulate the MTE, without being limited by familiar tools.
This tends to be the hardest step for most innovation teams because of the three reasons I previously shared. There can be a lot of resistance to going beyond the safety of fiddling with hardware and code.
Five thought starters for leaner prototypes
What I’ve found makes this process easier is to approach the entire exercise with a series of structured thought starters. It works best when you include the entire team, and everyone is clear up-front on how lean prototyping works.
After you’ve explained what Minimum Testable Experience you’re trying to create, you lead the team through a series of questions, one by one:
The exercise works best if you follow standard brainstorming best practices:
Have everyone document their ideas on sticky notes.
Timebox exercises, and have a parking lot for unrelated topics.
Give people time to work alone before sharing ideas with the group.
Prioritize quantity over quality–avoid criticizing other people’s ideas at this stage.
Give everyone an opportunity to build upon each other’s ideas.
It is also essential to work through the questions one at a time in the order they appear here. (You are, however, encouraged to go back and add new ideas to previous sections at any time.)
The ultimate goal of the exercise is to generate as many ideas as possible. This becomes the input for Step 4, which involves plotting all of your ideas on a value/effort matrix to determine which one will give you the biggest bang for your user testing buck.
As you brainstorm, it’s important to realize that not all of the ideas you come up with will be winners. Some may not be feasible; others may be too much work. Sometimes, certain prompts won’t have good answers at all. And in some cases (but fewer than you might expect), there really isn’t a good way to build the Minimum Testable Experience without fully developing it. But by systematically considering all the alternatives, you’re ensuring that that the time you spend developing a coded prototype is really worth it, and not needless effort.
Benefits of using a framework
There are many benefits to using a framework in your low-cost prototyping efforts.
We’ve already discussed three of them:
Having a framework reminds you to consider tools that are less familiar, but that may be faster and more cost-effective ways of learning.
Having a framework makes it easier to tackle big, ambiguous problems in a consistent, structured way.
Having a framework that you use as a team creates a safer culture where members feel comfortable experimenting together, which means people are less likely to self-censor. This results in a wider variety of ideas.
But using a framework for lean prototyping has two other major benefits as well.
First, using a framework grants your prototyping efforts more legitimacy with your stakeholders. A framework means no one is making it all up. Your team can still exercise their creativity, but channel it in directions that are proven to be effective. This builds trust.
Second, a framework means you can approach prototyping as a team, not as a solo activity. Progressing through the steps as a group means that everyone is bought in from the beginning. As a result, there’s less need to pitch or sell partners on it as you go. This means the team’s efforts can be fully focused on building and testing your prototypes.
Using a lean prototyping framework makes your innovation process more productive
Even for experienced innovation teams, thinking outside the box with lean prototypes can be time-consuming and intimidating. But it doesn’t have to be. If you don’t already have a proven framework for your team’s prototyping efforts, give this one a try.
👋 I’m Dr. Llewyn Paine, and I’ve been training people to experiment more effectively since 2005. If you’d like additional support for your prototyping efforts, I offer workshops based on the process outlined here. If you’re interested in learning more, you can set up a quick call with me.
--
Footnote:
I’m using the generic label of “lean prototyping” to avoid the baggage associated with two other terms: low-/no-code prototyping, and Minimum Viable Product (MVP). For the purposes of this post, low/no-code prototyping over-indexes on the use of GUI/drag-and-drop app creation tools, when our goal is to encourage teams to think beyond existing tools. The term “MVP” is limiting because it suggests that you need to be testing an entire product, rather than a single, atomic component of the experience.