Augmented reality, wearable tech, and Muppet*Vision 3D: Why spatial computing never seems to arrive, and how to invent the future anyway
Image: “Muppet Theater” by Brett Kiger, licensed under CC BY-NC-ND 2.0
Muppet*Vision 3D, a silly, immersive theater attraction at Disney World–and, quite possibly, the reason I got into spatial computing–is closing after 33 years. It has me thinking about the role of public-facing demos in our work.
Young Llewyn was gobsmacked by Muppet*Vision. I’d seen anaglyph 3D movies before (think red and blue glasses), but this was clear and real in a way that blew my mind. And that was before Disney’s other sensory tricks: smoke, bubbles, water droplets, motorized seats–all coordinated to onscreen events. I wasn’t a Muppets fan, but the transportive quality of the experience meant that THIS was the magical Disney moment I told everyone about when I got home.
Twenty years later I went on to work on 3D experiences at the Disney Media & Advertising Lab, and from there, augmented reality and spatial computing elsewhere. From my current vantage point, I’ve recognized that spatial computing principles underlie many of the tech industry’s major UX breakthroughs over the years. And I believe immersive entertainment and public-facing demos–such as theme park attractions–play an important role in this work by enabling a collective vision for a world with fewer screens.
Parallel experiences with spatial computing
Immersive media, 3D tech, and augmented reality all fall under the umbrella term of “spatial computing”–or, to use older, differently-flavored language, “ambient computing.” Definitions vary, but these terms essentially reference experiences where digital information is integrated with our physical environment. This means it also covers tech like internet of things, “smart” devices, as well as computer vision and physical AI.
If you’ve worked in or near spatial computing for more than a few years, you’ll recognize that the perennial question for the field is: when will spatial computing finally “arrive”? (In fact, as I’m writing this, I’m skipping an Augmented World Expo debate on whether 2024 might be the breakthrough year.)
The tech industry has been speculating about when spatial computing will “arrive” since at least the 1960s.
This question speaks to a unified, sci-fi vision of spatial computing, when in reality, our collective experience of these technologies is more fragmented. Business scholar Rita McGrath says that strategic inflection points never emerge everywhere all at once, and nowhere is that more evident than with spatial computing.
For the general public, the mainstream experience of spatial computing exists in two parallel states: the state of successfully commercialized tech that has become invisible, and the “aspirational” state of hard-to-access visionary demo work and destination experiences. One we take for granted; the other is mostly out of reach.
There’s a fundamental tension between the two that keeps us from recognizing they’re part of the same phenomenon.
Invisible spatial computing today
Do you remember when the internet of things was the next big thing?
In the early 2010s, the internet of things (IoT) was peak hype cycle. Smart and wearable tech felt magical (in fact, “enchanted” was a common descriptor for IoT devices around this time).
But once successfully commercialized and adopted, spatial computing devices become quotidian. Today, Alexa smart speakers, Ring doorbells, and Airpods have created their own tech categories that are now evaluated on their own terms: they’ve ceased to represent “spatial computing” to the general public any more than a smartphone in 2010, a digital camera in 2000, or a Walkman in 1990.
This was once cutting-edge portability.
Commercial success becomes a barrier for product creators to see beyond the technology category, which means it becomes harder to ideate novel form factors, as well as new, emergent means of building upon existing devices’ input/output capabilities.
So over time, formerly “enchanted” devices effectively become invisible, and we feel that spatial computing still hasn’t arrived because we now see an everyday “Apple Watch” where we used to see a magical “tiny portable display and sensors on my wrist.”
In the 2014 announcement of the original Apple Watch (top), presenters spent 45 minutes explaining the interaction, information, and sensing capabilities of a wrist-based computer. By 2024 (bottom), Apple had ditched the focus on interactions and was emphasizing aesthetics far more.
Inaccessible inspiration
The “invisible tech” experience exists in parallel with an opposite, but still mainstream spatial computing experience: that of immersive entertainment “destination” showcases, which are much less readily accessible.
Muppet*Vision was one of these. When I first experienced the attraction in the early 1990s, I had to fly to Florida, have parents who could afford a Disney park pass, and wait in line, all for a 25-minute runtime. The experience astounded me, but it would be literally years before I saw it again. (In fact, I’ve seen the Muppet*Vision 3D show exactly twice–once as a child, and once on a college graduation trip with friends).
This lack of access is the norm not only for really impressive immersive entertainment, but also for groundbreaking spatial computing demos within the tech industry.
Maybe you’ve heard of Dynamicland, a sort of spatial computing coworking space, built with projectors and lots of specialized code. To many people who work in augmented reality, Dynamicland is the goal to aspire to, and its inventor–Bret Victor–is a rockstar.
Screenshot of a Hackster.io video of a trip to Dynamicland
But Dynamicland was simply a demo (albeit, a very impressive one). It existed at a single location in Oakland, CA, accessible only by invitation. Hardly anyone who rapturizes Dynamicland has actually been there (as lamented by Vitorio Miliano in an essay on this subject); instead we’re all referencing the same videos and second-hand stories. What we know about the experience is only what others have chosen to share.
More accessible than Dynamicland are escape rooms, Meow Wolf, and, of course, theme parks (which become increasingly digital and video game-like)–but these are still “destination” experiences. (Note: If you wonder why I’m focusing on entertainment, perhaps it’s because, before being fully commercialized, immersion tends to feel like play. Immersive “seeing spaces” do exist in the workplace–such as CAVEs in engineering or mission control at NASA–but they are specialized for specific tasks and rarely open to the public. And, while impressive, they rarely evoke the same sense of wonder.)
CAVE at the University of Illinois (public domain, photographer: davepape)
There are, of course, reasons why immersive entertainment is somewhat walled off from the general public. For one thing, truly jaw-dropping immersion is technically complex and expensive to produce. But also, the novelty of these experiences is why they wow us and work as inspiration. Novel immersive experiences grab our attention and show us a glimpse of an alternate universe where technology is not mediated by screens.
These glimpses are important because we are endlessly falling into the trap of thinking that today’s technology form factors are already the best ways of solving people’s problems–when in reality they’re simply the best ways we’ve effectively marketed so far.
So theme parks and Meow Wolf work as public-facing demos that we can use to envision and inspire entirely new types of interfaces and interaction. And, like my childhood trip to Disney World, they’re magical in part because they’re rare.
So when will spatial computing arrive?
This brings us back to the question of when spatial computing will arrive. Immersive entertainment whets the public appetite, but it is ever showing us a vision that is a few steps beyond where we are today (which is the whole point).
This makes it hard to recognize the reality that spatial computing has, in fact, already arrived, many times over. It arrived with smart home devices, and the Kinect, and the Roomba; it arrived with the Walkman and the Nintendo Power Pad; it arrived with the invention of the mouse and graphical user interfaces.
They’re all part of a gradual progression toward ever more natural interfaces.
But as new form factors are successfully commercialized and new technology categories emerge, they fade into the technology backdrop of our everyday lives. Worse, they become fixed in form factors that represent the engineering compromises of the era when they were invented (a recent example: the Apple Vision Pro’s ultra-wide display mirroring; who, other than die-hard tech apologists, would honestly believe the “killer app” for spatial computing is a really big desktop for your Mac?)
Is this truly the best we have to look forward to from spatial computing? (Screenshot from Apple Vision Pro press release video)
We need show-stopping, visionary demos to inspire us to think beyond the limits of today’s technology categories and create transformational spatial computing experiences. These explorations help us discover new form factors that embrace more natural ways of integrating information into the physical world.
The truth is that spatial computing is not an offshoot of information technology–in fact, it is the ideal that all of human-centered design is striving for. Because humans themselves are spatial, and to serve them most effectively, technology must be as well.
“Spatial computing is not an offshoot of information technology–it is the ideal that all of human-centered design is striving for. Humans themselves are spatial, and to serve them most effectively, technology must be as well.”
Learning to see opportunities in three dimensions
The future of computing is spatial…but today, the majority of our work as designers and developers in tech still takes place on flat screens.
So why do we continue to ask when spatial computing will arrive?
I think it’s a combination of not wanting to get left behind, and wanting permission to invent something new.
The bad news is if you are not already viewing the world through the lens of spatial computing, you are already behind. Spatial computing has made its debut and integrated itself so firmly into our world that we no longer recognize it.
The good news is it’s not too late–not by a long shot–because it is a continual progression. There is no one technology you need to specialize in to be a spatial computing “expert.” And the tech industry will always need people who can look beyond our existing form factors and invent more natural ways of engaging with information–whether that’s a micro-interaction or a whole new hardware category.
The best thing you can do to discover these opportunities is to learn how to see interfaces not as informational abstractions, but as they exist in real, three-dimensional space.
This means making invisible technology visible, recognizing the ways information is already embedded into the physical world around us, and learning how to build upon it.
And it also means seeking out immersive experiences that help you see possibilities for a world with fewer screens, beyond the limitations of existing form factors.
The importance of this last point cannot be overstated, and you should recognize that the most impactful “vision demos” may not look like you’re used to tech looking.
As a young girl, the vision that inspired me happened to involve Muppets.
So what’s yours?
Llewyn Paine, Ph.D., founder of Llewyn Paine Consulting, LLC, is an innovation strategy consultant in Austin, TX. A veteran of the Disney Media & Advertising Lab and Microsoft’s mixed reality envisioning studio, she helps companies develop commercially successful emerging tech products through participatory envisioning techniques and lean market research.