Over the past few years, the tech industry has rushed to plug a new microchip into every device on the assumption that if you can add AI, you should. Smart vision toothbrushes, mood-reading household robots, and a host of physical products are invading consumers’ daily lives, reliant on artificial intelligence to learn, predict, and adjust our behavior.
But as anyone who’s struggled to update the firmware in their fridge knows, AI doesn’t improve every product. Some product designers are asking where the line is between genuine innovation and a solution looking for a problem.
Savannah Kunovsky leads a team of industrial product designers at Ideo, a design consultancy based in San Francisco focused on helping clients unpack when AI becomes a barrier rather than a breakthrough.
Kunovsky spoke with IEEE Spectrum about the pressure to integrate AI in everything, where that integration goes wrong, and what companies can do to determine when an AI integration is—and isn’t—a smart choice.
When AI makes products worse
How much of the current wave of AI integration into physical products is driven by pressure from investors and shareholders to keep up with industry trends?
Savannah Kunovsky: Some of it. Teams come to us feeling pressured to show they’re doing something with AI, sometimes because a CEO has promised the board an “AI story” before the next earnings call, or a competitor has just launched an AI-powered version of their product.
In all cases, the first thing we do is step back and ask who the product is for and what they’re trying to do. Once that’s clear, the role AI should play, whether large, small, or invisible, usually becomes obvious. There is an appetite for this kind of grounding in the industry right now. People want to cut through the hype and see what works in practice.
How do you distinguish between a genuinely helpful AI feature and a gimmick?
Kunovsky: The test for an AI gimmick is the same test we’d apply to any product experience: Does it help someone do what they’re trying to do? Answering that means understanding how people would use the product in the real context they’ll be using it.
Many companies look straight to large language models (LLMs) because that’s top of their mind, but there is a whole AI toolbox to draw from: computer vision, sensor fusion, on-device models, et cetera. Picking the right capability for the task matters as much as deciding whether to use AI in the first place.
As a new interaction paradigm, AI is asking people to change their behavior, so it needs to be simple. When IDEO’s predecessor firm worked on Apple’s first mouse in the early 1980s, the team gave it only one button because people were learning a completely unfamiliar system and the interaction mechanism needed to be effortless.
The wearable Humane AI pin used projection instead of a screen-based display, making a dramatic shift from the smartphone interface users already knew.Angel Garcia/Bloomberg/Getty Images
In which products has AI made the user experience worse?
Kunovsky: The clearest examples are products where AI replaced something that already worked. I have a lot of respect for Humane’s AI Pin as a provocation, but the technology [a screenless, wearable smartphone alternative that uses an AI assistant instead of apps] wasn’t ready. The Pin asked people to give up an interface they knew well for one that was slower and less reliable, which is a hard tradeoff to ask of anyone.
You see versions of this problem everywhere—a voice assistant that mishears a simple command and turns off the wrong light, a customer service bot that keeps you in a loop when you just want a person. In cars, climate and volume controls moved onto touchscreens and voice menus, so something you used to do by feel now takes your eyes off the road. Euro NCAP, Europe’s car safety rating program, now rewards physical controls for that reason.
People love certain products precisely mainly because they are analog. Part of the value of a film camera, a good kitchen knife, or a notebook is the craft and the feeling of being fully in control.
But it doesn’t have to be either/or. In a smart home you can ask a voice assistant to turn off the lights, use an app, or hit the switch on the wall. Having more than one way to reach the same goal is a good thing, as long as it doesn’t add complexity for its own sake.
Can you share an example where you convinced a client to ditch an AI implementation?
Kunovsky: A client wanted a cooktop with no knobs, you tell an AI what you are cooking and it manages the heat. When we tested it, the pan started smoking and the user reached for a knob that wasn’t there, they said “turn it down” twice and pulled the pan off the burner, all before the AI responded. Needless to say we ended up keeping the knobs.
Anything you need to do instantly, by feel, or under stress is usually best kept physical: a volume knob, a light switch, a stove burner, an emergency stop. In those moments a mechanical control is faster and more trustworthy than a system that has to interpret what you mean.
Toward better AI product design
What does “good” AI integration look like?
Kunovsky: Companies often start by considering the technical feasibility of AI because the technology is so exciting. But understanding what people need is the key, followed by technical feasibility and what makes sense for the business.
From there, the question is: What role should AI play? Sometimes it is the core of the experience, like a voice interface that allows you to talk to a product naturally. Sometimes it works in the background, learning your preferences so you stop repeating yourself. Either way, the measure is whether it helps someone do what they set out to.
How do you prevent the inclusion of AI from making products nonintuitive or burdensome?
Kunovsky: LLMs work in text, so the instinct is to give everything a chat window. But if instead you start by considering the person and the task, the interface can look very different.
Physical products can communicate in many ways beyond words—in light, sound, vibration, movement, the feel of a knob or a button. Leveraging the intelligence in AI can make all of those richer. The products I’m most excited about extend what we can perceive, understand, or do, whilst keeping us at the center of the experience.
Some of the clearest examples are in the field of accessibility. Hearing aids can use AI to pick out the voice you’re trying to hear in a noisy restaurant. Apps like Be My Eyes enable blind people or those with low vision to point their phone at the world and hear a description of what’s in front of them. Live translation in earbuds allows two people who don’t speak the same language to hold a conversation.
Proactive AI systems need to anticipate users’ needs without becoming intrusive, the equivalent of a good server at a restaurant who notices when your glass of water is empty, or reads if you want to chat or be left alone.
When people try something new, often they don’t know what to ask for, so a product that shows up with something useful can teach them what’s possible. We’re starting to see a wave of personal agents, like Instinct, Dots, and Muse, which work on things in the background and come back to users with new findings.
Relevance and timing matter most. How loudly a product speaks up should match the stakes and it should learn from how people respond. If someone keeps dismissing something, that’s a signal to adjust.
Generative AI is non-deterministic and may give a different response every time it’s used. How does that challenge conventional product testing and design?
Kunovsky: It changes what product definition looks like. With traditional software, you write a spec and test that the product does exactly what it says. With generative AI you can’t specify every output, so you define the space the product should operate in.
AI evaluations and constitutions [written principles that guide model training] have become a new form of product definition for LLMs. Evaluations describe what good performance looks like across thousands of situations; a constitution sets out the values and boundaries the system should hold to. Together with the model’s software, configuration, and runtime infrastructure, they shape the container the product lives in. Getting that right is now core design work, not just an engineering step at the end.
Consumer trust comes under pressure when product responses are non-deterministic. People learn to trust traditional products because they do the same thing every time. Generative AI requires much more design intention, so teams must decide up front where the product should be consistent and where it can explore. The product should make it clear to users what mode they’re in, and show what it is doing when it is unsure.
How important is it to incorporate human-override and manual pathways into AI systems?
Kunovsky: It’s essential. No AI system will be right every time. If people can step in easily, a mistake is a small moment. If they can’t, it becomes a reason to stop trusting the product.
Override doesn’t just mean an emergency button for a home robot, or in a car. It can be as simple as undoing an action, editing a suggestion, or turning a feature off.
In our work developing Ethiqly, an AI-powered writing tool that helps students organize their ideas instead of writing the essay for them, teachers approve every piece of feedback before a student sees it. That kind of control helps people feel comfortable handing more to the system over time.
Manual pathways should be obvious, fast, and available in the moment you need them, not buried in a settings menu. When people know they can always take back control, they’re much more willing to let AI help.
Read the original article: