We let AI interview UX researchers and designers: Here’s what it taught us about designing conversational AI
AI-moderated interviews are moving from experiment to everyday workflow. But for anyone who cares about user experience, the real question isn’t whether an AI can ask good questions. It’s whether the person on the other side wants to answer them. And that comes down to the design more than the model.
People are more comfortable talking to AI than you might expect. In an Adobe Express survey of 1,005 US consumers, 61% said they were comfortable having conversations with AI-powered systems. A COPC study across six countries found people who knew they were talking to AI reported 34% higher satisfaction than those who weren’t told.
But a quick chat with a bot isn’t a research interview.
So we picked the toughest crowd we could find: our own users. We ran a study with UX researchers and designers using Optimal’s AI Interviewer. These are people who spot a leading question from a mile away, wince at awkward pauses and are naturally skeptical of handing the moderator’s seat to AI (and rightfully so!). If anyone was going to call out a bad experience, it was them.
What participants told us
Most participants described the experience as seamless, smooth, natural and even fun, including several who had reservations going in. Three things stood out.
1. It felt like a conversation, not a script
Participants noticed that follow-ups were built on what they had just said. One put it simply:
“I really felt like I was talking to a real person.”
Another appreciated that the AI summarised their points before moving on, instead of asking the same question in different ways. That was a frustration they’d had with other AI interview tools.
2. Less pressure made it easier to open up
Several participants found the AI easier to talk to than a live interviewer. Without a person across the table, there was less pressure to perform.
“I have no problems speaking freely.”
On-screen text alongside the spoken conversation also mattered. One participant with auditory processing difficulties called it out as a genuinely useful accessibility feature.
Not everyone was fully won over. One participant felt a human would better grasp the emotion behind their answers, though they still rated the experience positively.
3. Choice built confidence
Participants could pick how the conversation flowed. One chose “submit response” mode, deciding when each answer was finished. After a good first experience, they said they’d be happy to try “free form” mode next, where the AI responds naturally once they stop speaking.
Want to hear it from participants directly?
Watch the highlight reels in our full study write-up →
What designers can take from this
The takeaway reaches well beyond research tools: whether an AI conversation works depends less on the model and more on the experience designed around it. If you’re designing any conversational AI, these are worth stealing.
- Prove you’re listening. Responses that reference the user’s own words build trust faster than clever phrasing. A short summary before moving on signals “I heard you.”
- Don’t make people repeat themselves. Rephrasing the same question feels robotic. Move the conversation forward instead.
- Lower the stakes. Removing social pressure can make people more candid. Clear instructions and a predictable flow help them settle in.
- Offer more than one channel. Pairing voice with on-screen text supports people who process information differently, and makes the experience easier for everyone.
- Give users control over pacing. Letting people choose when to submit, or when to go free form, turns uncertainty into confidence.
- Be honest about limits. Some people will still prefer a human for emotionally nuanced topics. Design AI as an option, not the only path.
See how Optimal’s AI Interviewer works
We built these lessons into Optimal’s AI Interviewer, an AI-moderated research tool designed to work with you, not instead of you. It helps teams run more interviews without losing the depth that makes qualitative research valuable.
- Add your context. Research goals, audience details and supporting material guide every follow-up.
- Build your discussion guide. Set sections, questions, goals and how deeply the AI should probe.
- Let it run the interview. The AI listens, adapts and explores relevant threads while staying on your goals.
- Get cited insights. An AI-generated summary and key themes, with citations at study, section and question level, plus highlight reels that jump to the moment behind each insight.
Take this two-minute interactive tour to see it for yourself.
The bottom line
AI interviews aren’t about replacing the human side of research. They give teams another way to listen, at a scale that used to need a lot more people and time. And when the experience is designed well, participants are happy to talk.
Optimal’s AI Interviewer is now in Early Access, book a demo to learn more.
Author
