Stratis Valachis, senior user experience designer at Aviva’s Digital Innovation Garage, took some time out of his busy schedule to answer some questions about designing for conversational user interfaces (CUI). Learn more about his processes for research and design for CUI, what he thinks the future will look like, and some of the biggest challenges he’s faced while designing for CUI.Stratis will be speaking at MUXL2017, the third annual conference around Mobile User Experience in London on the 10th of November at City, University of London. Using case studies through talks and workshops, the conference will cover Core UX principles as well as emerging topics such as AI (Chatbots), VR (AR) & IOT.

What does the research and design process for conversational interfaces look like?

Like any design project, you should always start by identifying user needs and real problems. Research how users solve that problem currently and then evaluate for which use cases you can remove friction and enhance the experience by utilizing a conversational interface.Don't try to chat-ify or voice-ify your product just because it's a cool trend. In many ways conversational interfaces (CUIs), both voice and visual, have more usability constraints than traditional GUI. For example, it’s hard to interrupt the conversation to recover from errors, you can't easily skim through information, progress is linear and you very often need to rely on recall.Users make conscious compromises about which type of interface they want to use.This means that a solution utilizing a CUI needs to offer an obvious benefit for your chosen use case, otherwise users won't use your product. That's why special emphasis should be placed on early research about the context in which users will use your product and on why a CUI could provide a better experience. When you begin the design phase, a good practice would be to craft a personality for your interface. Studies have shown that because humans are empathetic, they will assign human character attributes to your CUI anyway, so it's better to make sure this is defined through design. This works really well for platforms like Google Home and Facebook Messenger, which make it clear to the user that each product built on them is a different entity from the default assistant.Some channels like Alexa, though, don't make that distinction clear. In these cases, you need to make sure that the character of your CUI doesn’t significantly deviate from the personality of the default assistant, otherwise you'll mess with their mental model and create confusion. For example, when you're ordering an Uber with Alexa, it’s Alexa that speaks back to you: "Alexa, ask Uber for a ride." "Sure, there's an Uber less than a minute away, would you like me to order it?". While on Google Home, the Google Assistant makes it clear that it passes you over to Uber "Hi, I'm Uber, how can I help?".After you define the personality, start drafting out the core experience of your product.If you're working on a visual CUI, type the conversation down like a screenplay. If you're working with voice, act the dialogue out with your colleagues and use voice simulators to see how it feels in the channel you're designing for. This will make it easier to decide the direction you'd like to follow and will also help you initiate conversations with stakeholders.At this stage, you will be ready to start designing your user flows to define the functionality at a granular level. Again, understanding context is crucial. Make sure you think of the different scenarios in which users will interact with your product and the ways they're likely to phrase their input. User testing is key for this.

What are some of the biggest challenges you've faced designing for CUI?

Setting the right expectations for users. That applies to both visual and voice interfaces. There's a gap between the mental model users have of what most AI products with conversational interfaces can do, and what they are actually capable of doing. That was a common pattern I've seen in user testing sessions even with users who had previous experience in the conversational channel that was being tested. As a designer your challenge is to make the affordances and constraints clear in a way that feels like a natural part of the conversation and mitigates disappointment from unrealistic expectations. Another challenge is trying to cater for all the different ways people will phrase the requests. The key here is to invest time and resources in user research and NLP (natural language processing) services. If you feel that this is out of scope for your project, you may consider limiting the options for your users as trying to guide them to say things in a certain way will not work. Good examples of this are Facebook Messenger bots which now allow developers to remove the input field entirely from the experience in order to prevent users from making requests that can't be supported.

How do you think CUI is going to change the way designers and researchers do their work?

It might require designers and researchers to slightly alter some techniques they're using (for example thinking aloud during user testing doesn't work with voice interfaces) but the fundamentals will stay the same. You still need to focus on understanding the problem, explore different solutions through divergent thinking, converge, develop and continuously iterate based on user feedback. The exciting thing is that these new technologies significantly expand our toolbox and offer new interesting ways to solve problems for our users.

What improvements to this kind of technology do you wish to see? How would you like this technology to progress in the future?

I would like to see a more widespread integration of voice interfaces with visuals and GUI interaction patterns. A good example of the benefits of this approach is Amazon's Fire TV. Users can converse with the system via voice when it's more efficient than the alternative interaction options (for example, searching for a movie) but use their remote control to interact with visual UI elements for tasks that would be tedious to perform through voice. For example, selecting a movie cover to reveal descriptive text and then skimming through it helps you gauge whether the plot is interesting faster than if you had to consume this information through a conversation. This hybrid approach utilizes the best of each world to create a stronger experience. I think we will see this type of interface a lot more in the future. Think of Iron Man and J.A.R.V.I.S.

Any advice for young designers and researchers hoping to get into this part of the industry?

Invest time in learning best practices for crafting good dialogue. It's a crucial skill for designers in this field. Google and Amazon's design guidelines are a good starting point. This doesn't mean you should omit training and improving your knowledge in usability for traditional interfaces. Most of the principles are time-proof and channel agnostic and will help you greatly with conversational interfaces.Another thing you should make sure you do is stay up to date with the latest trends. The technology evolves very fast so you need to stay ahead of curve. Attend meetups, work on personal projects and participate in hackathons to practice and learn from the experts.As long as you're really passionate about the field, there will be plenty of opportunities for you to get involved and contribute. We're still in the early stages of mainstream adoption of the technology, so we have the chance to make significant impact on the evolution of the field and shape best practices for years to come, which is really exciting!

Wireframes. Mockups. HTML. Fonts. Elements. Users. If you’re familiar with user interface design, these terms will be your bread and butter.An integral part of any website or application, user interface design is also arguably one of the most important. This is because your design is what your users see and interact with. If your site or app functions poorly and looks terrible, that’s what your users are going to remember.

‍

But isn’t UX design and UI design the same thing? Or is there just an extremely blurred line between the two? What’s involved with UI design and, more importantly, what makes good design?

‍

What is UI design exactly?

‍

If you’re wondering how to test UI on your website, it’s a good idea to first learn some of the differences between UX and UI design. Although UI design and UX design look similar when written down, they’re actually two totally separate things. However, they should most definitely complement each other.

‍

UX design, according to Nielsen Norman Group, “encompasses all aspects of the end-user's interaction with the company, its services, and its products.” Meanwhile, UI design focuses more on a user’s interaction, the overall design, look and feel of a system. The two still sound similar, right?For those of you still trying to wrap your ahead around the difference, Nielsen Norman Group has a great analogy up on its site that helps to explain it:

‍

"As an example, consider a website with movie reviews. Even if the UI for finding a film is perfect, the UX will be poor for a user who wants information about a small independent release if the underlying database only contains movies from the major studios.”

‍

This just goes to show the complementary relationship between the two and why it’s so important.User interface was popularized in the early 1970s, partly thanks to Fuji Xerox’s ‘Xerox Alto Workstation’ — an early personal computer dubbed “the origin of the PC”. This machine used various icons, multi windows, a mouse, and e-mail, which meant that some sort of design and design principles were needed to create consistency for the future. It was here that human-centred UI was born. UI design also covers graphical user interface design (GUI design). A GUI is the software or interface that works as the medium between a user and the computer.

‍

It uses a number of graphical elements, such as screen cursors, menus, and icons so that users can easily navigate a system. This is also something that has stemmed from Fuji Xerox back in the late 1970s and early 1980s.Since then, UI has developed quickly and so has its design principles. When the Xerox Alto Workstation was first born, Fuji Xerox came up with eight of its own design principles. These were:

• Metaphorically digitize the desk environment
• Operating on display instead of entering on keyboard
• What you see is what you get
• Universal but fewer commands
• Same operation for the same job at different places
• Operating computers as easily as possible
• No need to transfer to different jobs
• System customized as desired by users

‍

Over time, these principles have evolved and now you’ll likely find many more added to this list. Here are just a few of the most important ones identified in “Characteristics of graphical and web user interfaces” by Wilbert Galitz.

‍

UI design principles:

‍

Principle #1: Clarity

‍

Usability.gov says that the “best interfaces are almost invisible to the user”.Everything in the system, from visual elements, functions, and text, needs to be clear and simple. This includes layout as well as the words used — stay away from jargon and complex terms or analogies that users won’t understand.Aesthetic appeal also fits into this principle. Ensure colors and graphics are used in a simple manner, and elements are grouped in a way that makes sense.

‍

Principle #2: Consistency

‍

The system should have the same or similar functions, uses and look throughout it for consistency. For example, the same color scheme should be used throughout an app, or the terminology on a website should be consistent throughout. Users should also have an idea of what to expect when they use your system. As an example, picture a retail shopping app. You’d expect that any other retail shopping app out there will have similar basic functions: a place to log in or create an account, account settings, a way to navigate and browse stock, a way to purchase stock at the press of a button. However, this doesn’t mean copying another app or website exactly; there should just be consistency so users know what to expect when they encounter your system.Apple even states an “app should respect its users and avoid forcing them to learn new ways to do things for no other reason than to be different”.

‍

Principle #3: Flexibility and customizability

‍

Is there more than one way people can access your system and its functions? Can people perform tasks in a number of different ways, too?Providing your users with a flexible system means people are more in control of what they’re doing. Galitz mentions this can also be done through allowing system customization.Don’t forget use on other kinds of devices, too. In a time when Google is using mobile-friendliness as a ranking signal, and research from Ericsson shows smartphones accounted for 75% of all mobile phone sales in Q4 2015, you know that being flexible is important.

‍

Examples of good UI design

‍

For a list of some of the best user interface examples, check out last year’s Webby Awards category for Best Interface Design. The 2016 category winner was the Reuters TV Web App, while the People’s Choice winner was AssessYourRisk.org.As an aside, this is the second year that the Webby Awards introduced this category — just goes to show how important it is to have good UI design!While you don’t want your site or application to look exactly the same as these winners, you still want yours to function well and be aesthetically pleasing.

‍

To help you get there, there are a number of UI design tools and UI software available.Here’s a list of some of the many out there:

• UXPin - An online UI design tool that allows you to create wireframes, mockups, and prototypes all on one platform.
• InVision - A prototyping and collaboration tool. More in-depth than Balsamiq, and it allows you to go from mockup to high-fidelity in minutes.
• Balsamiq - A simple mockups tool for wireframing, which allows users to test out ideas in the early stage of interface design.
• Atomic - An interface design tool that allows you to design in your browser and collaborate with others on your projects.

‍

Have you got any favorite UI design examples, or tips for beautiful design? We’d love to see them — comment below and let us know!

‍

Further reading

• “UI, UX: Who does what? A designer’s guide to the tech industry” - An article from Fastcodesign that provides a breakdown of some of the design roles in the tech industry.
• “7 rules for creating gorgeous UI (Part 1)” - A great guide from Erik D. Kennedy for creating good UI design, with some fantastic tips to help you get started.
• “10 usability heuristics for user interface design” - An article from Nielsen Norman Group’s Principal Jakob Nielsen explaining some of the key principles to keep in mind when designing a user interface.

About six months ago, I came across aninteresting question on Stack Exchange headlined 'Should you concede to user demands that are clearly inferior?' It stuck in my mind because the question in itself is complex, and contains a few complicated assumptions.

‍

In the world of user experience research and design, the users needs and wants are paramount. Dollars and hours are spent poring through data and interviewing and collating information into a cohesive explanation of what works and what doesn't for users. Designs are based on how users intuitively interact with products and websites. Organisations respond to suggestions that come through on support and on Twitter, and if a significant numbers of users want a particular change, chances are those organisations will act. But the question itself throws this most sacred of stances up in the air, because it contains the phrase 'user demands that are clearly inferior'. Now, that is a loaded statement.

‍

How the good reconcile the existence of the bad

‍

I imagine it's sometimes hard for designers to get rid of the feeling that they know best. As a writer, I know what I like and don't like. I 'know' good writing from bad, and I have strong opinions about books and articles that aren't worth the pages or bandwidth it takes to publish them. But this stance often puts me in conflict with the huge amount of empirical evidence that certain writing I disdain is actually 'good': and that evidence is readers. For Fifty Shades of Lame, it's millions of them. Aggghh!

‍

In the same way, I've never met a designer who didn't have strong opinions about what they adore and deplore in their own art forms. And I wonder how tough it sometimes is to implement changes that to a designers mind make no sense. Do any of you UX designers out there ever secretly think, when you discover what users are asking for, 'these people have no taste, they don't know what they want, how ridiculous!'? Is there a secret current of despair and frustration at user ignorance running deep and unspoken through the river of design?

‍

The main views from the Stack Exchange discussion

‍

‍

On Stack Exchange, Matt described how he and his team implemented a single tree view (75 items) with a scroll wheel, and because it was an internalchange,they were able to get quick feedback from existing users. The feedback wasn't positive, and many people wanted the change to be reversed. He explains: ‘To my mind, the way we redeveloped it is unambiguously better. But the user base was equally emphatic in rejecting it. So today, to the complaints of my fellow team members, I removed our new implementation and set it to work in the manner the users were used to.'

‍

He then goes on to ask 'What was the right course of action here? Is there a point at which the user's fear of change becomes an important UX consideration in its own right?' The responses are varied and fascinating, and can be roughly broken into three camps:

‍

1. If your users don't want something, you'd be stupid to try and implement it.
2. Users are often change averse, so if you really think your change will be better, then you need to ease them into it.
3. If you're convinced the change is positive, you still need to test it on your users, and be open to admitting you were wrong.

‍

So where do we stand?

‍

One of the problems with the term 'User Experience' is the word 'user'. It's a depersonalised and generic way of describing who it is you're serving. Because there is a person at the heart of the enterprise who is trying to achieve something. They may not be trying to achieve what you expect them to. They certainly may not be trying to achieve what you want them to.

‍

Context is everything.

‍

Who is the person who is asking for a change, or asking for something to stay the same?We would argue that people aren't 'change-averse', but 'confusion/discomfort/inefficiency-averse' people want easier ways of doing things. So if by changing a feature you mess up a person's workflow, then potentially you didn't do your research.

‍

If you look closely at the behavior of users — how people actually interact with a particular aspect of your design, rather than just hearing their opinions — then you'll be able to base your design on empirical evidence. So, we (roughly) come down on the side of the people who use the product. If they want to get something done, and they want to do that in a particular way, then they have right of way.

‍

It's your job not to serve your tastes, but to give people the experience you promise them. And to the author of Fifty Shades of Grey, I say, 'Good on you EL James. You gave them what they wanted.'

‍

What do you think?

If you’ve ever presented design findings or recommendations to clients or colleagues, then perhaps you’ve heard them say:

‍

• “We don’t have the budget or resources for those improvements.”
• “The new executive project has higher priority.”
• “Let’s postpone that to Phase 2.”

‍

As an information architect, I‘ve presented recommendations many times. And I’ve crashed and burned more than once by doing a poor job of selling some promising ideas. Here’s some things I’ve learned from getting it wrong.

‍

Buyers prefer sellers they like and trust

‍

You need to establish trust with peers, developers, executives and so on before you present your findings and recommendations . It sounds obvious, yet presentations often fail due to unfamiliarity, sloppiness or designer arrogance. A year ago I ran an IA test on a large company website. The project schedule was typically “aggressive” and the client’s VPs were endlessly busy. So I launched the test without their feedback. Saved time, right?Wrong. The client ignored all my IA recommendations, and their VPs ultimately rewrote my site map from scratch. I could have argued that they didn’t understand user-centered design. The truth is that I failed to establish credibility. I needed them to buy into the testing process, suggest test questions beforehand, or take the test as a control group. Anything to engage them would have helped – turning stakeholders into collaborators is a great way to establish trust.

‍

Techniques for presenting UX recommendations

‍

Many presentation tactics can be borrowed from salespeople, but a single blog post can’t do justice to the entire sales profession. So I’d just like to offer a few ideas for thought. No Jedi mind tricks though. Sincerity matters.

‍

Emphasize product benefits, not product features

‍

Beer commercials on TV don’t sell beer. They sell backyard parties and voluptuous strangers. Likewise, UX recommendations should emphasize product benefits rather than feature sets. This may be common marketing strategy. However, the benefits should resonate with stakeholders and not just test participants. Stakeholders often don’t care about Joe End User. They care about ROI, a more flexible platform, a faster way to publish content – whatever metrics determine their job performance.Several years ago, I researched call center data at a large corporation. To analyze the data, I eventually built a Web dashboard. The dashboard illustrated different types of customer calls by product. When I showed it to my co-workers, I presented the features and even the benefits of tracking usability issues this way.However, I didn’t research the specific benefits to my fellow designers. Consequently it was much, much harder to sell the idea. I should have investigated how a dashboard would fit into their daily routines. I had neglected the question that they silently asked: “What’s in it for me?”

‍

Have a go at contrast selling

‍

When selling your recommendations, consider submitting your dream plan first. If your stakeholders balk, introduce the practical solution next. The contrast in price will make the modest recommendation more palatable.While working on e-commerce UI, I once ran a usability test on a checkout flow. The test clearly suggested improvements to the payment page. To try slipping it into an upcoming sprint, I asked my boss if we could make a few crucial fixes. They wouldn’t take much time. He said...no. In essence, my boss was comparing extra work to doing nothing. My mistake was compromising the proposal before even presenting it. I should have requested an entire package first: a full redesign of the shopping cart experience on all web properties. Then the comparison would have been a huge effort vs. a small effort.Retailers take this approach every day. Car dealerships anchor buyers to lofty sticker prices, then offer cash back. Retailers like Amazon display strikethrough prices for similar effect. This works whenever buyers prefer justifying a purchase based on savings, not price.

‍

Use the alternative choice close

‍

Alternative Choice is a closing technique in which a buyer selects from two options. Cleverly, each answer implies a sale. Here are examples adapted for UX recommendations:

• “Which website could we implement these changes on first, X or Y?”
• “Which developer has more time available in the next sprint, Tom or Harry?”

‍

This is better than simply asking, “Can we start on Website X?” or “Do we have any developers available?” Avoid any proposition that can be rejected with a direct “No.”

‍

Convince with the embarrassment close

‍

Buying decisions are emotional. When presenting recommendations to stakeholders, try appealing to their pride (remember, you’re not actually trying to embarrass someone). Again, sincerity is important. Some UX examples include:

• “To be an industry leader, we need a best-of-breed design like Acme Co.”
• “I know that you want your co mpany to be the best. That’s why we’re recommending a full set of    improvements instead of a quick fix.”

‍

Techniques for answering objections once you’ve presented

‍

Once you’ve done your best to present your design recommendations, you may still encounter resistance (surprise!). To make it simple, I’ve classified objections using the three points in the triangle model of project management: Time, Price and Quality. Any project can only have two. And when presenting design research, you’re advocating Quality, i.e. design usability or enhancements. Pushback on Quality generally means that people disagree with your designs (a topic for another day).

Therefore, objections will likely be based on Time or Price instead.In a perfect world, all design recommendations yield ROI backed by quantitative data. But many don’t. When selling the intangibles of “user experience” or “usability” improvements, here are some responses to consider when you hear “We don’t have time” or “We can’t afford it”.

‍

“We don’t have time” means your project team values Time over Quality

‍

If possible, ask people to consider future repercussions. If your proposal isn’t implemented now, it may require even more time and money later. Product lines and features expand, and new websites and mobile apps get built. What will your design improvements cost across the board in 6 months? Opportunity costs also matter. If your design recommendations are postponed, then perhaps you’ll miss the holiday shopping season, or the launch of your latest software release. What is the cost of not approving your recommendations?

‍

“We can’t afford it” means your project team values Price over Quality

‍

Many project sponsors nix user testing to reduce the design price tag. But there’s always a long-term cost. A buggy product generates customer complaints. The flawed design must then be tested, redesigned, and recoded. So, which is cheaper: paying for a single usability test now, or the aggregate cost of user dissatisfaction and future rework? Explain the difference between price and cost to your team.

‍

Parting Thoughts

‍

I realize that this only scratches the surface of sales, negotiation, persuasion and influence. Entire books have been written on topics like body language alone. Uncommon books in a UX library might be “Influence: The Psychology of Persuasion” by Robert Cialdini and “Secrets of Closing the Sale” by Zig Ziglar. Feel free to share your own ideas or references as well.Any time we present user research, we’re selling. Stakeholder mental models are just as relevant as user mental models.

AI-driven personalization is redefining digital experiences, allowing companies to tailor content, recommendations, and interfaces to individual users at an unprecedented scale. From e-commerce product suggestions to content feeds, streaming recommendations, and even customized user interfaces, personalization has become a cornerstone of modern digital strategy. The appeal is clear: research shows that effective personalization can increase engagement by 72%, boost conversion rates by up to 30%, and drive revenue growth of 10-15%.

‍

However, the reality often falls short of these impressive statistics. Personalization can easily backfire, frustrating users instead of engaging them, creating experiences that feel invasive rather than helpful, and sometimes actively driving users away from the very content or products they might genuinely enjoy. Many organizations invest heavily in AI technology while underinvesting in understanding how these personalized experiences actually impact their users.

‍

‍

‍

The Widening Gap Between Capability and Quality

‍

The technical capability to personalize digital experiences has advanced rapidly, but the quality of these experiences hasn't always kept pace. According to a 2023 survey by Baymard Institute, 68% of users reported encountering personalization that felt "off-putting" or "frustrating" in the previous month, while only 34% could recall a personalized experience that genuinely improved their interaction with a digital product.

‍

This disconnect stems from a fundamental misalignment: while AI excels at pattern recognition and prediction based on historical data, it often lacks the contextual understanding and nuance that make personalization truly valuable. The result? Technically sophisticated personalization regularly misses the mark on actual user needs and preferences.

‍

‍

‍

The Pitfalls of AI-Driven Personalization

‍

Many companies struggle with personalization due to several common pitfalls that undermine even the most sophisticated AI implementations:

‍

‍

Over-Personalization: When Helpful Becomes Restrictive

‍

AI that assumes too much can make users feel restricted or trapped in a "filter bubble" of limited options. This phenomenon, often called "over-personalization," occurs when algorithms become too confident in their understanding of user preferences.

‍

Signs of over-personalization include:

‍

• Content feeds that become increasingly homogeneous over time
• Disappearing options that might interest users but don't match their history
• User frustration at being unable to discover new content or products
• Decreased engagement as experiences become predictable and stale

‍

A study by researchers at University of Minnesota found that highly personalized news feeds led to a 23% reduction in content diversity over time, even when users actively sought varied content. This "filter bubble" effect not only limits discovery but can leave users feeling manipulated or constrained.

‍

‍

Incorrect Assumptions: When Data Tells the Wrong Story

‍

AI recommendations based on incomplete or misinterpreted data can lead to irrelevant, inappropriate, or even offensive suggestions. These incorrect assumptions often stem from:

‍

• Limited data points that don't capture the full context of user behavior
• Misinterpreting casual interest as strong preference
• Failing to distinguish between the user's behavior and actions taken on behalf of others
• Not recognizing temporary or situational needs versus ongoing preferences

‍

These misinterpretations can range from merely annoying (continuously recommending products similar to a one-time purchase) to deeply problematic (showing weight loss ads to users with eating disorders based on their browsing history).

‍

A particularly striking example occurred when a major retailer's algorithm began sending pregnancy-related offers to a teenage girl before her family knew she was pregnant. While technically accurate in its prediction, this incident highlights how even "correct" personalization can fail to consider the broader human context and implications.

‍

‍

Lack of Transparency: The Black Box Problem

‍

Users increasingly want to understand why they're being shown specific content or recommendations. When personalization happens behind a "black box" without explanation, it can create:

‍

• Distrust in the system and the brand behind it
• Confusion about how to influence or improve recommendations
• Feelings of being manipulated rather than assisted
• Concerns about what personal data is being used and how

‍

Research from the Pew Research Center shows that 74% of users consider it important to know why they are seeing certain recommendations, yet only 22% of personalization systems provide clear explanations for their suggestions.

‍

‍

Inconsistent Experiences Across Channels

‍

Many organizations struggle to maintain consistent personalization across different touchpoints, creating disjointed experiences:

‍

• Product recommendations that vary wildly between web and mobile
• Personalization that doesn't account for previous customer service interactions
• Different personalization strategies across email, website, and app experiences
• Recommendations that don't adapt to the user's current context or device

‍

This inconsistency can make personalization feel random or arbitrary rather than thoughtfully tailored to the user's needs.

‍

‍

Neglecting Privacy Concerns and Control

‍

As personalization becomes more sophisticated, user concerns about privacy intensify. Key issues include:

‍

• Collecting more data than necessary for effective personalization
• Lack of user control over what information influences their experience
• Unclear opt-out mechanisms for personalization features
• Personalization that reveals sensitive information to others

‍

A recent study found that 79% of users want control over what personal data influences their recommendations, but only 31% felt they had adequate control in their most-used digital products.

‍

‍

‍

How Product Managers Can Leverage UX Insight for Better AI Personalization

‍

To create a personalized experience that feels natural and helpful rather than creepy or restrictive, UX teams need to validate AI-driven decisions through systematic research with real users. Rather than treating personalization as a purely technical challenge, successful organizations recognize it as a human-centered design problem that requires continuous testing and refinement.

‍

‍

Understanding User Mental Models Through Card Sorting & Tree Testing

‍

Card sorting and tree testing help structure content in a way that aligns with users' expectations and mental models, creating a foundation for personalization that feels intuitive rather than imposed:

‍

• Open and Closed Card Sorting – Helps understand how different user segments naturally categorize content, products, or features, providing a baseline for personalization strategies
• Tree Testing – Validates whether personalized navigation structures work for different user types and contexts
• Hybrid Approaches – Combining card sorting with interviews to understand not just how users categorize items, but why they do so

‍

Case Study: A financial services company used card sorting with different customer segments to discover distinct mental models for organizing financial products. Rather than creating a one-size-fits-all personalization system, they developed segment-specific personalization frameworks that aligned with these different mental models, resulting in a 28% increase in product discovery and application rates.

‍

‍

‍

Validating Interaction Patterns Through First-Click Testing

‍

First-click testing ensures users interact with personalized experiences as intended across different contexts and scenarios:

‍

• Testing how users respond to personalized elements vs. standard content
• Evaluating whether personalization cues (like "Recommended for you") influence click behavior
• Comparing how different user segments respond to the same personalization approaches
• Identifying potential confusion points in personalized interfaces

‍

Research by the Nielsen Norman Group found that getting the first click right increases the overall task success rate by 87%. For personalized experiences, this is even more critical, as users may abandon a site entirely if early personalized recommendations seem irrelevant or confusing.

‍

‍

‍

Gathering Qualitative Insights Through User Interviews & Usability Testing

‍

Direct observation and conversation with users provides critical context for personalization strategies:

‍

• Moderated Usability Testing – Reveals how users react to personalized elements in real-time
• Think-Aloud Protocols – Help understand users' expectations and reactions to personalization
• Longitudinal Studies – Track how perceptions of personalization change over time and repeated use
• Contextual Inquiry – Observes how personalization fits into users' broader goals and environments

‍

These qualitative approaches help answer critical questions like:

‍

• When does personalization feel helpful versus intrusive?
• What level of explanation do users want for recommendations?
• How do different user segments react to similar personalization strategies?
• What control do users expect over their personalized experience?

‍

‍

‍

Measuring Sentiment Through Surveys & User Feedback

‍

Systematic feedback collection helps gauge users' comfort levels with AI-driven recommendations:

‍

• Targeted Microsurveys – Quick pulse checks after personalized interactions
• Preference Centers – Direct input mechanisms for refining personalization
• Satisfaction Tracking – Monitoring how personalization affects overall satisfaction metrics
• Feature-Specific Feedback – Gathering input on specific personalization features

‍

A streaming service discovered through targeted surveys that users were significantly more satisfied with content recommendations when they could see a clear explanation of why items were suggested (e.g., "Because you watched X"). Implementing these explanations increased content exploration by 34% and reduced account cancellations by 8%.

‍

‍

‍

A/B Testing Personalization Approaches

‍

Experimental validation ensures personalization actually improves key metrics:

‍

• Testing different levels of personalization intensity
• Comparing explicit versus implicit personalization methods
• Evaluating various approaches to explaining recommendations
• Measuring the impact of personalization on both short and long-term engagement

‍

Importantly, A/B testing should look beyond immediate conversion metrics to consider longer-term impacts on user satisfaction, trust, and retention.

‍

‍

‍

Building a User-Centered Personalization Strategy That Works

‍

To implement personalization that truly enhances user experience, organizations should follow these research-backed principles:

‍

‍

1. Start with User Needs, Not Technical Capabilities

‍

The most effective personalization addresses genuine user needs rather than showcasing algorithmic sophistication:

‍

• Identify specific pain points that personalization could solve
• Understand which aspects of your product would benefit most from personalization
• Determine where users already expect or desire personalized experiences
• Recognize which elements should remain consistent for all users

‍

‍

2. Implement Transparent Personalization

‍

Users increasingly expect to understand and control how their experiences are personalized:

‍

• Clearly communicate what aspects of the experience are personalized
• Explain the primary factors influencing recommendations
• Provide simple mechanisms for users to adjust or reset their personalization
• Consider making personalization opt-in for sensitive domains

‍

‍

3. Design for Serendipity and Discovery

‍

Effective personalization balances predictability with discovery:

‍

• Deliberately introduce variety into recommendations
• Include "exploration" categories alongside highly targeted suggestions
• Monitor and prevent increasing homogeneity in personalized feeds over time
• Allow users to easily branch out beyond their established patterns

‍

‍

4. Apply Progressive Personalization

‍

Rather than immediately implementing highly tailored experiences, consider a gradual approach:

‍

• Begin with light personalization based on explicit user choices
• Gradually introduce more sophisticated personalization as users engage
• Calibrate personalization depth based on relationship strength and context
• Adjust personalization based on user feedback and behavior

‍

‍

5. Establish Continuous Feedback Loops

‍

Personalization should never be "set and forget":

‍

• Implement regular evaluation cycles for personalization effectiveness
• Create easy feedback mechanisms for users to rate recommendations
• Monitor for signs of over-personalization or filter bubbles
• Regularly test personalization assumptions with diverse user groups

‍

‍

‍

The Future of Personalization: Human-Centered AI

‍

As AI capabilities continue to advance, the companies that will succeed with personalization won't necessarily be those with the most sophisticated algorithms, but those who best integrate human understanding into their approach. The future of personalization lies in creating systems that:

‍

• Learn from qualitative human feedback, not just behavioral data
• Respect the nuance and complexity of human preferences
• Maintain transparency in how personalization works
• Empower users with appropriate control
• Balance algorithm-driven efficiency with human-centered design principles

‍

AI should learn from real people, not just data. UX research ensures that personalization enhances, rather than alienates, users by bringing human insight to algorithmic decisions.

‍

By combining the pattern-recognition power of AI with the contextual understanding provided by UX research, organizations can create personalized experiences that feel less like surveillance and more like genuine understanding: experiences that don't just predict what users might click, but truly respond to what they need and value.

‍