5 min read

Addressing AI Bias in UX: How to Build Fairer Digital Experiences

The Growing Challenge of AI Bias in Digital Products

AI is rapidly reshaping our digital landscape, powering everything from recommendation engines to automated customer service and content creation tools. But as these technologies become more widespread, we're facing a significant challenge: AI bias. When AI systems are trained on biased data, they end up reinforcing stereotypes, excluding marginalized groups, and creating inequitable digital experiences that harm both users and businesses.

This isn't just theoretical, we're seeing real-world consequences. Biased AI has led to resume screening tools that favor male candidates, facial recognition systems that perform poorly on darker skin tones, and language models that perpetuate harmful stereotypes. As AI becomes more deeply integrated into our digital experiences, addressing these biases isn't just an ethical imperative t's essential for creating products that truly work for everyone.

Why Does AI Bias Matter for UX?

For those of us in UX and product teams, AI bias isn't just an ethical issue it directly impacts usability, adoption, and trust. Research has shown that biased AI can result in discriminatory hiring algorithms, skewed facial recognition software, and search engines that reinforce societal prejudices (Buolamwini & Gebru, 2018).

When AI is applied to UX, these biases show up in several ways:

  • Navigation structures that favor certain user behaviors
  • Chatbots that struggle to recognize diverse dialects or cultural expressions
  • Recommendation engines that create "filter bubbles" 
  • Personalization algorithms that make incorrect assumptions 

These biases create real barriers that exclude users, diminish trust, and ultimately limit how effective our products can be. A 2022 study by the Pew Research Center found that 63% of Americans are concerned about algorithmic decision-making, with those concerns highest among groups that have historically faced discrimination.

The Root Causes of AI Bias

To tackle AI bias effectively, we need to understand where it comes from:

1. Biased Training Data

AI models learn from the data we feed them. If that data reflects historical inequities or lacks diversity, the AI will inevitably perpetuate these patterns. Think about a language model trained primarily on text written by and about men,  it's going to struggle to represent women's experiences accurately.

2. Lack of Diversity in Development Teams

When our AI and product teams lack diversity, blind spots naturally emerge. Teams that are homogeneous in background, experience, and perspective are simply less likely to spot potential biases or consider the needs of users unlike themselves.

3. Insufficient Testing Across Diverse User Groups

Without thorough testing across diverse populations, biases often go undetected until after launch when the damage to trust and user experience has already occurred.

How UX Research Can Mitigate AI Bias

At Optimal, we believe that continuous, human-centered research is key to designing fair and inclusive AI-driven experiences. Good UX research helps ensure AI-driven products remain unbiased and effective by:

Ensuring Diverse Representation

Conducting usability tests with participants from varied backgrounds helps prevent exclusionary patterns. This means:

  • Recruiting research participants who truly reflect the full diversity of your user base
  • Paying special attention to traditionally underrepresented groups
  • Creating safe spaces where participants feel comfortable sharing their authentic experiences
  • Analyzing results with an intersectional lens, looking at how different aspects of identity affect user experiences

Establishing Bias Monitoring Systems

Product owners can create ongoing monitoring systems to detect bias:

  • Develop dashboards that track key metrics broken down by user demographics
  • Schedule regular bias audits of AI-powered features
  • Set clear thresholds for when disparities require intervention
  • Make it easy for users to report perceived bias through simple feedback mechanisms

Advocating for Ethical AI Practices

Product owners are in a unique position to advocate for ethical AI development:

  • Push for transparency in how AI makes decisions that affect users
  • Champion features that help users understand AI recommendations
  • Work with data scientists to develop success metrics that consider equity, not just efficiency
  • Promote inclusive design principles throughout the entire product development lifecycle

The Future of AI and Inclusive UX

As AI becomes more sophisticated and pervasive, the role of customer insight and UX in ensuring fairness will only grow in importance. By combining AI's efficiency with human insight, we can ensure that AI-driven products are not just smart but also fair, accessible, and truly user-friendly for everyone. The question isn't whether we can afford to invest in this work, it's whether we can afford not to.

Share this article
Author
Optimal
Workshop

Related articles

View all blog articles
Learn more
1 min read

The AI Automation Breakthrough: Key Insights from Our Latest Community Event

Last night, Optimal brought together an incredible community of product leaders and innovators for "The Automation Breakthrough: Workflows for the AI Era" at Q-Branch in Austin, Texas. This two-hour in-person event featured expert perspectives on how AI and automation are transforming the way we work, create, and lead.

The event featured a lightning Talk on "Designing for Interfaces" featured Cindy Brummer, Founder of Standard Beagle Studio, followed by a dynamic panel discussion titled "The Automation Breakthrough" with industry leaders including Joe Meersman (Managing Partner, Gyroscope AI), Carmen Broomes (Head of UX, Handshake), Kasey Randall (Product Design Lead, Posh AI), and Prateek Khare (Head of Product, Amazon). We also had a fireside chat with our CEO, Alex Burke and Stu Smith, Head of Design at Atlassian. 

Here are the key themes and insights that emerged from these conversations:

Trust & Transparency: The Foundation of AI Adoption

Cindy emphasized that trust and transparency aren't just nice-to-haves in the AI era, they're essential. As AI tools become more integrated into our workflows, building systems that users can understand and rely on becomes paramount. This theme set the tone for the entire event, reminding us that technological advancement must go hand-in-hand with ethical considerations.

Automation Liberates Us from Grunt Work

One of the most resonant themes was how AI fundamentally changes what we spend our time on. As Carmen noted, AI reduces the grunt work and tasks we don't want to do, freeing us to focus on what matters most. This isn't about replacing human workers, it's about eliminating the tedious, repetitive tasks that drain our energy and creativity.

Enabling Creativity and Higher-Quality Decision-Making

When automation handles the mundane, something remarkable happens: we gain space for deeper thinking and creativity. The panelists shared powerful examples of this transformation:

Carmen described how AI and workflows help teams get to insights and execution on a much faster scale, rather than drowning in comments and documentation. Prateek encouraged the audience to use automation to get creative about their work, while Kasey shared how AI and automation have helped him develop different approaches to coaching, mentorship, and problem-solving, ultimately helping him grow as a leader.

The decision-making benefits were particularly striking. Prateek explained how AI and automation have helped him be more thoughtful about decisions and make higher-quality choices, while Kasey echoed that these tools have helped him be more creative and deliberate in his approach.

Democratizing Product Development

Perhaps the most exciting shift discussed was how AI is leveling the playing field across organizations. Carmen emphasized the importance of anyone, regardless of their role, being able to get close to their customers. This democratization means that everyone can get involved in UX, think through user needs, and consider the best experience.

The panel explored how roles are blurring in productive ways. Kasey noted that "we're all becoming product builders" and that product managers are becoming more central to conversations. Prateek predicted that teams are going to get smaller and achieve more with less as these tools become more accessible.

Automation also plays a crucial role in iteration, helping teams incorporate customer feedback more effectively, according to Prateek.

Practical Advice for Navigating the AI Era

The panelists didn't just share lofty visions, they offered concrete guidance for professionals navigating this transformation:

Stay perpetually curious. Prateek warned that no acquired knowledge will stay with you for long, so you need to be ready to learn anything at any time.

Embrace experimentation. "Allow your process to misbehave," Prateek advised, encouraging attendees to break from rigid workflows and explore new approaches.

Overcome fear. Carmen urged the audience not to be afraid of bringing in new tools or worrying that AI will take their jobs. The technology is here to augment, not replace.

Just start. Kasey's advice was refreshingly simple: "Just start and do it again." Whether you're experimenting with AI tools or trying "vibe coding," the key is to begin and iterate.

The energy in the room at Q-Branch reflected a community that's not just adapting to change but actively shaping it. The automation breakthrough isn't just about new tools, it's about reimagining how we work, who gets to participate in product development, and what becomes possible when we free ourselves from repetitive tasks.

As we continue to navigate the AI era, events like this remind us that the most valuable insights come from bringing diverse perspectives together. The conversation doesn't end here, it's just beginning.

Interested in joining future Optimal community events? Stay tuned for upcoming gatherings where we'll continue exploring the intersection of design, product, and emerging technologies.

Learn more
1 min read

My journey running a design sprint

Recently, everyone in the design industry has been talking about design sprints. So, naturally, the team at Optimal Workshop wanted to see what all the fuss was about. I picked up a copy of The Sprint Book and suggested to the team that we try out the technique.

In order to keep momentum, we identified a current problem and decided to run the sprint only two weeks later. The short notice was a bit of a challenge, but in the end we made it work. Here’s a run down of how things went, what worked, what didn’t, and lessons learned.

A sprint is an intensive focused period of time to get a product or feature designed and tested with the goal of knowing whether or not the team should keep investing in the development of the idea. The idea needs to be either validated or not validated by the end of the sprint. In turn, this saves time and resource further down the track by being able to pivot early if the idea doesn’t float.

If you’re following The Sprint Book you might have a structured 5 day plan that looks likes this:

  • Day 1 - Understand: Discover the business opportunity, the audience, the competition, the value proposition and define metrics of success.
  • Day 2 - Diverge: Explore, develop and iterate creative ways of solving the problem, regardless of feasibility.
  • Day 3 - Converge: Identify ideas that fit the next product cycle and explore them in further detail through storyboarding.
  • Day 4 - Prototype: Design and prepare prototype(s) that can be tested with people.
  • Day 5 - Test: User testing with the product's primary target audience.
Design sprint cycle
 With a Design Sprint, a product doesn't need to go full cycle to learn about the opportunities and gather feedback.

When you’re running a design sprint, it’s important that you have the right people in the room. It’s all about focus and working fast; you need the right people around in order to do this and not have any blocks down the path. Team, stakeholder and expert buy-in is key — this is not a task just for a design team!After getting buy in and picking out the people who should be involved (developers, designers, product owner, customer success rep, marketing rep, user researcher), these were my next steps:

Pre-sprint

  1. Read the book
  2. Panic
  3. Send out invites
  4. Write the agenda
  5. Book a meeting room
  6. Organize food and coffee
  7. Get supplies (Post-its, paper, Sharpies, laptops, chargers, cameras)

Some fresh smoothies for the sprinters made by our juice technician
 Some fresh smoothies for the sprinters made by our juice technician

The sprint

Due to scheduling issues we had to split the sprint over the end of the week and weekend. Sprint guidelines suggest you hold it over Monday to Friday — this is a nice block of time but we had to do Thursday to Thursday, with the weekend off in between, which in turn worked really well. We are all self confessed introverts and, to be honest, the thought of spending five solid days workshopping was daunting. At about two days in, we were exhausted and went away for the weekend and came back on Monday feeling sociable and recharged again and ready to examine the work we’d done in the first two days with fresh eyes.

Design sprint activities

During our sprint we completed a range of different activities but here’s a list of some that worked well for us. You can find out more information about how to run most of these over at The Sprint Book website or checkout some great resources over at Design Sprint Kit.

Lightning talks

We kicked off our sprint by having each person give a quick 5-minute talk on one of these topics in the list below. This gave us all an overview of the whole project and since we each had to present, we in turn became the expert in that area and engaged with the topic (rather than just listening to one person deliver all the information).

Our lightning talk topics included:

  • Product history - where have we come from so the whole group has an understanding of who we are and why we’ve made the things we’ve made.
  • Vision and business goals - (from the product owner or CEO) a look ahead not just of the tools we provide but where we want the business to go in the future.
  • User feedback - what have users been saying so far about the idea we’ve chosen for our sprint. This information is collected by our User Research and Customer Success teams.
  • Technical review - an overview of our tech and anything we should be aware of (or a look at possible available tech). This is a good chance to get an engineering lead in to share technical opportunities.
  • Comparative research - what else is out there, how have other teams or products addressed this problem space?

Empathy exercise

I asked the sprinters to participate in an exercise so that we could gain empathy for those who are using our tools. The task was to pretend we were one of our customers who had to present a dendrogram to some of our team members who are not involved in product development or user research. In this frame of mind, we had to talk through how we might start to draw conclusions from the data presented to the stakeholders. We all gained more empathy for what it’s like to be a researcher trying to use the graphs in our tools to gain insights.

How Might We

In the beginning, it’s important to be open to all ideas. One way we did this was to phrase questions in the format: “How might we…” At this stage (day two) we weren’t trying to come up with solutions — we were trying to work out what problems there were to solve. ‘We’ is a reminder that this is a team effort, and ‘might’ reminds us that it’s just one suggestion that may or may not work (and that’s OK). These questions then get voted on and moved into a workshop for generating ideas (see Crazy 8s).Read a more detailed instructions on how to run a ‘How might we’ session on the Design Sprint Kit website.

Crazy 8s

This activity is a super quick-fire idea generation technique. The gist of it is that each person gets a piece of paper that has been folded 8 times and has 8 minutes to come up with eight ideas (really rough sketches). When time is up, it’s all pens down and the rest of the team gets to review each other's ideas.In our sprint, we gave each person Post-it notes, paper, and set the timer for 8 minutes. At the end of the activity, we put all the sketches on a wall (this is where the art gallery exercise comes in).

Mila our data scientist sketching intensely during Crazy 8s
 Mila our data scientist sketching intensely during Crazy 8s

A close up of some sketches from the team
 A close up of some sketches from the team

Art gallery/Silent critique

The art gallery is the place where all the sketches go. We give everyone dot stickers so they can vote and pull out key ideas from each sketch. This is done silently, as the ideas should be understood without needing explanation from the person who made them. At the end of it you’ve got a kind of heat map, and you can see the ideas that stand out the most. After this first round of voting, the authors of the sketches get to talk through their ideas, then another round of voting begins.

Mila putting some sticky dots on some sketches
 Mila putting some sticky dots on some sketches

Bowie, our head of security/office dog, even took part in the sprint...kind of.
 Bowie, our head of security, even took part in the sprint...kind of

Usability testing and validation

The key part of a design sprint is validation. For one of our sprints we had two parts of our concept that needed validating. To test one part we conducted simple user tests with other members of Optimal Workshop (the feature was an internal tool). For the second part we needed to validate whether we had the data to continue with this project, so we had our data scientist run some numbers and predictions for us.

6-dan-design-sprintOur remote worker Rebecca dialed in to watch one of our user tests live
 Our remote worker Rebecca dialed in to watch one of our user tests live
"I'm pretty bloody happy" — Actual feedback.
 Actual feedback

Challenges and outcomes

One of our key team members, Rebecca, was working remotely during the sprint. To make things easier for her, we set up 2 cameras: one pointed to the whiteboard, the other was focused on the rest of the sprint team sitting at the table. Next to that, we set up a monitor so we could see Rebecca.

Engaging in workshop activities is a lot harder when working remotely. Rebecca would get around this by completing the activities and take photos to send to us.

8-rebecca-design-sprint
 For more information, read this great Medium post about running design sprints remotely

Lessons

  • Lightning talks are a great way to have each person contribute up front and feel invested in the process.
  • Sprints are energy intensive. Make sure you’re in a good place with plenty of fresh air with comfortable chairs and a break out space. We like to split the five days up so that we get a weekend break.
  • Give people plenty of notice to clear their schedules. Asking busy people to take five days from their schedule might not go down too well. Make sure they know why you’d like them there and what they should expect from the week. Send them an outline of the agenda. Ideally, have a chat in person and get them excited to be part of it.
  • Invite the right people. It’s important that you get the right kind of people from different parts of the company involved in your sprint. The role they play in day-to-day work doesn’t matter too much for this. We’re all mainly using pens and paper and the more types of brains in the room the better. Looking back, what we really needed on our team was a customer support team member. They have the experience and knowledge about our customers that we don’t have.
  • Choose the right sprint problem. The project we chose for our first sprint wasn’t really suited for a design sprint. We went in with a well defined problem and a suggested solution from the team instead of having a project that needed fresh ideas. This made the activities like ‘How Might We’ seem very redundant. The challenge we decided to tackle ended up being more of a data prototype (spreadsheets!). We used the week to validate assumptions around how we can better use data and how we can write a script to automate some internal processes. We got the prototype working and tested but due to the nature of the project we will have to run this experiment in the background for a few months before any building happens.

Overall, this design sprint was a great team bonding experience and we felt pleased with what we achieved in such a short amount of time. Naturally, here at Optimal Workshop, we're experimenters at heart and we will keep exploring new ways to work across teams and find a good middle ground.

Further reading

Learn more
1 min read

How AI is Augmenting, Not Replacing, UX Researchers

Despite AI being the buzzword in UX right now, there are still lots of concerns about how it’s going to impact research roles. One of the biggest concerns we hear is: is AI just going to replace UX researchers altogether?

The answer, in our opinion, is no. The longer, more interesting answer is that AI is fundamentally transforming what it means to be a UX researcher, and in ways that make the role more strategic, more impactful, and more interesting than ever before.

What AI Actually Does for Research 

A 2024 survey by the UX Research Collective found that 68% of UX researchers are concerned about AI's impact on their roles. The fear makes sense, we've all seen how automation has transformed other industries. But what's actually happening is that rather than AI replacing researchers, it's eliminating the parts of research that researchers hate most.

According to Gartner's 2024 Market Guide for User Research, AI tools can reduce analysis time by 60-70%, but not by replacing human insight. Instead, they handle:

  • Pattern Recognition at Scale: AI can process hundreds of user interviews and identify recurring themes in hours. For a human researcher that same work would take weeks. But those patterns will need human validation because AI doesn't understand why those patterns matter. That's where researchers will continue to add value, and we would argue, become more important than ever. 
  • Synthesis Acceleration: According to research by the Nielsen Norman Group, AI can generate first-draft insight summaries 10x faster than humans. But these summaries still need researcher oversight to ensure context, accuracy, and actionable insights aren't lost. 
  • Multi-language Analysis: AI can analyze feedback in 50+ languages simultaneously, democratizing global research. But cultural context and nuanced interpretation still require human understanding. 
  •  Always-On Insights:  Traditional research is limited by human availability. Tools like AI interviewers can  run 24/7 while your team sleeps, allowing you to get continuous, high-quality user insights. 

AI is Elevating the Role of Researchers 

We think that what AI is actually doing  is making UX researchers more important, not less. By automating the less sophisticated  aspects of research, AI is pushing researchers toward the strategic work that only humans can do.

From Operators to Strategists: McKinsey's 2024 research shows that teams using AI research tools spend 45% more time on strategic planning and only 20% on execution, compared to 30% strategy and 60% execution for traditional teams.

From Reporters  to Storytellers: With AI handling data processing, researchers can focus on crafting compelling narratives. 

From Analysts to Advisors: When freed from manual analysis, researchers become embedded strategic partners. 

Human + AI Collaboration 

The most effective research teams aren't choosing between human or AI, they're creating collaborative workflows that incorporate AI to augment researchers roles, not replace them: 

  • AI-Powered Data Collection: Automated transcription, sentiment analysis, and preliminary coding happen in real-time during user sessions.
  • Human-Led Interpretation: Researchers review AI-generated insights, add context, challenge assumptions, and identify what AI might have missed.
  • Collaborative Synthesis: AI suggests patterns and themes; researchers validate, refine, and connect to business context.
  • Human Storytelling: Researchers craft narratives, implications, and recommendations that AI cannot generate.

Is it likely that with AI more and more research tasks will become automated? Absolutely. Basic transcription, preliminary coding, and simple pattern recognition are already AI’s bread and butter. But research has never been about these tasks, it's been about understanding users and driving better decisions and that should always be left to humans. 

The researchers thriving in 2025 and beyond aren't fighting AI, they're embracing it. They're using AI to handle the tedious 40% of their job so they can focus on the strategic 60% that creates real business value. You  have a choice. You can choose to adopt AI as a tool to elevate your role, or you can view it as a threat and get left behind. Our customers tell us that the researchers choosing elevation are finding their roles more strategic, more impactful, and more essential to product success than ever before.

AI isn't replacing UX researchers. It's freeing them to do what they've always done best, understand humans and help build better products. And in a world drowning in data but starving for insight, that human expertise has never been more valuable.

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.