June 6, 2024
4 min

Event Recap: Measuring the Value of UX Research at UXDX

Last week Optimal Workshop was delighted to sponsor UXDX USA 2024 in New York. The User Experience event brings together Product, Design, UX, CX, and Engineering professionals and our team had an amazing time meeting with customers, industry experts, and colleagues throughout the conference. This year, we also had the privilege of sharing some of our industry expertise by running an interactive forum on “Measuring the Value of UX Research” - a topic very close to our hearts.

Our forum, hosted by Optimal Workshop CEO Alex Burke and Product Lead Ella Fielding, was focused on exploring the value of User Experience Research (UXR) from both an industry-wide perspective and within the diverse ecosystem of individual companies and teams conducting this type of research today.

The session brought together a global mix of UX professionals for a rich discussion on measuring and demonstrating the effectiveness of and the challenges facing organizations who are trying to tie UXR to tangible business value today.

The main topics for the discuss were: 

  • Metrics that Matter: How do you measure UXR's impact on sales, customer satisfaction, and design influence?
  • Challenges & Strategies: What are the roadblocks to measuring UXR impact, and how can we overcome them?
  • Beyond ROI:  UXR's value beyond just financial metrics

Some of the key takeaways from our discussions during the session were: 

  1. The current state of UX maturity and value
    • Many UX teams don’t measure the impact of UXR on core business metrics and there were more attendees who are not measuring the impact of their work than those that are measuring it. 
    • Alex & Ella discussed with the attendees the current state of UX research maturity and the ability to prove value across different organizations represented in the room. Most organizations were still early in their UX research maturity with only 5% considering themselves advanced in having research culturally embedded.
  1. Defining and proving the value of UX research
    • The industry doesn’t have clear alignment or understanding of what good measurement looks like. Many teams don’t know how to accurately measure UXR impact or don’t have the tools or platforms to measure it, which serve as core roadblocks for measuring UXRs’ impact. 
    • Alex and Ella discussed challenges in defining and proving the value of UX research, with common values being getting closer to customers, innovating faster, de-risking product decisions, and saving time and money. However, the value of research is hard to quantify compared to other product metrics like lines of code or features shipped.
  1. Measuring and advocating for UX research
    • When teams are measuring UXR today there is a strong bias for customer feedback, but little ability or understanding about how to measure impact on business metrics like revenue. 
    • The most commonly used metrics for measuring UXR are quantitative and qualitative feedback from customers as opposed to internal metrics like stakeholder involvement or tieing UXR to business performance metrics (including financial performance). 
    • Attendees felt that in organizations where research is more embedded, researchers spend significant time advocating for research and proving its value to stakeholders rather than just conducting studies. This included tactics like research repositories and pointing to past study impacts as well as ongoing battles to shape decision making processes. 
    • One of our attendees highlighted that engaging stakeholders in the process of defining key research metrics prior to running research was a key for them in proving value internally. 
  1. Relating user research to financial impact
    • Alex and Ella asked the audience if anyone had examples of demonstrating financial impact of research to justify investment in the team and we got some excellent examples from the audience proving that there are tangible ways to tie research outcomes to core business metrics including:
    • Calculating time savings for employees from internal tools as a financial impact metric. 
    • Measuring a reduction in calls to service desks as a way to quantify financial savings from research.
  1. Most attendees recognise the value in embedding UXR more deeply in all levels of their organization - but feel like they’re not succeeding at this today. 
    • Most attendees feel that UXR is not fully embedded in their orgnaization or culture, but that if it was - they would be more successful in proving its overall value.
    • Stakeholder buy-in and engagement with UXR, particularly from senior leadership varied enormously across organizations, and wasn’t regularly measured as an indicator of UXR value 
    • In organizations where research was more successfully embedded, researchers had to spend significant time and effort building relationships with internal stakeholders before and after running studies. This took time and effort away from actual research, but ended up making the research more valuable to the business in the long run. 

With the large range of UX maturity and the democratization of research across teams, we know there’s a lot of opportunity for our customers to improve their ability to tie their user research to tangible business outcomes and embed UX more deeply in all levels of their organizations. To help fill this gap, Optimal Workshop is currently running a large research project on Measuring the Value of UX which will be released in a few weeks.

Keep up to date with the latest news and events by following us on LinkedIn.

Share this article
Author
Amberlie
Denny

Related articles

View all blog articles
Learn more
1 min read

What’s the difference between UI and UX?

UI and UX are two terms that are often used interchangeably and confused for one another, but what do they actually mean? And is there a crossover between them?

These two terms have only grown in use in recent years, thanks largely to the exploding technology sector. This is great news. For organizations, effectively harnessing UX and UI enables them to build products and services that people will actually want to use – and continue using. For users, they’ll have access to products designed for them. 

What is UX? 🤳🎯

User experience (UX as it’s commonly called) refers to the experience that a person has with a product or service. 

We can determine whether a user experience is good or bad based on how easy (or difficult) it is for users to interact with the various elements of a product or service. Is the sign-up flow easy to use? Does the CTA button on the homepage encourage users to click? UX design exists to answer questions like these – and here’s how.

At the core of UX design is user research, which you can use to understand customer pain points and actually build products designed for the people using them. Typically, user research involves the use of a number of different research methods designed to answer specific questions. Card sorting, for example, can show you how people think the information on your website should be arranged.

Designer and information architect Peter Morville came up with the user experience honeycomb, which demonstrates the various components of UX design.

The UX honeycomb. Source.

Don Norman of Nielsen Norman Group defines UX as “[encompassing] all aspects of the end-users interaction with the company, its services, and its products”.

If this seems broad, that’s because it is. UX actually extends beyond just the digital products of an organization and can be used for areas like retail, customer service and more. In fact, there’s actually a growing movement to replace UX with customer experience (CX), as a way of encompassing all of these disparate elements.

What is UI? 🪄📲

User interface (UI), in the most stripped-back definition, is the interface by which a user and a computer system communicate with one another. This includes the touchscreen on your smartphone, the screen on your laptop, your mouse and keyboard and countless other mechanisms.

With this in mind, user interface design is focused on the elements that users will see on these interfaces, such as buttons, text and images. UI design is all about layout, look and feel. The objective of UI design is to visually guide users through an interface so they can complete their task. In a nutshell, you don’t want a user to think too hard about what they’re doing.

Shown here: The user interface of the Tesla Model S. Source.

UI has its origins in the 1980s, when Xerox developed the very first graphical user interface (GUI). Instead of needing to interact with a computer through a programming language, people could now use icons, menus and buttons. The rest, as they say, is history. Apple came along with the Macintosh computer in 1984 (bringing with it the first point and click mouse), and now we’re all carrying smartphones with touch screens that even a baby can operate.

Like UX, UI has grown significantly – going far beyond what you’ll see on a computer screen. Those involved in the field of UI design today will work as much on the interfaces of computer programs and apps as they will on the user interfaces of cars, wearable devices and technologies in the home. If current trends continue, UI design is likely to become an even bigger field in the years ahead.

What’s the difference between UX and UI? 👀

UX and UI are both essential components of a product or service. You can’t have one without the other, and, as we’ve explored, neglecting one could have serious consequences for your product’s success.

The difference between UX and UI is that UX is focused on the experience of using something and UI is focused on the look and feel of the interface. 

“User Experience (UX) and User Interface (UI) are some of the most confused and misused terms in our field. A UI without UX is like a painter slapping paint onto a canvas without thought; while UX without UI is like the frame of a sculpture with no paper mache on it. A great product experience starts with UX followed by UI. Both are essential for the product’s success”. - Rahul Varshney, co-creator of Foster.fm

The difference between UX and UI is that UX is focused on the experience of using something and UI is focused on the look and feel of the interface. 

Or, if you’d prefer a statement from venerable Nielsen Norman Group: “It’s important to distinguish the total user experience from the UI, even though the UI is obviously an extremely important part of the design. As an example, consider a website with movie reviews. Even if the UI for finding a film is perfect, the UX will be poor for a user who wants information about a small independent release if the underlying database only contains movies from the major studios”.

With this in mind, let’s now take a look at the people behind UX and UI. What do the roles look like in these fields? And, more importantly, what do they involve?

UX and UI jobs guide 📱🧑🏻💻

  • Visual designer: This role works with other design roles in the organization (brand, marketing, etc) to ensure designs match brand guidelines. Visual designers also work with UX designers to verify that designs meet accessibility and usability requirements.
  • UX strategist: At the core, a UX strategist should act as a champion of good UX. That is to say, work to ensure the principles of usability and human-centered design are well understood and utilized. They should also assume some of the responsibility of product-market fit, and work with product managers and the ‘business’ side of the organization to mesh business requirements with user requirements.
  • UX designer: The most common UX profession, UX designers should have a strong understanding of the principles of UX design as well as some research ability. Essentially a jack of all trades, the UX designer will float between all stages of the UX lifecycle, helping out with usability tests, putting together prototypes and working with other areas of the organization.
  • Service designer: The service designer looks at the entire end-to-end process and works with other designers, pulling them when required to liaise on visual designs and UI work. In a smaller organization, the responsibilities of this role will typically be absorbed by other roles, but eventually, there comes a time for the service designer. 

Wrap up 🎬

UX and UI as terms are only going to continue to grow, especially as technology and technology companies continue to proliferate across the globe. If you want to make sure that the user experience and user interfaces of your product or service are fit for the people using them, there’s no better place to start than with user research using powerful tools.

Learn more
1 min read

Welcome to our latest addition: Prototype testing 🐣

Today, we’re thrilled to announce the arrival of the latest member of the Optimal family:  Prototype Testing! This exciting and much-requested new tool allows you to test designs early and often with users to gather fast insights, and make confident design decisions to create more intuitive and user-friendly digital experiences. 

Optimal gives you tools you need to easily build a prototype to test using images and screens and creating clickable areas, or you can import a prototype from Figma and get testing. The first iteration of prototype testing is an open beta, and we’ll be working closely with our customers and community to gather feedback and ideas for further improvements in the months to come.

When to use prototype testing 

Prototype testing is a great way to validate design ideas, identify usability issues, and gather feedback from users before investing too heavily in the development of products, websites, and apps. To further inform your insights, it’s a good idea to include sentiment questions or rating scales alongside your tasks.

Early in the design process: Test initial ideas and concepts to gauge user reactions and feelings about your conceptual solutions. 

Iterative design phases: Continuously test and refine prototypes as you make changes and improvements to the designs. 

Before major milestones: Validate designs before key project stages, such as stakeholder reviews or final approvals.

Usability Testing: Conduct summative research to assess a design's overall performance and gauge real user feedback to guide future design decisions and enhancements.

How it works 🧑🏽‍💻

No existing prototype? No problem. We've made it easy to create one right within Optimal. Here's how:

  1. Import your visuals

Start by uploading a series of screenshots or images that represent your design flow. These will form the backbone of your prototype.

  1. Create interactive elements

Once your visuals are in place, it's time to bring them to life. Use our intuitive interface to designate clickable areas on each screen. These will act as navigation points for your test participants.

  1. Set up the flow

Connect your screens in a logical sequence, mirroring the user journey you want to test. This creates a seamless, interactive experience for your participants.

  1. Preview and refine

Before launching your study, take a moment to walk through your prototype. Ensure all clickable areas work as intended and the flow feels natural.

The result? A fully functional prototype that looks and feels like a real digital product. Your test participants will be able to navigate through it just as they would a live website or app, providing you with authentic, actionable insights.

By empowering you to build prototypes from scratch, we're removing barriers to early-stage testing. This means you can validate ideas faster, iterate with confidence, and ultimately deliver better digital experiences.

Or…import your prototypes directly from Figma 

There’s a bit of housekeeping you’ll need to do in Figma in order to provide your participants with the best testing experience and not impact loading times of the prototype. You can import a link to your Figma prototype into your study,  and it will carry across all the interactions you have set up. You’ll need to make sure your Figma presentation mode is made public in order to share the file with participants. If you make any updates to your Figma file, you can sync the changes in just one click. 

Help Article: Find out more about how to set up your Figma file for testing

How to create tasks 🧰

When you set up your study, you’ll create tasks for participants to complete. 

There are two different ways to build tasks in your prototype tests. You can set a correct destination by adding a start screen and a correct destination screen. That way, you can watch how participants navigate your design to find their way to the correct destination. Another option is to set a correct pathway and evaluate how participants navigate a product, app, or website based on the pathway sequence you set. You can add as many pathways or destinations as you like. 

Adding post-task questions is a great way to help gather qualitative feedback on the user's experience, capturing their thoughts, feelings, and perceptions.

Help Article: Find out how to analyze your results

Prototype testing analysis and metrics 📊

Prototype testing offers a variety of analysis options and metrics to evaluate the effectiveness and usability of your design.  By using these analysis options and metrics, you can get comprehensive insights into your prototype's performance, identify areas for improvement, and make informed design decisions:

Task results 

The task results provide a deep analysis at a task level, including the success score, directness score, time taken, misclicks, and the breakdown of the task's success and failure. They provide great insight into the usability of your design to achieve a task. 

  • Success score tells you the total percentage of participants who reached the correct destination or pathway that you defined for this task. It’s a good indicator of a prototype's usability. 
  • Directness score is the total completed results minus the ‘indirect’ results.
  • A path is ‘indirect’ when a participant backtracks, viewing the same page multiple times, or if they nominate the correct destination but don’t follow the correct pathway
  • Time taken is how long it took a participant to complete your task and can be a good indicator of how easy or difficult it was to complete. 
  • Misclicks measure the total number of clicks made on areas of your prototype that weren’t clickable, clicks that didn’t result in a page change.

Clickmaps

Clickmaps provide an aggregate view of user interactions with prototypes, visualizing click patterns to reveal how users navigate and locate information. They display hits and misses on designated clickable areas, average task completion times, and heatmaps showing where users believed the next steps to be. Filters for first, second, and third page visits allow analysis of user behavior over time, including how they adapt when backtracking. This comprehensive data helps designers understand user navigation patterns and improve prototype usability.

Participant paths 

The Paths tab in Optimal provides a powerful visualization to understand and identify common navigation patterns and potential obstacles participants encounter while completing tasks. You can include thumbnails of your screens to enhance your analysis, making it easier to pinpoint where users may face difficulties or where common paths occured.

Coming soon to prototyping 🔮

Later this year, we’re running a closed beta for video recording with prototype testing. This feature captures behaviors and insights not evident in click data alone. The browser-based recording requires no plugins, simplifying setup. Consent for recording is obtained at the start of the testing process and can be customized to align with your organization's policies. This new feature will provide deeper insights into user experience and prototype usability.

These enhancements to prototype testing offer a comprehensive toolkit for user experience analysis. By combining quantitative click data with qualitative video insights, designers and researchers can gain a more nuanced understanding of user behavior, leading to more informed decisions and improved product designs.

Start prototype testing today

Learn more
1 min read

Usability Experts Unite: The Power of Heuristic Evaluation in User Interface Design

Usability experts play an essential role in the user interface design process by evaluating the usability of digital products from a very important perspective - the users! Usability experts utilize various techniques such as heuristic evaluation, usability testing, and user research to gather data on how users interact with digital products and services. This data helps to identify design flaws and areas for improvement, leading to the development of user-friendly and efficient products.

Heuristic evaluation is a usability research technique used to evaluate the user interface design of a digital product based on a set of ‘heuristics’ or ‘usability principles’. These heuristics are derived from a set of established principles of user experience design - attributed to the landmark article “Improving a Human-Computer Dialogue” published by web usability pioneers Jakob Nielsen and Rolf Molich in 1990. The principles focus on the experiential aspects of a user interface. 

In this article, we’ll discuss what heuristic evaluation is and how usability experts use the principles to create exceptional design. We’ll also discuss how usability testing works hand-in-hand with heuristic evaluation, and how minimalist design and user control impact user experience. So, let’s dive in!

Understanding Heuristic Evaluation


Heuristic evaluation helps usability experts to examine interface design against tried and tested rules of thumb. To conduct a heuristic evaluation, usability experts typically work through the interface of the digital product and identify any issues or areas for improvement based on these broad rules of thumb, of which there are ten. They broadly cover the key areas of design that impact user experience - not bad for an article published over 30 years ago!

The ten principles are:

  1. Prevention error: Well-functioning error messages are good, but instead of messages, can these problems be removed in the first place? Remove the opportunity for slips and mistakes to occur.
  2. Consistency and standards: Language, terms, and actions used should be consistent to not cause any confusion.
  3. Control and freedom for users: Give your users the freedom and control to undo/redo actions and exit out of situations if needed.
  4. System status visibility: Let your users know what’s going on with the site. Is the page they’re on currently loading, or has it finished loading?
  5. Design and aesthetics: Cut out unnecessary information and clutter to enhance visibility. Keep things in a minimalist style.
  6. Help and documentation: Ensure that information is easy to find for users, isn’t too large and is focused on your users’ tasks.
  7. Recognition, not recall: Make sure that your users don’t have to rely on their memories. Instead, make options, actions and objects visible. Provide instructions for use too.
  8. Provide a match between the system and the real world: Does the system speak the same language and use the same terms as your users? If you use a lot of jargon, make sure that all users can understand by providing an explanation or using other terms that are familiar to them. Also ensure that all your information appears in a logical and natural order.
  9. Flexibility: Is your interface easy to use and it is flexible for users? Ensure your system can cater to users to all types, from experts to novices.
  10. Help users to recognize, diagnose and recover from errors: Your users should not feel frustrated by any error messages they see. Instead, express errors in plain, jargon-free language they can understand. Make sure the problem is clearly stated and offer a solution for how to fix it.

Heuristic evaluation is a cost-effective way to identify usability issues early in the design process (although they can be performed at any stage) leading to faster and more efficient design iterations. It also provides a structured approach to evaluating user interfaces, making it easier to identify usability issues. By providing valuable feedback on overall usability, heuristic evaluation helps to improve user satisfaction and retention.

The Role of Usability Experts in Heuristic Evaluation

Usability experts play a central role in the heuristic evaluation process by providing feedback on the usability of a digital product, identifying any issues or areas for improvement, and suggesting changes to optimize user experience.

One of the primary goals of usability experts during the heuristic evaluation process is to identify and prevent errors in user interface design. They achieve this by applying the principles of error prevention, such as providing clear instructions and warnings, minimizing the cognitive load on users, and reducing the chances of making errors in the first place. For example, they may suggest adding confirmation dialogs for critical actions, ensuring that error messages are clear and concise, and making the navigation intuitive and straightforward.

Usability experts also use user testing to inform their heuristic evaluation. User testing involves gathering data from users interacting with the product or service and observing their behavior and feedback. This data helps to validate the design decisions made during the heuristic evaluation and identify additional usability issues that may have been missed. For example, usability experts may conduct A/B testing to compare the effectiveness of different design variations, gather feedback from user surveys, and conduct user interviews to gain insights into users' needs and preferences.

Conducting user testing with users that represent, as closely as possible, actual end users, ensures that the product is optimized for its target audience. Check out our tool Reframer, which helps usability experts collaborate and record research observations in one central database.

Minimalist Design and User Control in Heuristic Evaluation

Minimalist design and user control are two key principles that usability experts focus on during the heuristic evaluation process. A minimalist design is one that is clean, simple, and focuses on the essentials, while user control refers to the extent to which users can control their interactions with the product or service.

Minimalist design is important because it allows users to focus on the content and tasks at hand without being distracted by unnecessary elements or clutter. Usability experts evaluate the level of minimalist design in a user interface by assessing the visual hierarchy, the use of white space, the clarity of the content, and the consistency of the design elements. Information architecture (the system and structure you use to organize and label content) has a massive impact here, along with the content itself being concise and meaningful.

Incorporating minimalist design principles into heuristic evaluation can improve the overall user experience by simplifying the design, reducing cognitive load, and making it easier for users to find what they need. Usability experts may incorporate minimalist design by simplifying the navigation and site structure, reducing the number of design elements, and removing any unnecessary content (check out our tool Treejack to conduct site structure, navigation, and categorization research). Consistent color schemes and typography can also help to create a cohesive and unified design.

User control is also critical in a user interface design because it gives users the power to decide how they interact with the product or service. Usability experts evaluate the level of user control by looking at the design of the navigation, the placement of buttons and prompts, the feedback given to users, and the ability to undo actions. Again, usability testing plays an important role in heuristic evaluation by allowing researchers to see how users respond to the level of control provided, and gather feedback on any potential hiccups or roadblocks.

Usability Testing and Heuristic Evaluation

Usability testing and heuristic evaluation are both important components of the user-centered design process, and they complement each other in different ways.

Usability testing involves gathering feedback from users as they interact with a digital product. This feedback can provide valuable insights into how users perceive and use the user interface design, identify any usability issues, and help validate design decisions. Usability testing can be conducted in different forms, such as moderated or unmoderated, remote or in-person, and task-based or exploratory. Check out our usability testing 101 article to learn more.

On the other hand, heuristic evaluation is a method in which usability experts evaluate a product against a set of usability principles. While heuristic evaluation is a useful method to quickly identify usability issues and areas for improvement, it does not involve direct feedback from users.

Usability testing can be used to validate heuristic evaluation findings by providing evidence of how users interact with the product or service. For example, if a usability expert identifies a potential usability issue related to the navigation of a website during heuristic evaluation, usability testing can be used to see if users actually have difficulty finding what they need on the website. In this way, usability testing provides a reality check to the heuristic evaluation and helps ensure that the findings are grounded in actual user behavior.

Usability testing and heuristic evaluation work together in the design process by informing and validating each other. For example, a designer may conduct heuristic evaluation to identify potential usability issues and then use the insights gained to design a new iteration of the product or service. The designer can then use usability testing to validate that the new design has successfully addressed the identified usability issues and improved the user experience. This iterative process of designing, testing, and refining based on feedback from both heuristic evaluation and usability testing leads to a user-centered design that is more likely to meet user needs and expectations.

Conclusion

Heuristic evaluation is a powerful usability research technique that usability experts use to evaluate digital product interfaces based on a set of established principles of user experience design. After all these years, the ten principles of heuristic evaluation still cover the key areas of design that impact user experience, making it easier to identify usability issues early in the design process, leading to faster and more efficient design iterations. Usability experts play a critical role in the heuristic evaluation process by identifying design flaws and areas for improvement, using user testing to validate design decisions, and ensuring that the product is optimized for its intended users.

Minimalist design and user control are two key principles that usability experts focus on during the heuristic evaluation process. A minimalist design is clean, simple, and focuses on the essentials, while user control gives users the freedom and control to undo/redo actions and exit out of situations if needed. By following these principles, usability experts can create an exceptional design that enhances visibility, reduces cognitive load, and provides a positive user experience. 

Ultimately, heuristic evaluation is a cost-effective way to identify usability issues at any point in the design process, leading to faster and more efficient design iterations, and improving user satisfaction and retention. How many of the ten heuristic design principles does your digital product satisfy? 

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.