August 30, 2024
7 min

Measuring the impact of UXR: beyond CSAT and NPS

In the rapidly evolving world of user experience research (UXR), demonstrating value and impact has become more crucial than ever. While traditional metrics like Customer Satisfaction (CSAT) scores and Net Promoter Scores (NPS) have long been the go-to measures for UX professionals, they often fall short in capturing the full scope and depth of UXR's impact. As organizations increasingly recognize the strategic importance of user-centered design, it's time to explore more comprehensive and nuanced approaches to measuring UXR's contribution.

Limitations of traditional metrics

CSAT and NPS, while valuable, have significant limitations when it comes to measuring UXR impact. These metrics provide a snapshot of user sentiment but fail to capture the direct influence of research insights on product decisions, business outcomes, or long-term user behavior. Moreover, they can be influenced by factors outside of UXR's control, such as marketing campaigns or competitor actions, making it challenging to isolate the specific impact of research efforts.

Another limitation is the lack of context these metrics provide. They don't offer insights into why users feel a certain way or how specific research-driven improvements contributed to their satisfaction. This absence of depth can lead to misinterpretation of data and missed opportunities for meaningful improvements.

Alternative measurement approaches

To overcome these limitations, UX researchers are exploring alternative approaches to measuring impact. One promising method is the use of proxy measures that more directly tie to research activities. For example, tracking the number of research-driven product improvements implemented or measuring the reduction in customer support tickets related to usability issues can provide more tangible evidence of UXR's impact.

Another approach gaining traction is the integration of qualitative data into impact measurement. By combining quantitative metrics with rich, contextual insights from user interviews and observational studies, researchers can paint a more comprehensive picture of how their work influences user behavior and product success.

Linking UXR to business outcomes

Perhaps the most powerful way to demonstrate UXR's value is by directly connecting research insights to key business outcomes. This requires a deep understanding of organizational goals and close collaboration with stakeholders across functions. For instance, if a key business objective is to increase user retention, UX researchers can focus on identifying drivers of user loyalty and track how research-driven improvements impact retention rates over time.

Risk reduction is another critical area where UXR can demonstrate significant value. By validating product concepts and designs before launch, researchers can help organizations avoid costly mistakes and reputational damage. Tracking the number of potential issues identified and resolved through research can provide a tangible measure of this impact.

Case studies of successful impact measurement

While standardized metrics for UXR impact remain elusive, some organizations have successfully implemented innovative measurement approaches. For example, one technology company developed a "research influence score" that tracks how often research insights are cited in product decision-making processes and the subsequent impact on key performance indicators.

Another case study involves a financial services firm that implemented a "research ROI calculator." This tool estimates the potential cost savings and revenue increases associated with research-driven improvements, providing a clear financial justification for UXR investments.

These case studies highlight the importance of tailoring measurement approaches to the specific context and goals of each organization. By thinking creatively and collaborating closely with stakeholders, UX researchers can develop meaningful ways to quantify their impact and demonstrate the strategic value of their work.

As the field of UXR continues to evolve, so too must our approaches to measuring its impact. By moving beyond traditional metrics and embracing more holistic and business-aligned measurement strategies, we can ensure that the true value of user research is recognized and leveraged to drive organizational success. The future of UXR lies not just in conducting great research, but in effectively communicating its impact and cementing its role as a critical strategic function within modern organizations.

Maximize UXR ROI with Optimal 

While innovative measurement approaches are crucial, having the right tools to conduct and analyze research efficiently is equally important for maximizing UXR's return on investment. This is where the Optimal Workshop platform comes in, offering a comprehensive solution to streamline your UXR efforts and amplify their impact.

The Optimal Platform provides a suite of user-friendly tools designed to support every stage of the research process, from participant recruitment to data analysis and insight sharing. By centralizing your research activities on a single platform, you can significantly reduce the time and resources spent on administrative tasks, allowing your team to focus on generating valuable insights.

Key benefits of using Optimal for improving UXR ROI include:

  • Faster research cycles: With automated participant management and data collection tools, you can complete studies more quickly, enabling faster iteration and decision-making.

  • Enhanced collaboration: The platform's sharing features make it easy to involve stakeholders throughout the research process, increasing buy-in and ensuring insights are actioned promptly.

  • Robust analytics: Advanced data visualization and analysis tools help you uncover deeper insights and communicate them more effectively to decision-makers.

  • Scalable research: The platform's user-friendly interface enables non-researchers to conduct basic studies, democratizing research across your organization and increasing its overall impact.

  • Comprehensive reporting: Generate professional, insightful reports that clearly demonstrate the value of your research to stakeholders at all levels.

By leveraging the Optimal Workshop, you're not just improving your research processes – you're positioning UXR as a strategic driver of business success. Our platform's capabilities align perfectly with the advanced measurement approaches discussed earlier, enabling you to track research influence, calculate ROI, and demonstrate tangible impact on key business outcomes.

Ready to transform how you measure and communicate the impact of your UX research? Sign up for a free trial of the Optimal platform today and experience firsthand how it can drive your UXR efforts to new heights of efficiency and effectiveness. 

Share this article
Author
Optimal
Workshop

Related articles

View all blog articles
Learn more
1 min read

Get a headstart on your research with templates

We’re excited to announce our first six project templates are now available in Optimal Workshop! We understand that not everyone knows where to start with customer research, so these ready-made templates have been created with UX industry experts to give you the confidence to quickly launch studies and get the results you need to make data-driven decisions.  

Templates aren’t only a great solution for people who need guidance with which study type to use and when; our detailed templates also give you the tools to develop your IA thinking, compare the performance of studies over time, and get detailed project plans to guide you through your information architecture. 

How do templates work?

On the dashboard, you’ll see a new button called Browse Templates. From the templates menu, you can select a template that matches your use case, e.g. ‘I need to organise content into categories’. The templates are a helpful starting point, for you to adapt to suit your research goals. 

Let’s take a look at some of our favourite project templates. 

Organize content into categories

This template helps you design the best categories to organize your information based on how your  users think. It's useful for designing your product, website, or knowledge base experience, as well as re-evaluating any part of it. In this template, we will first conduct an open card sort, and then use that information to design a navigation structure that will be tested on end users.

1. First up run a card sort with OptimalSort

During this study, users will organise all information presented to them into categories they create themselves using an open card sort. This method is great for generating category ideas based on how users process this information allowing you to better design an experience in a more user-focused way. To find out more on how to set up your card sort, refer to our card sorting 101 guide.

2. Test your navigation structure with a Treejack

Based on the groupings that were produced from the card sort, you can now generate a hierarchy for your users to test using Treejack. Users search for the information you’ve categorised and represented as a hierarchy, which is valuable because it helps to confirm whether information placed within your hierarchy is findable and understandable.  

To learn more about tree testing, refer to our tree testing 101 guide.


Evaluate an existing navigation experience 

Regularly evaluating an existing navigation experience is a good way to monitor the health and performance of your website and product. This template is useful for both redesigning your experience and for re-evaluating part of it by helping you design ideal categories to organize your information based on how your target users think and improve findability and task completion. 

1. Start by identifying your top tasks using Reframer

Using Reframer, conduct interviews with various stakeholders in your business to evaluate and theme which tasks your organization believes are the most important within your existing environment. This is a solid first step towards building a list of top tasks for testing. Reframer allows you to easily visualize and group your observations by proximity using the affinity map.  

2. Survey users to understand their top tasks

Next, survey users to confirm their top tasks and identify any existing issues with our survey tool Questions. This will provide insight into what users believe are their top tasks and whether anything is getting in their way to achieve them. This step helps to ensure all design work is informed by up-to-date user tasks.

3. Design and test your current experience in Treejack  

Using the prioritised top tasks create a tree test using Treejack to test your navigation experience with your users. For example “How would you open a home loan” or “How would you upgrade your broadband plan” This will enable you to see how your users navigate your website in order to achieve the most business critical tasks in your organization. This is a valuable step that helps to identify information and design problems to solve early in the design process. 

More templates from our community

This is just the beginning of templates in Optimal Workshop and while we continue to add value and build up our collection, we’d love your input! If there are templates that you regularly use and think the community could benefit from, let us know at hello@optimalworkshop.com.

Learn more
1 min read

From Exposition to Resolution: Looking at User Experience as a Narrative Arc

“If storymapping could unearth patterns and bring together a cohesive story that engages audiences in the world of entertainment and film, why couldn’t we use a similar approach to engage our audiences?’Donna Lichaw and Lis Hubert

User Experience work makes the most sense to me in the context of storytelling. So when I saw Donna Lichaw and Lis Hubert’s presentation on storymapping at edUi recently, it resonated. A user’s path through a website can be likened to the traditional storytelling structure of crisis or conflict, exposition — and even a climax or two.

The narrative arc and the user experience

So just how can the same structure that suits fairytales help us to design a compelling experience for our customers? Well, storyboarding is an obvious example of how UX design and storytelling mesh. A traditional storyboard for a movie or TV episode lays out sequential images to help visualize what the final production will show. Similarly, we map out users' needs and journeys via wireframes, sketches, and journey maps, all the while picturing how people will actually interact with the product.

But the connection between storytelling and the user experience design process goes even deeper than that. Every time a user interacts with our website or product, we get to tell them a story. And a traditional literary storytelling structure maps fairly well to just how users interact with the digital stories we’re telling.Hence Donna and Lis’ conception of storymapping as ‘a diagram that maps out a story using a traditional narrative structure called a narrative arc.’ They concede that while ‘using stories in UX design...is nothing new’, a ‘narrative-arc diagram could also help us to rapidly assess content strengths, weaknesses, and opportunities.’

Storytelling was a common theme at edUI

The edUi conference in Richmond, Virginia brought together an assembly of people who produce websites or web content for large institutions. I met people from libraries, universities, museums, various levels of government, and many other places. The theme of storytelling was present throughout, both explicitly and implicitly.Keynote speaker Matt Novak from Paleofuture talked about how futurists of the past tried to predict the future, and what we can learn from the stories they told. Matthew Edgar discussed what stories our failed content tell — what story does a 404 page tell? Or a page telling users they have zero search results? Two great presentations that got me thinking about storytelling in a different way.

Ultimately, it all clicked for me when I attended Donna and Lis’ presentation ‘Storymapping: A Macguyver Approach to Content Strategy’ (and yes, it was as compelling as the title suggests). They presented a case study of how they applied a traditional narrative structure to a website redesign process. The basic story structure we all learned in school usually includes a pretty standard list of elements. Donna and Lis had tweaked the definitions a bit, and applied them to the process of how users interact with web content.

Points on the Narrative Arc (from their presentation)

narrative arc UX

Exposition — provides crucial background information and often ends with ‘inciting incident’ kicking off the rest of the story

Donna and Lis pointed out that in the context of doing content strategy work, the inciting incident could be the problem that kicks off a development process. I think it can also be the need that brings users to a website to begin with.

Rising Action — Building toward the climax, users explore a website using different approaches

Here I think the analogy is a little looser. While a story can sometimes be well-served by a long and winding rising action, it’s best to keep this part of the process a bit more straightforward in web work. If there’s too much opportunity for wandering, users may get lost or never come back.

Crisis / Climax — The turning point in a story, and then when the conflict comes to a peak

The crisis is what leads users to your site in the first place — a problem to solve, an answer to find, a purchase to make. And to me the climax sounds like the aha! moment that we all aspire to provide, when the user answers their question, makes a purchase, or otherwise feels satisfied from using the site. If a user never gets to this point, their story just peters out unresolved. They’re forced to either begin the entire process again on your site (now feeling frustrated, no doubt), or turn to a competitor.

Falling Action — The story or user interaction starts to wind down and loose ends are tied up

A confirmation of purchase is sent, or maybe the user signs up for a newsletter.

Denouement / Resolution — The end of the story, the main conflict is resolved

The user goes away with a hopefully positive experience, having been able to meet their information or product needs. If we’re lucky, they spread the word to others!Check out Part 2 of Donna and Lis' three-part article on storymapping.  I definitely recommend exploring their ideas in more depth, and having a go at mapping your own UX projects to the above structure.

A word about crises. The idea of a ‘crisis’ is at the heart of the narrative arc. As we know from watching films and reading novels, the main character always has a problem to overcome. So crisis and conflict show up a few times through this process.While the word ‘crisis’ carries some negative connotations (and that clearly applies to visiting a terribly designed site!), I think it can be viewed more generally when we apply the term to user experience. Did your user have a crisis that brought them to your site? What are they trying to resolve by visiting it? Their central purpose can be the crisis that gives rise to all the other parts of their story.

Why storymapping to a narrative arc is good for your design

Mapping a user interaction along the narrative arc makes it easy to spot potential points of frustration, and also serves to keep the inciting incident or fundamental user need in the forefront of our thinking. Those points of frustration and interaction are natural fits for testing and further development.

For example, if your site has a low conversion rate, that translates to users never hitting the climactic point of their story. It might be helpful to look at their interactions from the earlier phases of their story before they get to the climax. Maybe your site doesn’t clearly establish its reason for existing (exposition), or it might be too hard for users to search and explore your content (rising action).Guiding the user through each phase of the structure described above makes it more difficult to skip an important part of how our content is found and used.

We can ask questions like:

  • How does each user task fit into a narrative structure?
  • Are we dumping them into the climax without any context?
  • Does the site lack a resolution or falling action?
  • How would it feel to be a user in those situations?

These questions bring up great objectives for qualitative testing — sitting down with a user and asking them to show us their story.

What to do before mapping to narrative arc

Many sessions at edUi also touched on analytics or user testing. In crafting a new story, we can’t ignore what’s already in place — especially if some of it is appreciated by users. So before we can start storymapping the user journey, we need to analyze our site analytics, and run quantitative and qualitative user tests. This user research will give us insights into what story we’re already telling (whether it’s on purpose or not).

What’s working about the narrative, and what isn’t? Even if a project is starting from scratch on a new site, your potential visitors will bring stories of their own. It might be useful to check stats to see if users leave early on in the process, during the exposition phase. A high bounce rate might mean a page doesn't supply that expositional content in a way that's clear and engaging to encourage further interaction.Looking at analytics and user testing data can be like a movie's trial advance screening — you can establish how the audience/users actually want to experience the site's content.

How mapping to the narrative arc is playing out in my UX practice

Since I returned from edUi, I've been thinking about the narrative structure constantly. I find it helps me frame user interactions in a new way, and I've already spotted gaps in storytelling that can be easily filled in. My attention instantly went to the many forms on our site. What’s the Rising Action like at that point? Streamlining our forms and using friendly language can help keep the user’s story focused and moving forward toward clicking that submit button as a climax.

I’m also trying to remember that every user is the protagonist of their own story, and that what works for one narrative might not work for another. I’d like to experiment with ways to provide different kinds of exposition to different users. I think it’s possible to balance telling multiple stories on one site, but maybe it’s not the best idea to mix exposition for multiple stories on the same page.And I also wonder if we could provide cues to a user that direct them to exposition for their own inciting incident...a topic for another article perhaps.What stories are you telling your users? Do they follow a clear arc, or are there rough transitions? These are great questions to ask yourself as you design experiences and analyze existing ones. The edUi conference was a great opportunity to investigate these ideas, and I can’t wait to return next year.

Learn more
1 min read

Transforming digital experiences: Optimal Workshop's radical refresh

At Optimal Workshop, we've always prided ourselves on being pioneers in the field of UX research tools. Since our inception, we've been the go-to platform for UX researchers and designers looking to conduct card sorting, tree testing, and other critical user research activities. Our tools have helped shape the digital experiences of some of the world's leading companies.

The digital landscape is ever-changing, and with it, the needs of the professionals who create and optimize digital experiences. As we've grown and evolved alongside the industry, we've recognized the need to expand our reach and refine our value proposition to better serve not just UX researchers, but all professionals involved in delivering exceptional digital experiences.

That's why we're excited to announce a significant brand refresh and value pivot for Optimal Workshop. This evolution isn't just about a new coat of paint – it's a fundamental shift in how we position ourselves and the value we provide to our users and the broader digital experience community.

Why fix it if it ain’t broke? 💔🔨

Expanding Our User Base

While we've built a strong reputation among UX researchers, we've recognized that there's a broader audience of professionals who benefit from our tools and expertise. Product managers, marketers, customer experience specialists, and others involved in shaping digital experiences often lack access to professional-grade user research tools or the knowledge to effectively implement them.

Our brand refresh aims to make Optimal Workshop more accessible and relevant to this wider circle of professionals, without losing the deep respect we've earned from UX experts.

Adapting to market changes

The UX research landscape has evolved significantly since we first entered the market. With the proliferation of prototyping tools and the increasing speed of digital product development, there's been a shift away from extensive upfront research. However, we firmly believe that incorporating the voice of the customer throughout the development process is more critical than ever.

Our pivot allows us to emphasize the importance of integrating user research and insights at various stages of the digital experience lifecycle, not just during initial design phases.

Leveraging our expertise

As pioneers in the field, we have accumulated a wealth of knowledge about best practices in UX research and digital experience optimization. Our brand refresh positions us not just as a tool provider, but as a trusted partner and thought leader in the industry. We're doubling down on our commitment to sharing knowledge, standardizing best practices, and elevating the work of exceptional practitioners in the field.

Meeting evolving business needs through effective UX

The UX industry is evolving rapidly, with increasing investment and a diverse range of roles getting involved. From junior designers leveraging AI to seasoned researchers with deep expertise, the landscape is rich with talent and tools. However, the true value lies in how effectively these resources translate into business outcomes.

Optimal Workshop recognizes that exceptional digital experiences are no longer just nice-to-have – they're critical for engagement, conversion, and overall business success. Our tools bridge the gap between UX insights and tangible ROI by:

  • Democratizing research: Enabling teams across experience levels to gather valuable user data quickly and efficiently.
  • Accelerating decision-making: Providing fast, actionable insights that reduce design iterations and time-to-market.
  • Enhancing team effectiveness: Facilitating collaboration and knowledge sharing between junior and senior team members.
  • Driving business value: Directly linking UX improvements to key performance indicators and bottom-line results.

In a landscape where basic UX practices are becoming table stakes, Optimal Workshop empowers organizations to go beyond the basics. We help teams leverage the full spectrum of UX expertise – from AI-assisted analysis to seasoned human insight – to create digital experiences that truly set businesses apart and deliver measurable returns on UX investment.

What's changing at Optimal Workshop 🐛🦋

1. Best-in-class research & insights platform

While our core tools remain a critical part of our offering, we're broadening our focus to position Optimal Workshop as a best-in-class research & insights platform for digital experience professionals. This means developing new tools, workflows, and integrations that cater to a wider range of use cases and user types.

2. Accessibility without compromise

We're committed to making our professional-grade tools more accessible to a broader audience without sacrificing the depth and rigor that our expert users expect. This means developing new onboarding experiences, creating more intuitive interfaces, and providing educational resources to help users at all levels get the most out of our platform.

3. Championing best practices

As part of our evolution, we're placing a greater emphasis on identifying, sharing, and standardizing best practices in digital experience research and optimization. Through case studies, partnerships with industry leaders, and our own thought leadership content, we aim to elevate the entire field of digital experience design.

4. Fostering a community of excellence

We're doubling down on our commitment to building and nurturing a community of digital experience professionals. This includes expanding our educational offerings, hosting more events and webinars, and creating opportunities for our users to connect and learn from each other. 

5. Emphasizing outcomes and ROI

We're aligning our messaging and product development with the real-world impact our users are seeking. That’s why the new names for our tools emphasize what each tool helps you achieve, rather than the technical methodology behind it. This outcome-focused approach helps users quickly identify which tool is right for their specific needs.

How our evolution benefits you 🚀

For our long-time users, rest assured that the tools and features you know and love aren't going anywhere, but their names are changing to plain English terms to be more approachable for professionals who may not have a background in UX research. In fact, our UXR platform is only going to get better as we invest in improvements and new capabilities. You'll also benefit from an expanded community of practitioners and a wealth of new resources to help you continue to grow and excel in your field.

For professionals who may be new to Optimal Workshop or to formalized user research in general, our refresh means it's easier than ever to get started with professional-grade tools and methodologies. We're here to support you at every step, from your first card sort to building a comprehensive, data-driven approach to optimizing digital experiences.

Join us in shaping tomorrow's digital experiences 🌟

This brand refresh and value pivot mark an exciting new chapter for Optimal Workshop. We're committed to continuing our tradition of excellence while expanding our reach and impact in the world of digital experiences.

As we move forward, we'll be rolling out new features, resources, and initiatives that align with our refreshed brand and value proposition. We're excited to partner with our users – both new and long-standing – to push the boundaries of what's possible in creating truly exceptional digital experiences.

Thank you for being part of the Optimal Workshop community. Whether you're a UX research veteran or just starting your journey in optimizing digital experiences, we're here to support you with the best tools, knowledge, and community in the industry. Together, we can shape the future of digital experiences and make the online world more intuitive, efficient, and enjoyable for everyone.

Onwards and upwards,

Alex Burke, CEO Optimal Workshop

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.