June 6, 2024
4 min

Event Recap: Measuring the Value of UX Research at UXDX

Last week Optimal Workshop was delighted to sponsor UXDX USA 2024 in New York. The User Experience event brings together Product, Design, UX, CX, and Engineering professionals and our team had an amazing time meeting with customers, industry experts, and colleagues throughout the conference. This year, we also had the privilege of sharing some of our industry expertise by running an interactive forum on “Measuring the Value of UX Research” - a topic very close to our hearts.

Our forum, hosted by Optimal Workshop CEO Alex Burke and Product Lead Ella Fielding, was focused on exploring the value of User Experience Research (UXR) from both an industry-wide perspective and within the diverse ecosystem of individual companies and teams conducting this type of research today.

The session brought together a global mix of UX professionals for a rich discussion on measuring and demonstrating the effectiveness of and the challenges facing organizations who are trying to tie UXR to tangible business value today.

The main topics for the discuss were: 

  • Metrics that Matter: How do you measure UXR's impact on sales, customer satisfaction, and design influence?
  • Challenges & Strategies: What are the roadblocks to measuring UXR impact, and how can we overcome them?
  • Beyond ROI:  UXR's value beyond just financial metrics

Some of the key takeaways from our discussions during the session were: 

  1. The current state of UX maturity and value
    • Many UX teams don’t measure the impact of UXR on core business metrics and there were more attendees who are not measuring the impact of their work than those that are measuring it. 
    • Alex & Ella discussed with the attendees the current state of UX research maturity and the ability to prove value across different organizations represented in the room. Most organizations were still early in their UX research maturity with only 5% considering themselves advanced in having research culturally embedded.
  1. Defining and proving the value of UX research
    • The industry doesn’t have clear alignment or understanding of what good measurement looks like. Many teams don’t know how to accurately measure UXR impact or don’t have the tools or platforms to measure it, which serve as core roadblocks for measuring UXRs’ impact. 
    • Alex and Ella discussed challenges in defining and proving the value of UX research, with common values being getting closer to customers, innovating faster, de-risking product decisions, and saving time and money. However, the value of research is hard to quantify compared to other product metrics like lines of code or features shipped.
  1. Measuring and advocating for UX research
    • When teams are measuring UXR today there is a strong bias for customer feedback, but little ability or understanding about how to measure impact on business metrics like revenue. 
    • The most commonly used metrics for measuring UXR are quantitative and qualitative feedback from customers as opposed to internal metrics like stakeholder involvement or tieing UXR to business performance metrics (including financial performance). 
    • Attendees felt that in organizations where research is more embedded, researchers spend significant time advocating for research and proving its value to stakeholders rather than just conducting studies. This included tactics like research repositories and pointing to past study impacts as well as ongoing battles to shape decision making processes. 
    • One of our attendees highlighted that engaging stakeholders in the process of defining key research metrics prior to running research was a key for them in proving value internally. 
  1. Relating user research to financial impact
    • Alex and Ella asked the audience if anyone had examples of demonstrating financial impact of research to justify investment in the team and we got some excellent examples from the audience proving that there are tangible ways to tie research outcomes to core business metrics including:
    • Calculating time savings for employees from internal tools as a financial impact metric. 
    • Measuring a reduction in calls to service desks as a way to quantify financial savings from research.
  1. Most attendees recognise the value in embedding UXR more deeply in all levels of their organization - but feel like they’re not succeeding at this today. 
    • Most attendees feel that UXR is not fully embedded in their orgnaization or culture, but that if it was - they would be more successful in proving its overall value.
    • Stakeholder buy-in and engagement with UXR, particularly from senior leadership varied enormously across organizations, and wasn’t regularly measured as an indicator of UXR value 
    • In organizations where research was more successfully embedded, researchers had to spend significant time and effort building relationships with internal stakeholders before and after running studies. This took time and effort away from actual research, but ended up making the research more valuable to the business in the long run. 

With the large range of UX maturity and the democratization of research across teams, we know there’s a lot of opportunity for our customers to improve their ability to tie their user research to tangible business outcomes and embed UX more deeply in all levels of their organizations. To help fill this gap, Optimal Workshop is currently running a large research project on Measuring the Value of UX which will be released in a few weeks.

Keep up to date with the latest news and events by following us on LinkedIn.

Share this article
Author
Amberlie
Denny

Related articles

View all blog articles
Learn more
1 min read

Meera Pankhania: From funding to delivery - Ensuring alignment from start to finish

It’s a chicken and egg situation when it comes to securing funding for a large transformation program in government. On one hand, you need to submit a business case and, as part of that, you need to make early decisions about how you might approach and deliver the program of work. On the other hand, you need to know enough about the problem you are going to solve to ensure you have sufficient funding to understand the problem better, hire the right people, design the right service, and build it the right way. 

Now imagine securing hundreds of millions of dollars to design and build a service, but not feeling confident about what the user needs are. What if you had the opportunity to change this common predicament and influence your leadership team to carry out alignment activities, all while successfully delivering within the committed time frames?

Meera Pankhania, Design Director and Co-founder of Propel Design, recently spoke at UX New Zealand, the leading UX and IA conference in New Zealand hosted by Optimal Workshop, on traceability and her learnings from delivering a $300 million Government program.

In her talk, Meera helps us understand how to use service traceability techniques in our work and apply them to any environment - ensuring we design and build the best service possible, no matter the funding model.

Background on Meera Pankhania

As a design leader, Meera is all about working on complex, purpose-driven challenges. She helps organizations take a human-centric approach to service transformation and helps deliver impactful, pragmatic outcomes while building capability and leading teams through growth and change.

Meera co-founded Propel Design, a strategic research, design, and delivery consultancy in late 2020. She has 15 years of experience in service design, inclusive design, and product management across the private, non-profit, and public sectors in both the UK and Australia. 

Meera is particularly interested in policy and social design. After a stint in the Australian Public Service, Meera was appointed as a senior policy adviser to the NSW Minister for Customer Service, Hon. Victor Dominello MP. In this role, she played a part in NSW’s response to the COVID pandemic, flexing her design leadership skills in a new, challenging, and important context.

Contact Details:

Email address: meera@propeldesign.com.au

Find Meera on LinkedIn  

From funding to delivery: ensuring alignment from start to finish 🏁🎉👏

Meera’s talk explores a fascinating case study within the Department of Employment Services (Australia) where a substantial funding investment of around $300 million set the stage for a transformative journey. This funding supported the delivery of a revamped Employment Services Model, which had the goal of delivering better services to job seekers and employers, and a better system for providers within this system. The project had a focus on aligning teams prior to delivery, which resulted in a huge amount of groundwork for Meera.

Her journey involved engaging various stakeholders within the department, including executives, to understand the program as a whole and what exactly needed to be delivered. “Traceability” became the watchword for this project, which is laid out in three phases.

  • Phase 1: Aligning key deliverables
  • Phase 2: Ensuring delivery readiness
  • Phase 3: Building sustainable work practices

Phase 1: Aligning key deliverables 🧮

Research and discovery (pre-delivery)

Meera’s work initially meant conducting extensive research and engagement with executives, product managers, researchers, designers, and policymakers. Through this process, a common theme was identified – the urgent (and perhaps misguided) need to start delivering! Often, organizations focus on obtaining funding without adequately understanding the complexities involved in delivering the right services to the right users, leading to half-baked delivery.

After this initial research, some general themes started to emerge:

  1. Assumptions were made that still needed validation
  2. Teams weren’t entirely sure that they understood the user’s needs
  3. A lack of holistic understanding of how much research and design was needed

The conclusion of this phase was that “what” needed to be delivered wasn’t clearly defined. The same was true for “how” it would be delivered.

Traceability

Meera’s journey heavily revolved around the concept of "traceability” and sought to ensure that every step taken within the department was aligned with the ultimate goal of improving employment services. Traceability meant having a clear origin and development path for every decision and action taken. This is particularly important when spending taxpayer dollars!

So, over the course of eight weeks (which turned out to be much longer), the team went through a process of combing through documents in an effort to bring everything together to make sense of the program as a whole. This involved some planning, user journey mapping, and testing and refinement. 

Documenting Key Artifacts

Numerous artifacts and documents played a crucial role in shaping decisions. Meera and her team gathered and organized these artifacts, including policy requirements, legislation, business cases, product and program roadmaps, service maps, and blueprints. The team also included prior research insights and vision documents which helped to shape a holistic view of the required output.

After an effort of combing through the program documents and laying everything out, it became clear that there were a lot of gaps and a LOT to do.

Prioritising tasks

As a result of these gaps, a process of task prioritization was necessary. Tasks were categorized based on a series of factors and then mapped out based on things like user touch points, pain points, features, business policy, and technical capabilities.

This then enabled Meera and the team to create Product Summary Tiles. These tiles meant that each product team had its own summary ahead of a series of planning sessions. It gave them as much context (provided by the traceability exercise) as possible to help with planning. Essentially, these tiles provided teams with a comprehensive overview of their projects i.e. what their user needs, what certain policies require them to deliver, etc.  

Phase 2: Ensuring delivery readiness 🙌🏻

Meera wanted every team to feel confident that we weren’t doing too much or too little in order to design and build the right service, the right way.

Standard design and research check-ins were well adopted, which was a great start, but Meera and the team also built a Delivery Readiness Tool. It was used to assess a team's readiness to move forward with a project. This tool includes questions related to the development phase, user research, alignment with the business case, consideration of policy requirements, and more. Ultimately, it ensures that teams have considered all necessary factors before progressing further. 

Phase 3: Building sustainable work practices 🍃

As the program progressed, several sustainable work practices emerged which Government executives were keen to retain going forward.

Some of these included:

  • ResearchOps Practice: The team established a research operations practice, streamlining research efforts and ensuring that ongoing research was conducted efficiently and effectively.
  • Consistent Design Artifacts: Templates and consistent design artifacts were created, reducing friction and ensuring that teams going forward started from a common baseline.
  • Design Authority and Ways of Working: A design authority was established to elevate and share best practices across the program.
  • Centralized and Decentralized Team Models: The program showcased the effectiveness of a combination of centralized and decentralized team models. A central design team provided guidance and support, while service design leads within specific service lines ensured alignment and consistency.

Why it matters 🔥

Meera's journey serves as a valuable resource for those working on complex design programs, emphasizing the significance of aligning diverse stakeholders and maintaining traceability. Alignment and traceability are critical to ensuring that programs never lose sight of the problem they’re trying to solve, both from the user and organization’s perspective. They’re also critical to delivering on time and within budget!

Traceability key takeaways 🥡

  • Early Alignment Matters: While early alignment is ideal, it's never too late to embark on a traceability journey. It can uncover gaps, increase confidence in decision-making, and ensure that the right services are delivered.
  • Identify and audit: You never know what artifacts will shape your journey. Identify everything early, and don’t be afraid to get clarity on things you’re not sure about.
  • Conducting traceability is always worthwhile: Even if you don’t find many gaps in your program, you will at least gain a high level of confidence that your delivery is focused on the right things.

Delivery readiness key takeaways 🥡

  • Skills Mix is Vital: Assess and adapt team member roles to match their skills and experiences, ensuring they are positioned optimally.
  • Not Everyone Shares the Same Passion: Recognize that not everyone will share the same level of passion for design and research. Make the relevance of these practices clear to all team members.

Sustainability key takeaways 🥡

  • One Size Doesn't Fit All: Tailor methodologies, templates, and practices to the specific needs of your organization.
  • Collaboration is Key: Foster a sense of community and collective responsibility within teams, encouraging shared ownership of project outcomes.

Learn more
1 min read

Optimal Workshop's core values: The foundation of our success 🧱🔥

In the fast-paced world of tech startups and scale-ups, having a strong set of core values is crucial for sustainable growth and success. As the CEO of Optimal Workshop, a best-in-class research and insights platform, I've learned firsthand the importance of establishing and nurturing the right company culture. Today, I'd like to share our journey of revisiting and refining our values, and how these principles are shaping our path forward.

The power of values in leadership 🫶

This is my third run as a CEO, following leadership roles at the digital consultancy Tigerspike and the EdTech company Education Perfect. Each time, I've faced the challenge of taking the business to a whole new level. From experience, I've learned that this transformation begins with getting all the right pieces in play for future growth.

At Optimal Workshop, we've been busy laying the groundwork for our next phase of expansion:

1. Forming a new leadership team

2. Launching a fresh brand identity

3. Developing a new strategy and product direction

4. Introducing our refined team values

These elements, particularly our values, are not just words on a wall. They underpin our future way of working and set the standard for the behaviors we expect from every team member. By doing so, we're creating a culture of high performance with solid foundations to build upon and scale globally.

Our core values: The heart of Optimal Workshop 💖

In April 2024, we embarked on a journey to revisit and refine our company values. This process was collaborative and thoughtful, involving input from team members across all levels of the organization. The result is a set of four core values that truly represent who we are and who we aspire to be:

1. Live in our customers' shoes

At Optimal Workshop, we believe that our success is intrinsically tied to the success of our customers. This value emphasizes empathy, understanding, and a customer-centric approach to everything we do. Team members who embody this value:

  • Deeply understand our products and champion user research and data-driven methodologies
  • Go above and beyond to support our customers, ensuring they derive maximum value from our products and services
  • Show a profound understanding of customer pain points and challenges, always seeking ways to enhance the customer experience
  • Prioritize customer satisfaction and strive to provide the highest level of service in all their endeavors

2. Own it

We take pride in our expertise and resilience. This value is about taking initiative, holding ourselves to high standards, and fostering an environment of trust and commitment. An "Own It" mindset means:

  • Demonstrating reliability and accountability, stepping up to challenges and delivering on promises
  • Being solution-focused, supporting others, and finding the best outcomes for all stakeholders
  • Bringing a high level of quality to daily work, advancing projects while maintaining excellent standards
  • Communicating openly and transparently, and professionally challenging the status quo when necessary

3. Game On!

In the dynamic world of tech and user research, adaptability and continuous learning are key. Our "Game On!" value encapsulates our readiness to pivot, our curiosity, and our commitment to constant improvement. Team members who exemplify this value:

  • Demonstrate the ability to change direction quickly and adapt to new circumstances
  • Actively contribute to discussions, ask insightful questions, and share information effectively
  • Continuously learn and challenge the status quo, always seeking innovative ways to improve our processes and offerings

4. In It Together

Last but certainly not least, we believe in the power of collaboration, mutual support, and compassion. This value underscores our commitment to fostering a positive, respectful work environment where everyone can thrive. "In It Together" means:

  • Creating a collaborative work environment and showing respect to all team members and individuals
  • Bringing colleagues and customers together to achieve better results and reach common goals
  • Demonstrating honesty, transparency, and integrity in team settings
  • Speaking openly but respectfully, always listening actively, and being present and ready to participate

Celebrating our values in action 🥳

To bring these values to life, we recently celebrated our inaugural Value Awards. It was inspiring to see team members being recognized for exemplifying these principles in their daily work:

These winners have set a great benchmark for all of us to follow, showing how our values translate into tangible actions and outcomes.

Looking ahead 🔭

One of the most crucial lessons I've learned as a CEO is the immense value of establishing clear core values and guidelines early in a company's journey. When these fundamental pieces are in place, scaling becomes not just easier, but significantly faster – especially when it comes to global growth. 

By setting these core values and guidelines early, we've created a strong foundation that will enable Optimal Workshop to scale more efficiently and effectively on a global level, maintaining our identity, culture, and high standards of performance, no matter how large we become or how far we expand geographically.

Here's to the journey ahead – onwards and upwards, across borders and beyond!

Alex Burke

Learn more
1 min read

Measuring the impact of UXR: beyond CSAT and NPS

In the rapidly evolving world of user experience research (UXR), demonstrating value and impact has become more crucial than ever. While traditional metrics like Customer Satisfaction (CSAT) scores and Net Promoter Scores (NPS) have long been the go-to measures for UX professionals, they often fall short in capturing the full scope and depth of UXR's impact. As organizations increasingly recognize the strategic importance of user-centered design, it's time to explore more comprehensive and nuanced approaches to measuring UXR's contribution.

Limitations of traditional metrics

CSAT and NPS, while valuable, have significant limitations when it comes to measuring UXR impact. These metrics provide a snapshot of user sentiment but fail to capture the direct influence of research insights on product decisions, business outcomes, or long-term user behavior. Moreover, they can be influenced by factors outside of UXR's control, such as marketing campaigns or competitor actions, making it challenging to isolate the specific impact of research efforts.

Another limitation is the lack of context these metrics provide. They don't offer insights into why users feel a certain way or how specific research-driven improvements contributed to their satisfaction. This absence of depth can lead to misinterpretation of data and missed opportunities for meaningful improvements.

Alternative measurement approaches

To overcome these limitations, UX researchers are exploring alternative approaches to measuring impact. One promising method is the use of proxy measures that more directly tie to research activities. For example, tracking the number of research-driven product improvements implemented or measuring the reduction in customer support tickets related to usability issues can provide more tangible evidence of UXR's impact.

Another approach gaining traction is the integration of qualitative data into impact measurement. By combining quantitative metrics with rich, contextual insights from user interviews and observational studies, researchers can paint a more comprehensive picture of how their work influences user behavior and product success.

Linking UXR to business outcomes

Perhaps the most powerful way to demonstrate UXR's value is by directly connecting research insights to key business outcomes. This requires a deep understanding of organizational goals and close collaboration with stakeholders across functions. For instance, if a key business objective is to increase user retention, UX researchers can focus on identifying drivers of user loyalty and track how research-driven improvements impact retention rates over time.

Risk reduction is another critical area where UXR can demonstrate significant value. By validating product concepts and designs before launch, researchers can help organizations avoid costly mistakes and reputational damage. Tracking the number of potential issues identified and resolved through research can provide a tangible measure of this impact.

Case studies of successful impact measurement

While standardized metrics for UXR impact remain elusive, some organizations have successfully implemented innovative measurement approaches. For example, one technology company developed a "research influence score" that tracks how often research insights are cited in product decision-making processes and the subsequent impact on key performance indicators.

Another case study involves a financial services firm that implemented a "research ROI calculator." This tool estimates the potential cost savings and revenue increases associated with research-driven improvements, providing a clear financial justification for UXR investments.

These case studies highlight the importance of tailoring measurement approaches to the specific context and goals of each organization. By thinking creatively and collaborating closely with stakeholders, UX researchers can develop meaningful ways to quantify their impact and demonstrate the strategic value of their work.

As the field of UXR continues to evolve, so too must our approaches to measuring its impact. By moving beyond traditional metrics and embracing more holistic and business-aligned measurement strategies, we can ensure that the true value of user research is recognized and leveraged to drive organizational success. The future of UXR lies not just in conducting great research, but in effectively communicating its impact and cementing its role as a critical strategic function within modern organizations.

Maximize UXR ROI with Optimal 

While innovative measurement approaches are crucial, having the right tools to conduct and analyze research efficiently is equally important for maximizing UXR's return on investment. This is where the Optimal Workshop platform comes in, offering a comprehensive solution to streamline your UXR efforts and amplify their impact.

The Optimal Platform provides a suite of user-friendly tools designed to support every stage of the research process, from participant recruitment to data analysis and insight sharing. By centralizing your research activities on a single platform, you can significantly reduce the time and resources spent on administrative tasks, allowing your team to focus on generating valuable insights.

Key benefits of using Optimal for improving UXR ROI include:

  • Faster research cycles: With automated participant management and data collection tools, you can complete studies more quickly, enabling faster iteration and decision-making.

  • Enhanced collaboration: The platform's sharing features make it easy to involve stakeholders throughout the research process, increasing buy-in and ensuring insights are actioned promptly.

  • Robust analytics: Advanced data visualization and analysis tools help you uncover deeper insights and communicate them more effectively to decision-makers.

  • Scalable research: The platform's user-friendly interface enables non-researchers to conduct basic studies, democratizing research across your organization and increasing its overall impact.

  • Comprehensive reporting: Generate professional, insightful reports that clearly demonstrate the value of your research to stakeholders at all levels.

By leveraging the Optimal Workshop, you're not just improving your research processes – you're positioning UXR as a strategic driver of business success. Our platform's capabilities align perfectly with the advanced measurement approaches discussed earlier, enabling you to track research influence, calculate ROI, and demonstrate tangible impact on key business outcomes.

Ready to transform how you measure and communicate the impact of your UX research? Sign up for a free trial of the Optimal platform today and experience firsthand how it can drive your UXR efforts to new heights of efficiency and effectiveness. 

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.