Usability Experts Unite: The Power of Heuristic Evaluation in User Interface Design

‍

In the rapidly evolving world of user experience research (UXR), demonstrating value and impact has become more crucial than ever. While traditional metrics like Customer Satisfaction (CSAT) scores and Net Promoter Scores (NPS) have long been the go-to measures for UX professionals, they often fall short in capturing the full scope and depth of UXR's impact. As organizations increasingly recognize the strategic importance of user-centered design, it's time to explore more comprehensive and nuanced approaches to measuring UXR's contribution.

‍

Limitations of traditional metrics

‍

CSAT and NPS, while valuable, have significant limitations when it comes to measuring UXR impact. These metrics provide a snapshot of user sentiment but fail to capture the direct influence of research insights on product decisions, business outcomes, or long-term user behavior. Moreover, they can be influenced by factors outside of UXR's control, such as marketing campaigns or competitor actions, making it challenging to isolate the specific impact of research efforts.

‍

Another limitation is the lack of context these metrics provide. They don't offer insights into why users feel a certain way or how specific research-driven improvements contributed to their satisfaction. This absence of depth can lead to misinterpretation of data and missed opportunities for meaningful improvements.

‍

Alternative measurement approaches

‍

To overcome these limitations, UX researchers are exploring alternative approaches to measuring impact. One promising method is the use of proxy measures that more directly tie to research activities. For example, tracking the number of research-driven product improvements implemented or measuring the reduction in customer support tickets related to usability issues can provide more tangible evidence of UXR's impact.

‍

Another approach gaining traction is the integration of qualitative data into impact measurement. By combining quantitative metrics with rich, contextual insights from user interviews and observational studies, researchers can paint a more comprehensive picture of how their work influences user behavior and product success.

‍

Linking UXR to business outcomes

‍

Perhaps the most powerful way to demonstrate UXR's value is by directly connecting research insights to key business outcomes. This requires a deep understanding of organizational goals and close collaboration with stakeholders across functions. For instance, if a key business objective is to increase user retention, UX researchers can focus on identifying drivers of user loyalty and track how research-driven improvements impact retention rates over time.

‍

Risk reduction is another critical area where UXR can demonstrate significant value. By validating product concepts and designs before launch, researchers can help organizations avoid costly mistakes and reputational damage. Tracking the number of potential issues identified and resolved through research can provide a tangible measure of this impact.

‍

Case studies of successful impact measurement

‍

While standardized metrics for UXR impact remain elusive, some organizations have successfully implemented innovative measurement approaches. For example, one technology company developed a "research influence score" that tracks how often research insights are cited in product decision-making processes and the subsequent impact on key performance indicators.

‍

Another case study involves a financial services firm that implemented a "research ROI calculator." This tool estimates the potential cost savings and revenue increases associated with research-driven improvements, providing a clear financial justification for UXR investments.

‍

These case studies highlight the importance of tailoring measurement approaches to the specific context and goals of each organization. By thinking creatively and collaborating closely with stakeholders, UX researchers can develop meaningful ways to quantify their impact and demonstrate the strategic value of their work.

‍

As the field of UXR continues to evolve, so too must our approaches to measuring its impact. By moving beyond traditional metrics and embracing more holistic and business-aligned measurement strategies, we can ensure that the true value of user research is recognized and leveraged to drive organizational success. The future of UXR lies not just in conducting great research, but in effectively communicating its impact and cementing its role as a critical strategic function within modern organizations.

‍

Maximize UXR ROI with Optimal 

‍

While innovative measurement approaches are crucial, having the right tools to conduct and analyze research efficiently is equally important for maximizing UXR's return on investment. This is where the Optimal Workshop platform comes in, offering a comprehensive solution to streamline your UXR efforts and amplify their impact.

‍

The Optimal Platform provides a suite of user-friendly tools designed to support every stage of the research process, from participant recruitment to data analysis and insight sharing. By centralizing your research activities on a single platform, you can significantly reduce the time and resources spent on administrative tasks, allowing your team to focus on generating valuable insights.

‍

Key benefits of using Optimal for improving UXR ROI include:

‍

• Faster research cycles: With automated participant management and data collection tools, you can complete studies more quickly, enabling faster iteration and decision-making.

‍

• Enhanced collaboration: The platform's sharing features make it easy to involve stakeholders throughout the research process, increasing buy-in and ensuring insights are actioned promptly.

‍

• Robust analytics: Advanced data visualization and analysis tools help you uncover deeper insights and communicate them more effectively to decision-makers.

‍

• Scalable research: The platform's user-friendly interface enables non-researchers to conduct basic studies, democratizing research across your organization and increasing its overall impact.

‍

• Comprehensive reporting: Generate professional, insightful reports that clearly demonstrate the value of your research to stakeholders at all levels.

‍

By leveraging the Optimal Workshop, you're not just improving your research processes – you're positioning UXR as a strategic driver of business success. Our platform's capabilities align perfectly with the advanced measurement approaches discussed earlier, enabling you to track research influence, calculate ROI, and demonstrate tangible impact on key business outcomes.

‍

Ready to transform how you measure and communicate the impact of your UX research? Sign up for a free trial of the Optimal platform today and experience firsthand how it can drive your UXR efforts to new heights of efficiency and effectiveness. 

‍

“If storymapping could unearth patterns and bring together a cohesive story that engages audiences in the world of entertainment and film, why couldn’t we use a similar approach to engage our audiences?’Donna Lichaw and Lis Hubert

‍

User Experience work makes the most sense to me in the context of storytelling. So when I saw Donna Lichaw and Lis Hubert’s presentation on storymapping at edUi recently, it resonated. A user’s path through a website can be likened to the traditional storytelling structure of crisis or conflict, exposition — and even a climax or two.

‍

The narrative arc and the user experience

‍

So just how can the same structure that suits fairytales help us to design a compelling experience for our customers? Well, storyboarding is an obvious example of how UX design and storytelling mesh. A traditional storyboard for a movie or TV episode lays out sequential images to help visualize what the final production will show. Similarly, we map out users' needs and journeys via wireframes, sketches, and journey maps, all the while picturing how people will actually interact with the product.

‍

But the connection between storytelling and the user experience design process goes even deeper than that. Every time a user interacts with our website or product, we get to tell them a story. And a traditional literary storytelling structure maps fairly well to just how users interact with the digital stories we’re telling.Hence Donna and Lis’ conception of storymapping as ‘a diagram that maps out a story using a traditional narrative structure called a narrative arc.’ They concede that while ‘using stories in UX design...is nothing new’, a ‘narrative-arc diagram could also help us to rapidly assess content strengths, weaknesses, and opportunities.’

‍

Storytelling was a common theme at edUI

‍

The edUi conference in Richmond, Virginia brought together an assembly of people who produce websites or web content for large institutions. I met people from libraries, universities, museums, various levels of government, and many other places. The theme of storytelling was present throughout, both explicitly and implicitly.Keynote speaker Matt Novak from Paleofuture talked about how futurists of the past tried to predict the future, and what we can learn from the stories they told. Matthew Edgar discussed what stories our failed content tell — what story does a 404 page tell? Or a page telling users they have zero search results? Two great presentations that got me thinking about storytelling in a different way.

‍

Ultimately, it all clicked for me when I attended Donna and Lis’ presentation ‘Storymapping: A Macguyver Approach to Content Strategy’ (and yes, it was as compelling as the title suggests). They presented a case study of how they applied a traditional narrative structure to a website redesign process. The basic story structure we all learned in school usually includes a pretty standard list of elements. Donna and Lis had tweaked the definitions a bit, and applied them to the process of how users interact with web content.

‍

Points on the Narrative Arc (from their presentation)

‍

‍

Exposition — provides crucial background information and often ends with ‘inciting incident’ kicking off the rest of the story

‍

Donna and Lis pointed out that in the context of doing content strategy work, the inciting incident could be the problem that kicks off a development process. I think it can also be the need that brings users to a website to begin with.

‍

Rising Action — Building toward the climax, users explore a website using different approaches

‍

Here I think the analogy is a little looser. While a story can sometimes be well-served by a long and winding rising action, it’s best to keep this part of the process a bit more straightforward in web work. If there’s too much opportunity for wandering, users may get lost or never come back.

‍

Crisis / Climax — The turning point in a story, and then when the conflict comes to a peak

‍

The crisis is what leads users to your site in the first place — a problem to solve, an answer to find, a purchase to make. And to me the climax sounds like the aha! moment that we all aspire to provide, when the user answers their question, makes a purchase, or otherwise feels satisfied from using the site. If a user never gets to this point, their story just peters out unresolved. They’re forced to either begin the entire process again on your site (now feeling frustrated, no doubt), or turn to a competitor.

‍

Falling Action — The story or user interaction starts to wind down and loose ends are tied up

‍

A confirmation of purchase is sent, or maybe the user signs up for a newsletter.

‍

Denouement / Resolution — The end of the story, the main conflict is resolved

‍

The user goes away with a hopefully positive experience, having been able to meet their information or product needs. If we’re lucky, they spread the word to others!Check out Part 2 of Donna and Lis' three-part article on storymapping.  I definitely recommend exploring their ideas in more depth, and having a go at mapping your own UX projects to the above structure.

‍

A word about crises. The idea of a ‘crisis’ is at the heart of the narrative arc. As we know from watching films and reading novels, the main character always has a problem to overcome. So crisis and conflict show up a few times through this process.While the word ‘crisis’ carries some negative connotations (and that clearly applies to visiting a terribly designed site!), I think it can be viewed more generally when we apply the term to user experience. Did your user have a crisis that brought them to your site? What are they trying to resolve by visiting it? Their central purpose can be the crisis that gives rise to all the other parts of their story.

‍

Why storymapping to a narrative arc is good for your design

‍

Mapping a user interaction along the narrative arc makes it easy to spot potential points of frustration, and also serves to keep the inciting incident or fundamental user need in the forefront of our thinking. Those points of frustration and interaction are natural fits for testing and further development.

‍

For example, if your site has a low conversion rate, that translates to users never hitting the climactic point of their story. It might be helpful to look at their interactions from the earlier phases of their story before they get to the climax. Maybe your site doesn’t clearly establish its reason for existing (exposition), or it might be too hard for users to search and explore your content (rising action).Guiding the user through each phase of the structure described above makes it more difficult to skip an important part of how our content is found and used.

‍

We can ask questions like:

• How does each user task fit into a narrative structure?
• Are we dumping them into the climax without any context?
• Does the site lack a resolution or falling action?
• How would it feel to be a user in those situations?

‍

These questions bring up great objectives for qualitative testing — sitting down with a user and asking them to show us their story.

‍

What to do before mapping to narrative arc

‍

Many sessions at edUi also touched on analytics or user testing. In crafting a new story, we can’t ignore what’s already in place — especially if some of it is appreciated by users. So before we can start storymapping the user journey, we need to analyze our site analytics, and run quantitative and qualitative user tests. This user research will give us insights into what story we’re already telling (whether it’s on purpose or not).

‍

What’s working about the narrative, and what isn’t? Even if a project is starting from scratch on a new site, your potential visitors will bring stories of their own. It might be useful to check stats to see if users leave early on in the process, during the exposition phase. A high bounce rate might mean a page doesn't supply that expositional content in a way that's clear and engaging to encourage further interaction.Looking at analytics and user testing data can be like a movie's trial advance screening — you can establish how the audience/users actually want to experience the site's content.

‍

How mapping to the narrative arc is playing out in my UX practice

‍

Since I returned from edUi, I've been thinking about the narrative structure constantly. I find it helps me frame user interactions in a new way, and I've already spotted gaps in storytelling that can be easily filled in. My attention instantly went to the many forms on our site. What’s the Rising Action like at that point? Streamlining our forms and using friendly language can help keep the user’s story focused and moving forward toward clicking that submit button as a climax.

‍

I’m also trying to remember that every user is the protagonist of their own story, and that what works for one narrative might not work for another. I’d like to experiment with ways to provide different kinds of exposition to different users. I think it’s possible to balance telling multiple stories on one site, but maybe it’s not the best idea to mix exposition for multiple stories on the same page.And I also wonder if we could provide cues to a user that direct them to exposition for their own inciting incident...a topic for another article perhaps.What stories are you telling your users? Do they follow a clear arc, or are there rough transitions? These are great questions to ask yourself as you design experiences and analyze existing ones. The edUi conference was a great opportunity to investigate these ideas, and I can’t wait to return next year.

Usability testing is one of the best ways to measure how easy and intuitive to use something is by testing it with real people. You can read about the basics of usability testing here.

‍

Earlier this year, a small team within Optimal Workshop completely redesigned the company blog. More than anything, we wanted to create something that was user-friendly for our readers and would give them a reason to return. I was part of that team, and we ran numerous sessions interviewing regular readers as well as people unfamiliar with our blog. We also ran card sorts, tree tests and other studies to find out all we could about how people search for UX content. Unsurprisingly, one of the most valuable activities we did was usability testing – sitting down with representative users and watching them as they worked through a series of tasks we provided. We asked general questions like “Where would you go to find information about card sorting”, and we also observed them as they searched through our website for learning content.

‍

By stripping away any barriers between ourselves and our users and observing them as they navigated through our website and learning resources, as well as those of other companies, we were able to build a blog with these people’s behaviors and motivations in mind.

‍

Usability testing is an invaluable research method, and every user researcher should be able to run sessions effectively. Here are 5 tips for doing so, in no particular order.

‍

1. Clarify your goals with stakeholders

‍

Never go into a usability test blind. Before you ever sit down with a participant, make sure you know exactly what you want to get out of the session by writing down your research goals. This will help to keep you focused, essentially giving you a guiding light that you can refer back as you go about the various logistical tasks of your research. But you also need to take this a step further. It’s important to make sure that the people who will utilize the results of your research – your stakeholders – have an opportunity to give you their input on the goals as early as possible.

‍

If you’re running usability tests with the aim of creating marketing personas, for example, meet with your organization’s marketing team and figure out the types of information they need to create these personas. In some cases, it’s also helpful to clarify how you plan to gather this data, which can involve explaining some of the techniques you’re going to use.

‍

Lastly, find out how your stakeholders plan to use your findings. If there are a lot of objectives, organize your usability test so you ask the most important questions first. That way, if you end up going off track or you run out of time you’ll have already gathered the most important data for your stakeholders.

‍

2. Be flexible with your questions

‍

A list of pre-prepared questions will help significantly when it comes time to sit down and run your usability testing sessions. But while a list is essential, sometimes it can also pay to ‘follow your nose’ and steer the conversation in a (potentially) more fruitful direction.

‍

How many times have you been having a conversation with a friend over a drink or dinner, only for you both to completely lose track of time and find yourselves discussing something completely unrelated? While it’s not good practice to let your usability testing sessions get off track to this extent, you can surface some very interesting insights by paying close attention to a user’s behavior and answers during a testing session and following interesting leads.

‍

Ideally, and with enough practice, you’ll be able to answer your core (prepared) questions and ask a number of other questions that spring to mind during the session. This is a skill that takes time to master, however.

‍

3. Write a script for your sessions

‍

While a usability test script may sound like a fancy name for your research questions, it’s actually a document that’s much more comprehensive. If you prepare it correctly (we’ll explain how below), you’ll have a document that you can use to capture in-depth insights from your participants.

‍

Here are some of the key things to keep in mind when putting together your script:

‍

• Write a friendly introduction – It may sound obvious, but taking the time to come up with a friendly, warm introduction will get your sessions off to a much better start. The bonus of writing it down is that you’re far less likely to forget it!
• Ask to record the session – It’s important to record your session (whether through video or audio), as you’ll want to go back later and analyze any details you may have missed. This means asking for explicit permission to record participants. In addition to making them feel more comfortable, it’s just good practice to do so.
• Allocate time for the basics – Don’t dive into the complex questions first, use the first few minutes to gather basic data. This could be things like where they work and their familiarity with your organization and/or product.
• Encourage them to explain their thought process – “I’d like you to explain what you’re doing as you make your way through the task”. This simple request will give you an opportunity to ask follow-up questions that you otherwise may not have thought to ask.
• Let participants know that they’re not being tested – Whenever a participant steps into the room for a test, they’re naturally going to feel like they’re being tested. Explain that you’re testing the product, not them. It’s also helpful to let them know that there are no right or wrong answers. This is an important step if you want to keep them relaxed.

‍

It’s often easiest to have a document with your script printed out and ready to go for each usability test.

‍

4. Take advantage of software

‍

You’d never see a builder without a toolbox full of a useful assortment of tools. Likewise, software can make the life of a user research that much easier. The paper-based ways of recording information are still perfectly valid, but introducing custom tools can make both the logistics of user research and the actual sessions themselves much easier to manage.

‍

Take a tool like Calendly, for example. This is a powerful piece of scheduling software that almost completely takes over the endless back and forth of scheduling usability tests. Calendly acts as a middle man between you and your participants, allowing you to set the times you’re free to host usability tests, and then allowing participants to choose a session that suits them from these times.

‍

Our very own Reframer makes the task of running usability tests and analyzing insights that much easier. During your sessions, you can use Reframer to take comprehensive notes and apply tags like “positive” or “struggled” to different observations. Then, after you’ve concluded your tests, Reframer’s analysis function will help you understand wider themes that are present across your participants.

‍

There’s another benefit to using a tool like Reframer. Keeping all of your notes in place will mean you easily pull up data from past research sessions whenever you need to.

‍

5. Involve others

‍

Usability tests (and user interviews, for that matter) are a great opportunity to open up research to your wider organization. Whether it’s stakeholders, other members of your immediate team or even members of entirely different departments, giving them the chance to sit down with users will show them how their products are really being used. If nothing else, these sessions will help those within your organization build empathy with the people they’re building products for.

‍

There are quite a few ways to bring others in, such as:

‍

• To help you set up the research – This can be a helpful exercise for both you (the researcher) and the people you’re bringing in. Collaborate on the overarching research objectives, ask them what types of results they’d like to see and what sort of tasks they think could be used to gather these results.
• As notetakers – Having a dedicated notetaker will make your life as a researcher significantly easier. This means you’ll have someone to record any interesting observations while you focus on running the session. Just let them know what types of notes you’d like to see.
• To help you analyze the data – Once you’ve wrapped up your usability testing sessions, bring others in to help analyze the findings. There’s a good chance that an outside perspective will catch something you may miss. Also, if you’re bringing stakeholders into the analysis stage, they'll get a clearer picture of what it means and where the data came from.

‍

There are myriad other tips and best practices to keep in mind when usability testing, many of which we cover in our introductory page. Important considerations include taking good quality notes, carefully managing participants during the session (not giving them too much guidance) and remaining neutral throughout when answering their questions. If you feel like we’ve missed any really important points, feel free to leave a comment!

‍

Read more

‍

• Usability testing 101 – About to run your first usability test? Check out our handy 101 guide. You’ll also learn how you can use our qualitative research tool Reframer most effectively.
• What is usability testing? – Learn about the concepts of usability testing and how you can develop a plan for your own testing sessions.
• How to run fast and affordable usability tests with hard-to-reach, high-earning users – Finding the right participants for your usability tests can seem pretty tricky. Here’s how you can get the right people for your particular project.