February 27, 2023

The Role of Usability Metrics in User-Centered Design

Optimal Workshop

The term ‘usability’ captures sentiments of how usable, useful, enjoyable, and intuitive a website or app is perceived by users. By its very nature, usability is somewhat subjective. But what we’re really looking for when we talk about usability is how well a website can be used to achieve a specific task or goal. Using this definition we can analyze usability metrics (standard units of measurement) to understand how well user experience design is performing.

Usability metrics provide helpful insights before and after any digital product is launched. They help us form a deeper understanding of how we can design with the user front of mind. This user-centered design approach is considered the best-practice in building effective information architecture and user experiences that help websites, apps, and software meet and exceed users' needs.

In this article, we’ll highlight key usability metrics, how to measure and understand them, and how you can apply them to improve user experience.

Understanding Usability Metrics

Usability metrics aim to understand three core elements of usability, namely: effectiveness, efficiency, and satisfaction. A variety of research techniques offer designers an avenue for quantifying usability. Quantifying usability is key because we want to measure and understand it objectively, rather than making assumptions.

Types of Usability Metrics

There are a few key metrics that we can measure directly if we’re looking to quantify effectiveness, efficiency, and satisfaction. Here are four common examples:

  • Success rate: Also known as ‘completion rate’, success rate is the percentage of users who were able to successfully complete the tasks.
  • Time-based efficiency: Also known as ‘time on task’, time-based efficiency measures how much time a user needs to complete a certain task.
  • Number of errors: Sounds like what it is! It measures the average number of times where an error occurred per user when performing a given task.
  • Post-task satisfaction: Measures a user's general impression or satisfaction after completing (or not completing) a given task.

How to Collect Usability Metrics


Usability metrics are outputs from research techniques deployed when conducting usability testing. Usability testing in web design, for example, involves assessing how a user interacts with the website by observing (and listening to) users completing defined tasks, such as purchasing a product or signing up for newsletters.

Conducting usability testing and collecting usability metrics usually involves:

  • Defining a set of tasks that you want to test
  • Recruitment of test participants
  • Observing participants (remotely or in-person)
  • Recording detailed observations
  • Follow-up satisfaction survey or questionnaire

Tools such Reframer are helpful in conducting usability tests remotely, and they enable live collaboration of multiple team members. It is extremely handy when trying to record and organize those insightful observations! Using paper prototypes is an inexpensive way to test usability early in the design process.

The Importance of Usability Metrics in User-Centered Design

User-centered design challenges designers to put user needs first. This means in order to deploy user-centered design, you need to understand your user. This is where usability testing and metrics add value to website and app performance; they provide direct, objective insight into user behavior, needs, and frustrations. If your user isn’t getting what they want or expect, they’ll simply leave and look elsewhere.

Usability metrics identify which parts of your design aren’t hitting the mark. Recognizing where users might be having trouble completing certain actions, or where users are regularly making errors, are vital insights when implementing user-centered design. In short, user-centered design relies on data-driven user insight.

But why hark on about usability metrics and user-centered design? Because at the heart of most successful businesses is a well-solved user problem. Take Spotify, for example, which solved the problem of dodgy, pirated digital files being so unreliable. People liked access to free digital music, but they had to battle viruses and fake files to get it. With Spotify, for a small monthly fee, or the cost of listening to a few ads, users have the best of both worlds. The same principle applies to user experience - identify recurring problems, then solve them.

Best Practices for Using Usability Metrics

Usability metrics should be analyzed by design teams of every size. However, there are some things to bear in mind when using usability metrics to inform design decisions:

  • Defining success: Usability metrics are only valuable if they are being measured against clearly defined benchmarks. Many tasks and processes are unique to each business, so use appropriate comparisons and targets; usually in the form of an ‘optimized’ user (a user with high efficiency).
  • Real user metrics: Be sure to test with participants that represent your final user base. For example, there’s little point in testing your team, who will likely be intimately aware of your business structure, terminology, and internal workflows.
  • Test early: Usability testing and subsequent usability metrics provide the most impact early on in the design process. This usually means testing an early prototype or even a paper prototype. Early testing helps to avoid any significant, unforeseen challenges that could be difficult to rewind in your information architecture.
  • Regular testing: Usability metrics can change over time as user behavior and familiarity with digital products evolve. You should also test and analyze the usability of new feature releases on your website or app.

Remember, data analysis is only as good as the data itself. Give yourself the best chance of designing exceptional user experiences by collecting, researching, and analyzing meaningful and accurate usability metrics.

Conclusion

Usability metrics are a guiding light when it comes to user experience. As the old saying goes, “you can’t manage what you can’t measure”. By including usability metrics in your design process, you invite direct user feedback into your product. This is ideal because we want to leave any assumptions or guesswork about user experience at the door.

User-centered design inherently relies on constant user research. Usability metrics such as success rate, time-based efficiency, number of errors, and post-task satisfaction will highlight potential shortcomings in your design. Subsequently, they identify where improvements can be made, AND they lay down a benchmark to check whether any resulting updates addressed the issues.

Ready to start collecting and analyzing usability metrics? Check out our guide to planning and running effective usability tests to get a head start!

Publishing date
February 27, 2023
Share this article

Related articles

min read
Usability Experts Unite: The Power of Heuristic Evaluation in User Interface Design

Usability experts play an essential role in the user interface design process by evaluating the usability of digital products from a very important perspective - the users! Usability experts utilize various techniques such as heuristic evaluation, usability testing, and user research to gather data on how users interact with digital products and services. This data helps to identify design flaws and areas for improvement, leading to the development of user-friendly and efficient products.

Heuristic evaluation is a usability research technique used to evaluate the user interface design of a digital product based on a set of ‘heuristics’ or ‘usability principles’. These heuristics are derived from a set of established principles of user experience design - attributed to the landmark article “Improving a Human-Computer Dialogue” published by web usability pioneers Jakob Nielsen and Rolf Molich in 1990. The principles focus on the experiential aspects of a user interface. 

In this article, we’ll discuss what heuristic evaluation is and how usability experts use the principles to create exceptional design. We’ll also discuss how usability testing works hand-in-hand with heuristic evaluation, and how minimalist design and user control impact user experience. So, let’s dive in!

Understanding Heuristic Evaluation


Heuristic evaluation helps usability experts to examine interface design against tried and tested rules of thumb. To conduct a heuristic evaluation, usability experts typically work through the interface of the digital product and identify any issues or areas for improvement based on these broad rules of thumb, of which there are ten. They broadly cover the key areas of design that impact user experience - not bad for an article published over 30 years ago!

The ten principles are:

  1. Prevention error: Well-functioning error messages are good, but instead of messages, can these problems be removed in the first place? Remove the opportunity for slips and mistakes to occur.
  2. Consistency and standards: Language, terms, and actions used should be consistent to not cause any confusion.
  3. Control and freedom for users: Give your users the freedom and control to undo/redo actions and exit out of situations if needed.
  4. System status visibility: Let your users know what’s going on with the site. Is the page they’re on currently loading, or has it finished loading?
  5. Design and aesthetics: Cut out unnecessary information and clutter to enhance visibility. Keep things in a minimalist style.
  6. Help and documentation: Ensure that information is easy to find for users, isn’t too large and is focused on your users’ tasks.
  7. Recognition, not recall: Make sure that your users don’t have to rely on their memories. Instead, make options, actions and objects visible. Provide instructions for use too.
  8. Provide a match between the system and the real world: Does the system speak the same language and use the same terms as your users? If you use a lot of jargon, make sure that all users can understand by providing an explanation or using other terms that are familiar to them. Also ensure that all your information appears in a logical and natural order.
  9. Flexibility: Is your interface easy to use and it is flexible for users? Ensure your system can cater to users to all types, from experts to novices.
  10. Help users to recognize, diagnose and recover from errors: Your users should not feel frustrated by any error messages they see. Instead, express errors in plain, jargon-free language they can understand. Make sure the problem is clearly stated and offer a solution for how to fix it.

Heuristic evaluation is a cost-effective way to identify usability issues early in the design process (although they can be performed at any stage) leading to faster and more efficient design iterations. It also provides a structured approach to evaluating user interfaces, making it easier to identify usability issues. By providing valuable feedback on overall usability, heuristic evaluation helps to improve user satisfaction and retention.

The Role of Usability Experts in Heuristic Evaluation

Usability experts play a central role in the heuristic evaluation process by providing feedback on the usability of a digital product, identifying any issues or areas for improvement, and suggesting changes to optimize user experience.

One of the primary goals of usability experts during the heuristic evaluation process is to identify and prevent errors in user interface design. They achieve this by applying the principles of error prevention, such as providing clear instructions and warnings, minimizing the cognitive load on users, and reducing the chances of making errors in the first place. For example, they may suggest adding confirmation dialogs for critical actions, ensuring that error messages are clear and concise, and making the navigation intuitive and straightforward.

Usability experts also use user testing to inform their heuristic evaluation. User testing involves gathering data from users interacting with the product or service and observing their behavior and feedback. This data helps to validate the design decisions made during the heuristic evaluation and identify additional usability issues that may have been missed. For example, usability experts may conduct A/B testing to compare the effectiveness of different design variations, gather feedback from user surveys, and conduct user interviews to gain insights into users' needs and preferences.

Conducting user testing with users that represent, as closely as possible, actual end users, ensures that the product is optimized for its target audience. Check out our tool Reframer, which helps usability experts collaborate and record research observations in one central database.

Minimalist Design and User Control in Heuristic Evaluation

Minimalist design and user control are two key principles that usability experts focus on during the heuristic evaluation process. A minimalist design is one that is clean, simple, and focuses on the essentials, while user control refers to the extent to which users can control their interactions with the product or service.

Minimalist design is important because it allows users to focus on the content and tasks at hand without being distracted by unnecessary elements or clutter. Usability experts evaluate the level of minimalist design in a user interface by assessing the visual hierarchy, the use of white space, the clarity of the content, and the consistency of the design elements. Information architecture (the system and structure you use to organize and label content) has a massive impact here, along with the content itself being concise and meaningful.

Incorporating minimalist design principles into heuristic evaluation can improve the overall user experience by simplifying the design, reducing cognitive load, and making it easier for users to find what they need. Usability experts may incorporate minimalist design by simplifying the navigation and site structure, reducing the number of design elements, and removing any unnecessary content (check out our tool Treejack to conduct site structure, navigation, and categorization research). Consistent color schemes and typography can also help to create a cohesive and unified design.

User control is also critical in a user interface design because it gives users the power to decide how they interact with the product or service. Usability experts evaluate the level of user control by looking at the design of the navigation, the placement of buttons and prompts, the feedback given to users, and the ability to undo actions. Again, usability testing plays an important role in heuristic evaluation by allowing researchers to see how users respond to the level of control provided, and gather feedback on any potential hiccups or roadblocks.

Usability Testing and Heuristic Evaluation

Usability testing and heuristic evaluation are both important components of the user-centered design process, and they complement each other in different ways.

Usability testing involves gathering feedback from users as they interact with a digital product. This feedback can provide valuable insights into how users perceive and use the user interface design, identify any usability issues, and help validate design decisions. Usability testing can be conducted in different forms, such as moderated or unmoderated, remote or in-person, and task-based or exploratory. Check out our usability testing 101 article to learn more.

On the other hand, heuristic evaluation is a method in which usability experts evaluate a product against a set of usability principles. While heuristic evaluation is a useful method to quickly identify usability issues and areas for improvement, it does not involve direct feedback from users.

Usability testing can be used to validate heuristic evaluation findings by providing evidence of how users interact with the product or service. For example, if a usability expert identifies a potential usability issue related to the navigation of a website during heuristic evaluation, usability testing can be used to see if users actually have difficulty finding what they need on the website. In this way, usability testing provides a reality check to the heuristic evaluation and helps ensure that the findings are grounded in actual user behavior.

Usability testing and heuristic evaluation work together in the design process by informing and validating each other. For example, a designer may conduct heuristic evaluation to identify potential usability issues and then use the insights gained to design a new iteration of the product or service. The designer can then use usability testing to validate that the new design has successfully addressed the identified usability issues and improved the user experience. This iterative process of designing, testing, and refining based on feedback from both heuristic evaluation and usability testing leads to a user-centered design that is more likely to meet user needs and expectations.

Conclusion

Heuristic evaluation is a powerful usability research technique that usability experts use to evaluate digital product interfaces based on a set of established principles of user experience design. After all these years, the ten principles of heuristic evaluation still cover the key areas of design that impact user experience, making it easier to identify usability issues early in the design process, leading to faster and more efficient design iterations. Usability experts play a critical role in the heuristic evaluation process by identifying design flaws and areas for improvement, using user testing to validate design decisions, and ensuring that the product is optimized for its intended users.

Minimalist design and user control are two key principles that usability experts focus on during the heuristic evaluation process. A minimalist design is clean, simple, and focuses on the essentials, while user control gives users the freedom and control to undo/redo actions and exit out of situations if needed. By following these principles, usability experts can create an exceptional design that enhances visibility, reduces cognitive load, and provides a positive user experience. 

Ultimately, heuristic evaluation is a cost-effective way to identify usability issues at any point in the design process, leading to faster and more efficient design iterations, and improving user satisfaction and retention. How many of the ten heuristic design principles does your digital product satisfy? 

min read
Designing information architecture for mobile apps

Designing for a mobile app is quite different to designing for websites. The content may be similar (even the same) but the intent for users is likely to be different, as are the environments and occasions they use them. When designing for mobile the information architecture (IA) uses a different set of rules. The key consideration needs to be around ease of use on a smaller screen for a user that is possibly on the move and very likely distracted. They have limited time, limited attention and need a quick result.

Your app could be the first point of contact. It might be the only way your user interacts with you on a regular basis. It needs to be super simple, clean, and quick to interact with. A well thought out, thoroughly researched, and organized information architecture plays a big part in helping to deliver an easy and enjoyable user experience.

What is information architecture?

According to Abby Covert, a leader in the field of information architecture, IA is ‘the way we arrange the parts to make sense of the whole.’ Information architecture (IA)is found in every digital product, from websites, apps to an intranet and can even be applied to the physical world in places like libraries and supermarkets.For the purposes of this blog we will focus on the importance of information architecture for mobile apps. Researching and designing an app’s IA with just the right amount of information is key. And providing a way of navigating that content in a way that is quick and intuitive is key to a good user experience.

How is designing for mobile different?

The first thing to understand about mobile app information architecture is that it’s different – and not just with regards to size. The sheer physicality and specifications of mobile devices mean we need to consider different design requirements. Because mobile devices are light and portable, users are in constant contact with them and they are by far the most convenient way to access information. With mobile apps it can be even more important to consider the user journey, to keep that journey as short as possible, and anticipate the user's needs. Consideration should be given to:

  1. Physicality and specifications
  2. Constant accessibility
  3. How people behave and feel

1. Physicality and specifications

Most mobile device interfaces are accessed through touch screens. Users rely on learnt gestures – in addition to a simple interface – to interact. Because of their smaller dimensions, users often expect the content structures to be simpler and smaller. Also, because of limited bandwidth and connectivity, mobile devices require app designs to be optimized for loading time, with reduced data demands.

2. Constant accessibility

Because we have constant access to our mobile devices, we tend to use them a lot more. They come with us on the bus, walking the dog, or even watching TV. We often use them while ‘doing’ something else. This means we often use the device under difficult viewing conditions, or among a variety of distractions.

3. How we behave and feel

We have different attitudes, behaviors and priorities while using mobile devices. Many of us often have our mobile device within arms reach at all times. We have become attached to these devices and feel ‘lost’ when we don’t have them nearby. Some people even consider them an extension of their being!

How do you factor information architecture into your mobile app design?

We need to think of mobile devices as having their own particular information architecture structure to work within their unique requirements and environments. While the structure of a responsive website may follow the same IA, native apps often employ navigational structures that are tab-based. There’s no one or ‘right’ way to architect a mobile site or application. Rather it’s dependent on factors like the size of the content you need to organize or what the intended user journey is that informs the choice of  information architecture structure. Let’s take a look at some of the most popular IA structures for mobile apps:

Hierarchy

A standard website structure with an index page and a series of sub pages. If you are designing a responsive website you may be restricted to this structure, however introducing additional structural patterns could allow you to tailor the user experience for mobile.

Hub and spoke

Gives you a central index from which users will navigate from. It’s the default pattern on Apple’s iPhone, with a home screen and the various apps users download. Users can’t navigate between spokes but must return to the hub (home screen), instead.

Example: Hierarchy
Example: Hub and spoke

Nested doll

Leads users in a linear fashion to more detailed content. It’s a simplified interface which quickly leads on to the next step. It can be most useful when users are in distracting conditions because  it’s a quick and easy method of navigation.

Tabbed view

Regular app users will be familiar with this structure. It’s a collection of sections tied together by a toolbar menu. This allows the user to quickly scan and understand the functionality of the app as soon as it’s opened. Easy to navigate throughout the app.

Example: Nested doll
Example: Tabbed view

Filtered view

Allows the user to navigate within a set of data by selecting filter options to create a view that suits them. Can be more difficult to view on mobile if there is too much content, as it can be difficult to display.

Example: Filtered view

Wrap up

When designing for mobile devices it is important to always keep in mind the user journey and how (and when) users  are likely to be interacting with your app. What is their primary objective? What is your organization's objective? How do you move them through their tasks to enable them to complete them quickly, simply and easily? Working within the size restrictions and limitations of mobile devices and users needs and desires with a thoroughly thought out IA structure will always win on the day.

min read
How to run a remote-friendly co-design

Co-design is a powerful way to improve design processes in two ways: speed and quality. When your work environment allows and encourages people to work remotely in a temporary or permanent way – as we do at Optimal Workshop – we need to get creative if we want to find ways to include everyone.

After a research project, my marketing team was looking at redesigning our Enterprise landing page. We wanted to speed up our experimentation and determined that collaboration was the best approach. Ideation exercises require more than just a team of designers –  we needed to involve the right people with the right knowledge. To generate more diverse, creative and insightful outputs, we invited different personalities as well as different roles. Our team was comprised of a mix of sales, customer success, marketing, developers and designers. All located in different locations around the world, including the United States, New Zealand, France and Ireland.

To make this global session happen, we used Mural, a digital workspace for visual collaboration. It’s a great way to explore ideas in a collaborative space from anywhere with an internet connection and a laptop.

What is a co-design?

A co-design is a fast-paced workshop that generates abundant and diverse ideas beyond what just one individual can produce.

What does it involve?

A clear plan

Define the problem you want to solve, articulate user needs and any pain points you’ve identified. It’s also important to explain why this problem is something worth investigating. This preparation work will help to cut back on inefficient conversations and ensure your team and stakeholders are aligned on goals, opportunities and problem statements.

HMW (How Might We)

HMW questions are a way to frame the challenge you are trying to solve using short, compelling questions to spark creative thinking and discover solutions.

Divergence

Diverging is a quantitative process where you create as many ideas as possible by answering the HMW questions. Each member then presents their ideas while the rest of the team writes their feedback on Post Its in two ways: ‘I like’ – what you like about the solutions and ‘I wonder’ – how you might improve the solutions.

Convergence

Converging is a qualitative process. Once everyone has presented their ideas and looked over the feedback, it’s time to consolidate the best designs into one solution. After presenting their solutions, team members vote on their favorite ideas or the most compelling features by placing stickers on the ones they like. Commonly, each person should have 3 stickers.

What do you want at the end of the co-design?

Once you’ve passed through the divergent and convergent stages for your HMW questions, you’ll hopefully have some key areas of impact to focus on that are worth prototyping in the form of a wireframe or a higher fidelity design. Using this process, you should be able to develop a design that incorporates input from all disciplines and departments.

Tips for running a co-design:

Plan ahead

  • 8 people max per workshop
  • Prior to creating a calendar invite, ask who would be interested to participate via a message or in person. Allow teams to choose who will represent their team.
  • Have your low-fidelity sketching supplies ready: Post-its, markers, blank sheets etc

Setting the context and explaining the rules

  • Write a description within the calendar invite to give some context prior to the meeting. e.g “What we are working on?”, What will we do with the output?” and “What if I am busy and can't come?”.
  • Send reminders the day before to make sure people are registered to the software you might use, specify if they need to bring anything specific(laptop, tablet, etc).
  • Adding a "What’s coming" section to the agenda helps people to stay on track.
  • Set a clear presentation to encourage creativity – inspire your team!

A timer

  • A mechanism for tracking time down to the minute is crucial for keeping track of the work achieved.
  • Schedule a good amount of time for feedback and voting.
  • Allow a 5 to 10 min break time.
  • Involve a second facilitator to help with time tracking.

How we ran a remote-friendly co-design

What worked well?

True effort  and planning

The classic paper/post-it system isn’t suitable for remote workers. Despite video conferencing, it is still hard to hear everyone or read a whiteboard. I wanted a tool that would allow me to include everyone the same way, and allow easy follow up on the achievements. This system needed to work even for people who couldn’t attend or had to leave halfway through the meeting.

Feedback received:

  • "It takes real effort to get these up and running and the use of other software to make use of the time of everyone" 
  • "I can’t imagine doing workshops in any other way in the future"

Communication with remote workers

Avoid questions like “please vote if you are okay to come in early” which makes remote workers feel bad.

Feedback received:

  • "Remote workers feel like they’re a burden in meetings, but you made us feel super included"

What could have been done differently?

Time and people management

Handling time and people is tricky, you can easily run out of time and be forced to rush through the last steps of your meeting. Don’t hesitate to ask for help facilitating the co-design.

Feedback received:

  • "More time to give feedback to everyone's ideas would have been nice"

Remote organization

Find a way to be as inclusive as possible, like finding a remote solution for snacks and drinks.

Feedback received:

  • "Consider snacks for overseas people or let us know that we should get something to be included in that"

Tips for running a remote-friendly co-design:

A well-chosen space and time

Involving people from multiple countries or time zones requires preparation and consideration.  Don't be afraid to ask people to arrive earlier than their standard working hours.

Collaborative tools

  • Use inclusive tools during meetings that allow everyone to follow the work being done. Also making sure that the tool makes it easy for people to come back and review the work later if they can’t attend the meeting.
  • Set up a Mural board that will replace your slide presentation. Everything should live in same place including Context, HMW, dedicated places to upload images etc.
  • Make sure everyone knows how to use all of the tools needed. It might help to start with a small exercise to help everyone get familiar with it. If there’s not enough time for it, set that exercise prior to the meeting (maybe as part of your reminder).

Manage noises

Depending on the number of participants, background music can prevent remote workers from hearing and understanding conversations.

  • Avoid background music.
  • Manage people and time to avoid banter and chit chat where possible..

Be inclusive

Make sure everyone feels like they’re part of a team rather than a burden for being remote. Allow time for questions, repetition and clarification.

  • Avoid questions that make remote workers feel bad like “please vote if you are okay to come in early”.
  • Allow time for questions, repetition and clarification.
  • If you plan some snacks and drinks, either include it in your workshop budget or let the remote workers know they can bring their own food (depending on timezone).

Seeing is believing

Dive into our platform, explore our tools, and discover how easy it can be to conduct effective UX research.