The Role of Usability Metrics in User-Centered Design

The term ‘usability’ captures sentiments of how usable, useful, enjoyable, and intuitive a website or app is perceived by users. By its very nature, usability is somewhat subjective. But what we’re really looking for when we talk about usability is how well a website can be used to achieve a specific task or goal. Using this definition we can analyze usability metrics (standard units of measurement) to understand how well user experience design is performing.

‍

Usability metrics provide helpful insights before and after any digital product is launched. They help us form a deeper understanding of how we can design with the user front of mind. This user-centered design approach is considered the best-practice in building effective information architecture and user experiences that help websites, apps, and software meet and exceed users' needs.

‍

In this article, we’ll highlight key usability metrics, how to measure and understand them, and how you can apply them to improve user experience.

‍

Understanding Usability Metrics

‍

Usability metrics aim to understand three core elements of usability, namely: effectiveness, efficiency, and satisfaction. A variety of research techniques offer designers an avenue for quantifying usability. Quantifying usability is key because we want to measure and understand it objectively, rather than making assumptions.

‍

Types of Usability Metrics
‍

There are a few key metrics that we can measure directly if we’re looking to quantify effectiveness, efficiency, and satisfaction. Here are four common examples:

‍

• Success rate: Also known as ‘completion rate’, success rate is the percentage of users who were able to successfully complete the tasks.
• Time-based efficiency: Also known as ‘time on task’, time-based efficiency measures how much time a user needs to complete a certain task.
• Number of errors: Sounds like what it is! It measures the average number of times where an error occurred per user when performing a given task.
• Post-task satisfaction: Measures a user's general impression or satisfaction after completing (or not completing) a given task.

‍

How to Collect Usability Metrics

‍
Usability metrics are outputs from research techniques deployed when conducting usability testing. Usability testing in web design, for example, involves assessing how a user interacts with the website by observing (and listening to) users completing defined tasks, such as purchasing a product or signing up for newsletters.

‍

Conducting usability testing and collecting usability metrics usually involves:

‍

• Defining a set of tasks that you want to test
• Recruitment of test participants
• Observing participants (remotely or in-person)
• Recording detailed observations
• Follow-up satisfaction survey or questionnaire

‍

Tools such Reframer are helpful in conducting usability tests remotely, and they enable live collaboration of multiple team members. It is extremely handy when trying to record and organize those insightful observations! Using paper prototypes is an inexpensive way to test usability early in the design process.

‍

The Importance of Usability Metrics in User-Centered Design

‍

User-centered design challenges designers to put user needs first. This means in order to deploy user-centered design, you need to understand your user. This is where usability testing and metrics add value to website and app performance; they provide direct, objective insight into user behavior, needs, and frustrations. If your user isn’t getting what they want or expect, they’ll simply leave and look elsewhere.

‍

Usability metrics identify which parts of your design aren’t hitting the mark. Recognizing where users might be having trouble completing certain actions, or where users are regularly making errors, are vital insights when implementing user-centered design. In short, user-centered design relies on data-driven user insight.

‍

But why hark on about usability metrics and user-centered design? Because at the heart of most successful businesses is a well-solved user problem. Take Spotify, for example, which solved the problem of dodgy, pirated digital files being so unreliable. People liked access to free digital music, but they had to battle viruses and fake files to get it. With Spotify, for a small monthly fee, or the cost of listening to a few ads, users have the best of both worlds. The same principle applies to user experience - identify recurring problems, then solve them.

‍

Best Practices for Using Usability Metrics

‍

Usability metrics should be analyzed by design teams of every size. However, there are some things to bear in mind when using usability metrics to inform design decisions:

‍

• Defining success: Usability metrics are only valuable if they are being measured against clearly defined benchmarks. Many tasks and processes are unique to each business, so use appropriate comparisons and targets; usually in the form of an ‘optimized’ user (a user with high efficiency).
• Real user metrics: Be sure to test with participants that represent your final user base. For example, there’s little point in testing your team, who will likely be intimately aware of your business structure, terminology, and internal workflows.
• Test early: Usability testing and subsequent usability metrics provide the most impact early on in the design process. This usually means testing an early prototype or even a paper prototype. Early testing helps to avoid any significant, unforeseen challenges that could be difficult to rewind in your information architecture.
• Regular testing: Usability metrics can change over time as user behavior and familiarity with digital products evolve. You should also test and analyze the usability of new feature releases on your website or app.

‍

Remember, data analysis is only as good as the data itself. Give yourself the best chance of designing exceptional user experiences by collecting, researching, and analyzing meaningful and accurate usability metrics.

‍

Conclusion

‍

Usability metrics are a guiding light when it comes to user experience. As the old saying goes, “you can’t manage what you can’t measure”. By including usability metrics in your design process, you invite direct user feedback into your product. This is ideal because we want to leave any assumptions or guesswork about user experience at the door.

‍

User-centered design inherently relies on constant user research. Usability metrics such as success rate, time-based efficiency, number of errors, and post-task satisfaction will highlight potential shortcomings in your design. Subsequently, they identify where improvements can be made, AND they lay down a benchmark to check whether any resulting updates addressed the issues.

‍

Ready to start collecting and analyzing usability metrics? Check out our guide to planning and running effective usability tests to get a head start!

Sitting inside any beautifully crafted and designed digital product, there must be a fully functional and considered information architecture.

‍

As much as information architecture shouldn’t be developed in a vacuum. Neither should the design and look of digital products. In fact, a large proportion of the function of digital designers is devoted to supporting users locating content they need and driving them towards content that the product owners want them to find.

‍

Incorporating visual markers to make sure that certain content is distinct from the rest or creating layers that demonstrate the diverse content on a product.

‍

If you do not have quality content, it is impossible to design a quality digital product. It all comes back to creating a user experience that makes sense and is designed to make task completion simple. And this relates back to designing the product with the content planned for it in mind.

‍

8 Principles of information architecture, according to Dan Brown 🏗️

‍

As a designer, the more you know about information architecture, the better the products you design will meet your user requirements and deliver what they need. If you work with an information architect, even better. If you’re still learning about information architecture the 8 Principles according to Dan Brown is a great place to begin.

‍

If you haven’t come across Dan Brown yet, you have more than likely come across his 8 principles. Dan Brown is one of the UX world's most prolific experts with a career that spans most areas of UX designs. He’s written 3 books on the subject and experience across a multitude of high profile projects. Aiding large organizations to make the most of their user experience.

‍

1. The principle of objects: Content should be treated as a living, breathing thing. It has lifecycles, behaviors, and attributes.
2. The principle of choices: Less is more. Keep the number of choices to a minimum.
3. The principle of disclosure: Show a preview of information that will help users understand what kind of information is hidden if they dig deeper.
4. The principle of examples: Show examples of content when describing the content of the categories.
5. The principle of front doors: Assume that at least 50% of users will use a different entry point than the home page.
6. The principle of multiple classifications: Offer users several different classification schemes to browse the site’s content.
7. The principle of focused navigation: Keep navigation simple and never mix different things.
8. The principle of growth: Assume that the content on the website will grow. Make sure the website is scalable.

‍

It’s highly likely that you’ve already used some, or all, of these IA principles in your designs. Don’t be shy about mastering them, or at the very least be familiar. They can only help you become a better user experience designer.

‍

Wrap up 🌯

‍

Mastering the 8 principles, according to IA expert Dan Brown will see you mastering the complex tasks of information architecture. Understanding IA is key to creating digital designs with a content structure that is functional, logical and just what your users need to navigate your product. Design without good IA doesn’t work as well, just as a content structure without a well designed interface will not engage users.

Recently, everyone in the design industry has been talking about design sprints. So, naturally, the team at Optimal Workshop wanted to see what all the fuss was about. I picked up a copy of The Sprint Book and suggested to the team that we try out the technique.

‍

In order to keep momentum, we identified a current problem and decided to run the sprint only two weeks later. The short notice was a bit of a challenge, but in the end we made it work. Here’s a run down of how things went, what worked, what didn’t, and lessons learned.

‍

A sprint is an intensive focused period of time to get a product or feature designed and tested with the goal of knowing whether or not the team should keep investing in the development of the idea. The idea needs to be either validated or not validated by the end of the sprint. In turn, this saves time and resource further down the track by being able to pivot early if the idea doesn’t float.

‍

If you’re following The Sprint Book you might have a structured 5 day plan that looks likes this:

• Day 1 - Understand: Discover the business opportunity, the audience, the competition, the value proposition and define metrics of success.
• Day 2 - Diverge: Explore, develop and iterate creative ways of solving the problem, regardless of feasibility.
• Day 3 - Converge: Identify ideas that fit the next product cycle and explore them in further detail through storyboarding.
• Day 4 - Prototype: Design and prepare prototype(s) that can be tested with people.
• Day 5 - Test: User testing with the product's primary target audience.

‍

When you’re running a design sprint, it’s important that you have the right people in the room. It’s all about focus and working fast; you need the right people around in order to do this and not have any blocks down the path. Team, stakeholder and expert buy-in is key — this is not a task just for a design team!After getting buy in and picking out the people who should be involved (developers, designers, product owner, customer success rep, marketing rep, user researcher), these were my next steps:

‍

Pre-sprint

1. Read the book
2. Panic
3. Send out invites
4. Write the agenda
5. Book a meeting room
6. Organize food and coffee
7. Get supplies (Post-its, paper, Sharpies, laptops, chargers, cameras)

‍

‍

The sprint

‍

Due to scheduling issues we had to split the sprint over the end of the week and weekend. Sprint guidelines suggest you hold it over Monday to Friday — this is a nice block of time but we had to do Thursday to Thursday, with the weekend off in between, which in turn worked really well. We are all self confessed introverts and, to be honest, the thought of spending five solid days workshopping was daunting. At about two days in, we were exhausted and went away for the weekend and came back on Monday feeling sociable and recharged again and ready to examine the work we’d done in the first two days with fresh eyes.

‍

Design sprint activities

‍

During our sprint we completed a range of different activities but here’s a list of some that worked well for us. You can find out more information about how to run most of these over at The Sprint Book website or checkout some great resources over at Design Sprint Kit.

‍

Lightning talks

‍

We kicked off our sprint by having each person give a quick 5-minute talk on one of these topics in the list below. This gave us all an overview of the whole project and since we each had to present, we in turn became the expert in that area and engaged with the topic (rather than just listening to one person deliver all the information).

‍

Our lightning talk topics included:

• Product history - where have we come from so the whole group has an understanding of who we are and why we’ve made the things we’ve made.
• Vision and business goals - (from the product owner or CEO) a look ahead not just of the tools we provide but where we want the business to go in the future.
• User feedback - what have users been saying so far about the idea we’ve chosen for our sprint. This information is collected by our User Research and Customer Success teams.
• Technical review - an overview of our tech and anything we should be aware of (or a look at possible available tech). This is a good chance to get an engineering lead in to share technical opportunities.
• Comparative research - what else is out there, how have other teams or products addressed this problem space?

‍

Empathy exercise

‍

I asked the sprinters to participate in an exercise so that we could gain empathy for those who are using our tools. The task was to pretend we were one of our customers who had to present a dendrogram to some of our team members who are not involved in product development or user research. In this frame of mind, we had to talk through how we might start to draw conclusions from the data presented to the stakeholders. We all gained more empathy for what it’s like to be a researcher trying to use the graphs in our tools to gain insights.

‍

How Might We

‍

In the beginning, it’s important to be open to all ideas. One way we did this was to phrase questions in the format: “How might we…” At this stage (day two) we weren’t trying to come up with solutions — we were trying to work out what problems there were to solve. ‘We’ is a reminder that this is a team effort, and ‘might’ reminds us that it’s just one suggestion that may or may not work (and that’s OK). These questions then get voted on and moved into a workshop for generating ideas (see Crazy 8s).Read a more detailed instructions on how to run a ‘How might we’ session on the Design Sprint Kit website.

‍

Crazy 8s

‍

This activity is a super quick-fire idea generation technique. The gist of it is that each person gets a piece of paper that has been folded 8 times and has 8 minutes to come up with eight ideas (really rough sketches). When time is up, it’s all pens down and the rest of the team gets to review each other's ideas.In our sprint, we gave each person Post-it notes, paper, and set the timer for 8 minutes. At the end of the activity, we put all the sketches on a wall (this is where the art gallery exercise comes in).

‍

‍

‍

Art gallery/Silent critique

‍

The art gallery is the place where all the sketches go. We give everyone dot stickers so they can vote and pull out key ideas from each sketch. This is done silently, as the ideas should be understood without needing explanation from the person who made them. At the end of it you’ve got a kind of heat map, and you can see the ideas that stand out the most. After this first round of voting, the authors of the sketches get to talk through their ideas, then another round of voting begins.

‍

‍

‍

Usability testing and validation

‍

The key part of a design sprint is validation. For one of our sprints we had two parts of our concept that needed validating. To test one part we conducted simple user tests with other members of Optimal Workshop (the feature was an internal tool). For the second part we needed to validate whether we had the data to continue with this project, so we had our data scientist run some numbers and predictions for us.

‍

‍

Challenges and outcomes

‍

One of our key team members, Rebecca, was working remotely during the sprint. To make things easier for her, we set up 2 cameras: one pointed to the whiteboard, the other was focused on the rest of the sprint team sitting at the table. Next to that, we set up a monitor so we could see Rebecca.

‍

Engaging in workshop activities is a lot harder when working remotely. Rebecca would get around this by completing the activities and take photos to send to us.

‍

‍

Lessons

‍

• Lightning talks are a great way to have each person contribute up front and feel invested in the process.
• Sprints are energy intensive. Make sure you’re in a good place with plenty of fresh air with comfortable chairs and a break out space. We like to split the five days up so that we get a weekend break.
• Give people plenty of notice to clear their schedules. Asking busy people to take five days from their schedule might not go down too well. Make sure they know why you’d like them there and what they should expect from the week. Send them an outline of the agenda. Ideally, have a chat in person and get them excited to be part of it.
• Invite the right people. It’s important that you get the right kind of people from different parts of the company involved in your sprint. The role they play in day-to-day work doesn’t matter too much for this. We’re all mainly using pens and paper and the more types of brains in the room the better. Looking back, what we really needed on our team was a customer support team member. They have the experience and knowledge about our customers that we don’t have.
• Choose the right sprint problem. The project we chose for our first sprint wasn’t really suited for a design sprint. We went in with a well defined problem and a suggested solution from the team instead of having a project that needed fresh ideas. This made the activities like ‘How Might We’ seem very redundant. The challenge we decided to tackle ended up being more of a data prototype (spreadsheets!). We used the week to validate assumptions around how we can better use data and how we can write a script to automate some internal processes. We got the prototype working and tested but due to the nature of the project we will have to run this experiment in the background for a few months before any building happens.

‍

Overall, this design sprint was a great team bonding experience and we felt pleased with what we achieved in such a short amount of time. Naturally, here at Optimal Workshop, we're experimenters at heart and we will keep exploring new ways to work across teams and find a good middle ground.

‍

Further reading

‍

• The Sprint Book - a book by Jake Knapp
• Design Sprint Kit — a whole bunch of presentation decks, and activities for sprints
• Sprint Stories — some case studies that show the processes and value of design sprints
• How to run a remote design sprint without going crazy - an article by Jake Knapp
• Choosing the right sprint problem - a blog by Lisa Jansen