Why information architecture is the foundation of UX

The term ‘usability’ captures sentiments of how usable, useful, enjoyable, and intuitive a website or app is perceived by users. By its very nature, usability is somewhat subjective. But what we’re really looking for when we talk about usability is how well a website can be used to achieve a specific task or goal. Using this definition we can analyze usability metrics (standard units of measurement) to understand how well user experience design is performing.

‍

Usability metrics provide helpful insights before and after any digital product is launched. They help us form a deeper understanding of how we can design with the user front of mind. This user-centered design approach is considered the best-practice in building effective information architecture and user experiences that help websites, apps, and software meet and exceed users' needs.

‍

In this article, we’ll highlight key usability metrics, how to measure and understand them, and how you can apply them to improve user experience.

‍

Understanding Usability Metrics

‍

Usability metrics aim to understand three core elements of usability, namely: effectiveness, efficiency, and satisfaction. A variety of research techniques offer designers an avenue for quantifying usability. Quantifying usability is key because we want to measure and understand it objectively, rather than making assumptions.

‍

Types of Usability Metrics
‍

There are a few key metrics that we can measure directly if we’re looking to quantify effectiveness, efficiency, and satisfaction. Here are four common examples:

‍

• Success rate: Also known as ‘completion rate’, success rate is the percentage of users who were able to successfully complete the tasks.
• Time-based efficiency: Also known as ‘time on task’, time-based efficiency measures how much time a user needs to complete a certain task.
• Number of errors: Sounds like what it is! It measures the average number of times where an error occurred per user when performing a given task.
• Post-task satisfaction: Measures a user's general impression or satisfaction after completing (or not completing) a given task.

‍

How to Collect Usability Metrics

‍
Usability metrics are outputs from research techniques deployed when conducting usability testing. Usability testing in web design, for example, involves assessing how a user interacts with the website by observing (and listening to) users completing defined tasks, such as purchasing a product or signing up for newsletters.

‍

Conducting usability testing and collecting usability metrics usually involves:

‍

• Defining a set of tasks that you want to test
• Recruitment of test participants
• Observing participants (remotely or in-person)
• Recording detailed observations
• Follow-up satisfaction survey or questionnaire

‍

Tools such Reframer are helpful in conducting usability tests remotely, and they enable live collaboration of multiple team members. It is extremely handy when trying to record and organize those insightful observations! Using paper prototypes is an inexpensive way to test usability early in the design process.

‍

The Importance of Usability Metrics in User-Centered Design

‍

User-centered design challenges designers to put user needs first. This means in order to deploy user-centered design, you need to understand your user. This is where usability testing and metrics add value to website and app performance; they provide direct, objective insight into user behavior, needs, and frustrations. If your user isn’t getting what they want or expect, they’ll simply leave and look elsewhere.

‍

Usability metrics identify which parts of your design aren’t hitting the mark. Recognizing where users might be having trouble completing certain actions, or where users are regularly making errors, are vital insights when implementing user-centered design. In short, user-centered design relies on data-driven user insight.

‍

But why hark on about usability metrics and user-centered design? Because at the heart of most successful businesses is a well-solved user problem. Take Spotify, for example, which solved the problem of dodgy, pirated digital files being so unreliable. People liked access to free digital music, but they had to battle viruses and fake files to get it. With Spotify, for a small monthly fee, or the cost of listening to a few ads, users have the best of both worlds. The same principle applies to user experience - identify recurring problems, then solve them.

‍

Best Practices for Using Usability Metrics

‍

Usability metrics should be analyzed by design teams of every size. However, there are some things to bear in mind when using usability metrics to inform design decisions:

‍

• Defining success: Usability metrics are only valuable if they are being measured against clearly defined benchmarks. Many tasks and processes are unique to each business, so use appropriate comparisons and targets; usually in the form of an ‘optimized’ user (a user with high efficiency).
• Real user metrics: Be sure to test with participants that represent your final user base. For example, there’s little point in testing your team, who will likely be intimately aware of your business structure, terminology, and internal workflows.
• Test early: Usability testing and subsequent usability metrics provide the most impact early on in the design process. This usually means testing an early prototype or even a paper prototype. Early testing helps to avoid any significant, unforeseen challenges that could be difficult to rewind in your information architecture.
• Regular testing: Usability metrics can change over time as user behavior and familiarity with digital products evolve. You should also test and analyze the usability of new feature releases on your website or app.

‍

Remember, data analysis is only as good as the data itself. Give yourself the best chance of designing exceptional user experiences by collecting, researching, and analyzing meaningful and accurate usability metrics.

‍

Conclusion

‍

Usability metrics are a guiding light when it comes to user experience. As the old saying goes, “you can’t manage what you can’t measure”. By including usability metrics in your design process, you invite direct user feedback into your product. This is ideal because we want to leave any assumptions or guesswork about user experience at the door.

‍

User-centered design inherently relies on constant user research. Usability metrics such as success rate, time-based efficiency, number of errors, and post-task satisfaction will highlight potential shortcomings in your design. Subsequently, they identify where improvements can be made, AND they lay down a benchmark to check whether any resulting updates addressed the issues.

‍

Ready to start collecting and analyzing usability metrics? Check out our guide to planning and running effective usability tests to get a head start!

“Questions are never indiscreet, answers sometimes are.”Oscar Wilde

‍

Frequently asked question pages. Love them or hate them, I don’t think they’re going anywhere anytime soon. This debate has been going on for quite some time and there is an equal number of opinions on both sides of the FAQ fence. Nielsen Norman Group’s Susan Farrell says FAQs can still add value to a website when done properly, and Gerry McGovern says FAQs are the dinosaurs of web navigation.

‍

So, how do we really know for sure if they will or won’t add value to a design? Like anything in UX, you have to test it! I don’t know about you, but I’m a shake-it-and-see-what-falls-out kind of UXer, so naturally I decided to run a Treejack study. Scouring the web one fine day, I came across Sainsbury’s Active Kids. Its FAQ page was unlike any I had ever seen and I knew I’d found the one. I was also curious to see how it would test with people who don’t use the website — after all, anyone should be able to use it. Since Active Kids is an active lifestyle program for UK schools and sports clubs, I recruited my participants entirely from the US.

‍

Pull up a chair and get comfy because what I found out should serve as a lesson to us all.

‍

Why Active Kids? 🤸🏼

‍

First of all, why did I choose this in the first place? The Active Kids FAQ page caught my attention for three main reasons:

‍

• structure
• labels
• content

‍

The structure of this FAQs page is quite deep, complex and very different from the rest of the site — almost like another information architecture (IA) had been built within the main structure. Imagine you have a large warehouse with hundreds of shelves, and then somewhere in the middle of it, someone builds a house — that’s how it felt to me.

‍

There are two ways to get to it: through the “Help” label on the top navigation bar and the “FAQ” label in the footer. It also uses a combination of drop-down filters that the user needs to apply, but it also has automatic filter options and confusing labels that can send you down a path you don’t necessarily want to take.

‍

I also found it very interesting that most of the information contained within the FAQs section cannot be located anywhere else on the website and most of this is essential to gaining a fundamental understanding of what Active Kids actually does. Adding to the house in the warehouse analogy, it’s like the house holds all the key information the warehouse needs to function, but no one knows which room it’s kept in.

‍

‍

‍

Setting up the study 🛠️

‍

Treejack was the perfect choice for testing the findability of information on the Active Kids FAQ page and I decided to test the IA of the website as a whole — this means both the warehouse and the house. I couldn’t just test the house in isolation because that’s not how a user would interact with it. The test needed the context of the whole site to gain an understanding of what’s going on. Creating a Treejack study is quick and easy and all you have to do is build the structure out in a basic Excel spreadsheet and then copy and paste it into Treejack.

‍

My next job was to determine the task based scenarios that my participants would use during the study. I decided to choose nine and all were derived from content located in the FAQs section and related to tasks a user might carry out when investigating or participating in the program. Once I had my tree and my tasks, all I had to do was set the correct answers based on where the information currently sits on the Active Kids website and I was ready to launch.

‍

Recruiting participants for the study🙋🏾

‍

In my experience,recruiting participants for a Treejack study is quick and easy. All you have to do is determine the screener criteria for your participants and Optimal Workshop takes care of the rest. For this study I requested 30 participants and they all had to reside in the US. I ended up with 31 completed responses and it was all over in less than two hours.

‍

Treejack results 🌲

‍

So, what fell out of that tree when I tested a website aimed at parents and teachers of kids in the UK with 31 Americans? I’ll be honest with you: it wasn’t pretty. Here’s what I discovered in this study:

‍

‍

• 81 per cent were unable to find out if home educators were eligible to apply (number 1 on the graph)
• 65 per cent were unable to find out what a Clubmark accreditation is (number 2 on the graph)
• 68 per cent were unable to find out how to share their wishlist with friends and family (number 3 on the graph)
• 64 per cent could not find the information that would explain the purpose of the £1 fee mentioned in the terms and conditions (number 4 on the graph)
• 97 per cent could not locate the information that would tell them if they could use a voucher from 2014 in 2015 (number 5 on the graph)
• No participant was able to determine if students from a middle school would be able to participate in Active Kids (number 8 on the graph)
• 58 per cent of participants in this study were unable to find out what the program is even about (number 9 on the graph)

‍

‍

On the flip side, 68 per cent of participants in this study were able to locate a phone number to contact Active Kids directly (number 6 on the graph) and 97 per cent were successfully able to work out how to redeem vouchers (number 7). Overall, it wasn’t great.In addition to some very useful quantitative data, Treejack also provides detailed information on the pathways followed by each participant.

‍

Understanding the journey they took is just as valuable as discovering how many found their way to the correct destination. This additional level of granularity will show you where and when your user is getting lost in your structure and where they went next. It’s also handy for spotting patterns (e.g., multiple participants navigating to the same incorrect response).

‍

I always set my studies to collect responses anonymously and when this occurs, Treejack assigns each participant a numerical identifier to help keep track of their experience without the participant having to share his or her personal details. For task 6, the paths chart below shows that participants numbered eight to 20 were able to navigate directly to the correct answer without deviating from the correct path I defined during setup.

‍

‍

‍

For Task 3 (below) , the story told by the paths was quite different. Participant number five navigated back and forth several times through the structure in their attempt to locate information on how to share a wishlist. After all that effort, they were unable to find the information they needed to complete the task and nominated to contact Active Kids directly. Not only is this a bad experience for the user but it also puts unnecessary pressure on the call centre because the information should be readily available on the website.

‍

‍

‍

Treejack also provides insights into where participants started their journey by recording first click data. Just like Chalkmark, this functionality will tell you if your users are starting out on the right foot from that all important first click.In this study I found it interesting that when looking for information regarding the eligibility of home educators in the Active Kids program, 42 per cent of participants clicked on “Schools & Groups” and 19 per cent clicked on “Parents & Community” for their first click. Only 6 per cent clicked on “Help”, which happens to be the only place this information can be found.

‍

‍

‍

‍

‍

I also found the first click results for Task 9 to be very interesting. When looking for basic information on the program, more than half (52 per cent) of the participants in this study went straight to “Help”. This indicates that, for these participants, none of the other options were going to provide them the information they needed.

‍

‍

‍

‍

What can be learned from this study? 🎓

‍

I mentioned earlier there was a lesson in this for everyone, and rather than dwell on how something tested, it’s time to move on to some lessons learned and constructive ideas for improvement. Based on the results of this Treejack study, here are my top three recommendations for improving the Active Kids website:

‍

Rethink the content housed in the FAQs section

‍

Most of the key information required to master the basics of what Active Kids is all about is housed entirely in the FAQs section. FAQs should not be the only place a user can find out basic information needed to understand the purpose of a product, program or service. I believe this website would benefit from some further thinking around what actually belongs in the FAQs section and what could be surfaced much higher.

‍

Another idea would be to follow the lead of the Government Digital Service and remove the FAQs section altogether — food for thought. Frequently asked questions would not be frequently asked questions if people could actually find the information on your site in the first place. Figure out where the answers to these questions really belong.

‍

If you’re using Treejack, just look at the fails in your results and figure out where people went first. Is there a trend? Is this the right place? Maybe think about putting the answer the user is looking for there instead.

‍

Restructure the FAQs section

‍

If you must have an FAQs section (and believe me I do understand that they don’t just disappear overnight! Just try to keep it as an interim solution only) please consider streamlining the way they are presented to the user. Ditch the filtering and display the list on one page only. Users should not have to drill down through several layers of content and then navigate through each category. For further reading on getting your FAQs straight, this Kissmetrics article is well worth a read.

‍

Review the intent of the website

‍

Looking at the Active Kids website and the results from this study, I feel the intent of this website could use some refining. If we come back to my warehouse and house analogy, the main chunk of the website (the warehouse) seems to be one giant advertisement, while the house (the FAQs) is where the action-oriented stuff lies. The house seems to hold the key information that people need to use the program and I think it could be displayed better. Don’t get me wrong, Active Kids does some incredibly good work for the community and should absolutely shout its achievements from the rooftops, however a sense of balance is required here. I think it’s time for the house and the warehouse to join forces into a solution that offers both rooftop shouting and usable information that facilitates participation.

‍

The value of fresh eyes 👀

‍

This study goes to show that regardless of where you are in your design process, whether that’s at the very beginning or a few years post-implementation, there is value to be gained from testing with a fresh set of eyes. I’m still undecided on which side of the FAQs debate I belong to — I’m going to sit on the fence and stand by the “if in doubt — test it” school of thought.

‍

Further reading:

• "Strategic design for frequently asked questions" - This ebook from Nielsen Norman Group explains how to design world-class FAQs.
• "Infrequently asked questions of FAQs" - An article from R. Stephen Gracey published on A List Apart, discussing FAQs and their place on the web.
• "Anatomy of a website: website architecture" - An article on our blog explaining website architecture and how to test your own structure.

In-person moderated user testing is a valuable part of any research project. Meaning you can see first-hand how your users interact with your prototypes and products. But in-person isn’t always a viable option. What to do if your project needs user testing but it’s just not possible to get in front of your users personally? 

‍

Let’s talk unmoderated user testing. This approach sidesteps the need to meet your participants face-to-face as it’s done entirely remotely, over the internet. By it’s very nature there are also considerable benefits to unmoderated user testing.

‍

What is unmoderated user testing? 💻👀

‍

In the most basic sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants complete the testing activity by themselves and in their own time. For the most part, everything else stays the same.

‍

The key differences are:

‍

• You can’t ask follow-up questions
• You can’t use low-fidelity prototypes
• You can’t support participants (beyond the initial instructions you send them).

‍

Is unmoderated user testing right for your research project?

‍

By nature, unmoderated user research does not include any direct interaction between the researcher and the study participants. This is really the biggest benefit and also the biggest drawback. 

‍

Benefits of unmoderated usability testing 👩🏻💻

‍

• Speed and turnaround  - As there is no need to schedule meetings with each participant, unmoderated testing is usually much faster to initiate and complete. Depending on the study, it may be possible to launch a study and receive results in just a few hours.

• Size of study (participant numbers) - Unmoderated user testing also allows you to collect feedback from dozens or even hundreds of users at the same time.

• Location (local and/or international) -Testing online removes reliance on participants being physically present for the testing which broadens the ability to make contact with participants within your country or across the globe. 

‍

If you’d like to know more about the benefits of unmoderated usability testing, take a look at our article five reasons you should consider unmoderated user testing.

‍

Limitations of unmoderated usability testing 🚧

‍

• Early-prototype testing is difficult without a moderator to explain and help participants recover from errors or limitations of the prototype.

• Participant behavior - Without a moderator, participants tend to be less engaged and behave less realistically in tasks that depend on imagination, decision-making, or emotional responses.

• Inability to ask follow-up questions - by not being in the testing with the participant, the facilitator can’t ask further questions to get a deeper understanding of the participant’s reasoning. As you can’t rely on human judgment through a moderator being in the room with the participants and the ability to adjust the test in the moment, unmoderated usability testing needs thorough up front planning.

‍

Because of these limitations unmoderated usability testing usually works best for evaluating live websites and apps or highly functional prototypes.  It’s great for testing activities that don’t require a lot of imagination or emotion from participants. Such as testing functionality or answering direct queries to do with your product.

‍

What’s involved when setting up unmoderated usability testing? 🤔💭

‍

1. Define testing goals

With any usability testing, it pays to define your goals before getting underway with setting up the software. What do you want to know from the participants? Goals vary from test to test. Understanding your goals upfront will help you to make the correct tool choice.

‍

2. Define your demographic

With a clear understanding of your goal, now it’s time to consider which participants are right for your study. Think about who they are, their demographic, and where they live. Are they new users or existing? Are they experts or novices?

‍

3. Selecting testing software

As unmoderated studies, are done remotely, the software used to faciliate the study plays a key role in ensuring you get useful results. Without a facilitator, the software must guide the participants through the session and record what happens. Take the time to test software and select one that is right for your study.

‍

4. Write your own tasks and questions

Think through your goals and what you want to achieve from the testing. Many of the unmoderated testing services include study templates with generic example tasks. Remember they are templates, and your tasks and questions should be specific to your particular study. Any task instructions guiding the participants should be clear and directive.

‍

5. Trial session

You’ve done all of the upfront work, now it’s time to test that it works, the software does what you expect and the instructions you have written can be followed. Doing a test run is crucial, especially with unmoderated usability testing, as there won’t be a facilitator in the testing to fix any problems.

‍

6. Recruit participants

Having defined your target audience and demographic, now is the time to recruit participants. Ensuring you have some control over the recruitment process is important, either through screening questions or recruiting your own. There are services that  recruit from a pool of willing participants. Thiscan be a great way to get a wide range of users.

‍

7. Analyze results

You are likely to accumulate a lot of data from your unmoderated testing. You’ll need a way to organize and analyze the data to derive insights that are valuable. Depending on the type of usability testing you do will vary the type of results. Quantitative testing gives data-driven results and direct answers. Whereas qualitative testing through audio or video recordings of participants’ actions or comments will need time to analyze and look at behavioral observations. 

‍

Wrap Up 🌯

‍

Unmoderated usability testing can be a good option for your study. It may not be right for all of your studies all of the time. While it can be quick to implement and often cheaper than moderated usability testing, it still requires time and planning to ensure you get the data insights you are looking for. Following a checklist can be a great way to ensure you approach your research methodically.

‍