Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Usability testing is a research technique aimed at evaluating how easy and intuitive something is to use. By testing early and often, you and your team can have confidence that what you’re designing and building makes sense to your users and helps them achieve their goals.
Usability testing involves observing participants as they complete a set of tasks. The goal is to reveal moments of frustration or confusion, identifying potential areas for improvement.
It can help you answer questions like:
You can choose to conduct usability testing in person with your participant in the same room as you, or remotely through video software with screen sharing functionality. There’s also unmoderated testing, in which your participant takes part in your test in their own time.
The benefits of conducting your test in person are you’re there with your participant to ask follow-up questions, pick up any non-verbal cues, as well as monitor their behavior.
Doing usability testing in this way gives you a good level of depth and quality of insights as you’re seeing the sessions being conducted live. This can also be done remotely where you can watch the participant live through a video feed. This is useful for when your participants aren’t in the same location as you or you don’t have the space or facilities to conduct the session in person.
This method is useful for when you need a fast and convenient way to get your research done. In most cases, participants will use dedicated software and will complete a series of tasks in their own time.
While it can be faster than traditional in-person usability testing, it doesn’t quite offer the same level of depth and insight, or give you the opportunity to follow up with your participant.
Test as early and frequently as you can. The further along in development you go, the harder and more costly it is to make changes. Work with your team to identify key issues to prioritize and retest in subsequent rounds to ensure the design makes sense to people before launch.
If you’re developing a new design or interaction, you can test this at different levels of fidelity – from rough sketches through to interactive prototypes, to developed features before release.
Use what you learn from each round of testing to improve your design, and then test it again. The more frequently you can iterate, the more confidence you’ll have in your design. Even in later stages of the development process, your team should be prepared to address larger usability problems and make relevant tweaks.
Usability testing should be an iterative process. Each round should be made up of several testing sessions, and it pays to do several rounds to get things right.
Conducting a usability test on an existing product or service helps you to assess the current solution and identify areas for improvement.
It also gives you an opportunity to benchmark the existing experience, making it easier to measure the impact of changes later down the track.
Once you have a new design in place, you can re-test it using the same tasks as your original test and see how things compare.
Reviewing the usability of your competitors is a great way to learn how your product or service compares to others in your market. It can also help glean insight into potential opportunities and gaps in your current offering and feed into strategic decision making.
Conducting usability tests on your competitors is no different to testing your own product. Identify a common set of tasks and scenarios to use during your study to easily compare the performance across different competitors.
The first and most important step is figuring out your research objectives with your team.
Perhaps your aim is to uncover current difficulties your users may have and identify opportunities for improvements? Or maybe you’d like to assess whether a current solution is meeting people’s needs?
Start by talking to your stakeholders to clarify the goals of your research and how findings will be used. If there are lots of objectives, prioritize them to ensure you’re focusing on the most important questions during your usability tests. Your stakeholders are people who have a vested interest in your project, for example, designers, product managers or clients.
Once you’ve narrowed down your research questions, it’s a good idea to give stakeholders the opportunity to provide feedback to ensure your research plan meets the needs of everyone involved.
Other things you need to do are:
Usability testing is a task-based activity. This means you need to figure out the flow of actions you want your participants to complete during your session. Tasks should be representative of typical goals your end-users would perform, such as purchasing a mobile phone on a website.
Scenarios help to guide the context in which the tasks take part. For example, finding the best mobile phone and mobile plan. Good scenarios are realistic and relatable. Make sure you’re not leading your participants by being too detailed and prescriptive.
Work with your team to identify the key tasks and scenarios you want to cover during your test. Be realistic around how much you can fit within a session and do a run through to make sure that your timing is right before booking in your research participants.
A script (or guide) is an outline of all the essential information to cover during your session.
It should include the key questions, scenarios and tasks you want your participant to complete, as well as any additional information you want to communicate to your participant, such as:
As a researcher, it’s your job to set clear expectations and make sure the participant feels comfortable. Ensure you cover off all essential information in your script, even if you do usability testing often.
Here’s a great example of a script from renowned usability testing expert Steve Krug.
Before your first session, you’ll want to consider the finer details to help you get your usability tests right:
Work with your team to find the answers to these questions before you conduct your first session to keep your study running smoothly.
When you recruit participants, it’s important to find people who are representative of your end users to ensure the tasks and scenarios make sense to them. For example, if you’re testing the usability of a new baby monitor, you’ll get more reliable results testing with soon-to-be or existing parents, rather than members of the general public who may or may not have children of their own.
You can recruit participants in a variety of different ways, and how you do so will depend on a few different factors such as your budget, company policy, resources and time.
Ask yourself the following questions before recruiting:
There’s no correct answer to these questions — it will depend on your company’s policies and how your company handles data. As a rule of thumb, only collect the essential information you need to get your research done.
The number of people you recruit will largely depend on the time and resources available to you, as well as the availability of your target participants.
Aim for a number that will allow you to start seeing patterns in your data. Avoid making any decisions based on what one person said or did, unless they’ve come across a bug or usability issue that blocks people from completing a task successfully.
Try and include at least five participants in each round of testing. Add more if you continue to discover issues that are unique to each participant, until you start to see patterns emerge.
If you’re working with personas and have several different types of tasks that each user type may perform, you’ll want to recruit five participants from each user group to get reliable results.
If you have access to a pool of participants (like employees if you’re working on an internal product, or your customer mailing list) then reaching out to potential participants via email can be an easy way to source them.
Similarly, you can recruit participants by advertising on social media channels or by running banners or pop ups on your website.
When sourcing participants for usability tests, you may want to direct them to a screener, depending on your needs. This is to ensure you’re reaching people who meet the specific criteria you’re looking for. What you decide to include in your screener will vary depending on your objectives.
Some useful things to include may be demographic information, previous experiences using your product or service, frequency of use, and anything else that will help you match your candidates against any target-user characteristics.
You can also make use of recruitment providers, which can be a good option if you don’t have easy access to the people you’d like to target. Dedicated recruiters will generally have access to a wide variety of potential participants, which can make this a fast and easy way to source relevant people with minimal effort.
In some cases, they’ll also oversee logistics, such as scheduling sessions and paying out incentives on your behalf. Keep in mind that recruiters and panels come at a higher price than sourcing participants on your own.
Make sure you provide your recruiter with a clear understanding of the participant characteristics you’re looking for – similar to a screening questionnaire.
An alternative to using recruiting agencies is sourcing participants on your own from online panels. These self-service panels contain databases of potential participants, allowing you to filter, match and submit orders based on your own criteria without relying on a dedicated recruiter.
To encourage potential participants to take part in your research, you’ll want to consider how you incentivize them. While you can get away with little to no incentive for internal participants (for example, existing users within your organization), external participants may be less obligated to participate if they’re not reimbursed for their time.
Incentives can be monetary in the form of cash or gift cards, or take other forms, such as company merchandise or free access to your product or service.
The amount you decide to pay your participants will depend on a few different factors. Consider the length of the session, the complexity of the tasks and whether you need participants with specific experience or skills. For example, if you’re testing a specialist medical interface with doctors, the incentive you offer should be higher than if you’re testing a banking app that has a wider range of users.
An incentive should be high enough to motivate people to participate in your study. A good starting point is to consider how much a participant would usually receive for an hour of their time and add in extra to cover the additional effort required for participating.
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Set up your study in advance so that you’re ready to go for your first usability testing session.
You can organize your usability testing sessions and the observations that you capture for each session within a Reframer study.
When you create a new study, you’ll be able to add study objectives and reference links in the overview screen. Use this space to outline the purpose of your research and any other important details.
This is also a place to keep links to prototypes, wireframes and other documents so your team has everything they need on the day of your usability test.
Keep all the essential information relating to your test on the overview page.
Once you have your first participants scheduled, you can start to create sessions in Reframer’s Sessions tab. Use the ‘Session information’ field to add any relevant information about your participant during or after your session, such as demographics, their experience using your product, or any other details worth noting.
This is also where you add Segments to each of your sessions. Segments are useful for identifying participant characteristics such as demographics, personas or user types. They come in handy when you’re analyzing your results and wanting to identify patterns in your data across different groups.
Any helpful information about your participant and the session can be added to the session overview.
Tags allow you to code your observations, helping you identify and organize relevant findings later on.
In the Tags tab, you can create and manage the tags you intend to use for your study. The tab will display all the tags used by you and your team.
Create tags that will make it easy for you to filter and make sense of your data when you move on to the next stage – analysis. For usability testing, you’ll want to create a set of tags dedicated to the tasks, flows or screens you’ll be testing, as well as tags that will help you identify any problems or issues your participants encounter. This could be as simple as a ‘pass’ or ‘fail’ for each of your tasks, or a group of tags that relate to sentiment, such as ‘delighted’, ‘frustrated’ or ‘surprised’.
Whether you defined tags upfront or not, you can still change or create new tags any time.
Once you move on to analysis, you’ll easily be able to filter your data by task and see at a glance where your participants may have run into issues. This will help you to identify potential improvements.
Tag groups can help you keep your tags organized.
Consider the environment in which your usability tests will take place so you have what you need for your first session.
Some things to ask yourself:
Getting team members and stakeholders involved in your research sessions is a great way to expose them to the people actually using your products and services, and makes for a great empathy building exercise.
It also helps to get people on board with the research findings — there’s no better evidence than seeing it for themselves.
Consider including members of your immediate team, such as developers, designers, product owners, as well as those from other areas of the business, such as marketing, support, management.
When to get others involved
Getting other people involved in your research can be done in a myriad of ways and can depend on how your team or company is run. Some ways to include others in your research:
How many people to involve
While you want to get as many people involved in your research as possible, try to limit the number of people in the same room as your participant when running in-person sessions. This will create a much more relaxed environment for your participant. Keep the number down to the essentials: the facilitator and a notetaker or observer.
If you’d like to invite others to observe the session, consider setting up an observer room with a live feed from the test room. It’s also a good idea to record your sessions so you can share them with those who are unable to attend on the day.
If you’ve recruited some coworkers or stakeholders to take part in your sessions, you’ll want to brief them to ensure they’re clear on what they can expect. This is crucial if they have never been part of a usability test before. It’s important to explain what a usability test is, how they’re run, what the value is and what sort of findings it will generate.
People you involve in your study could have a few different roles, such as observing, notetaking, asking questions or even being involved with the analysis later on.
No matter who you’re involving or what their roles are, go over the following details:
Tips for notetakers
Notetakers have an important job that requires a lot of focus. Make sure you teach your notetakers how to take good quality notes and what to listen and look for during the session. Let your notetakers know what happens to their notes after the session and how you’ll be using them.
If there are any tools you’ll be using during the test, let your notetakers know in advance so they can become familiar with them ahead of time.
For more help with taking effective observations, read this article on UX Mastery’s blog.
Tips for observers
While observing is the easiest way to involve a large number of people, it’s important to ensure everyone stays engaged so they can understand the value. Encourage observers (especially key stakeholders) to write down their top insights following each session, and invite them to participate in a debrief afterwards to identify patterns among everyone involved.
Inviting notetakers or team members in Reframer
In Reframer, you can invite other people to your study as notetakers. If the person you want to invite already has a license to your Optimal Workshop account, you can simply invite them to the study in the ‘Study members’ tab. They can then take notes, tag and help analyze.
If they haven’t got a license, you can invite them to your study as a guest notetaker. They’ll be able to take notes and create or add tags to their own observations, however they won’t be able to edit your study or take part in analysis. Also, they will only have access to the specific study you invite them to.
Make sure you invite your notetakers before the session starts, so that they’re ready to go when you meet your participants.
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Ideally, your usability testing sessions will involve:
Occasionally, you may have to play the role of the facilitator and notetaker at the same time. This can be the case if you’re running testing sessions on your own, or others are unable to make it at the scheduled time. If this is the case, stay focussed on your participant by recording your session and writing up observations afterwards.
At a basic level, the role of the facilitator is to follow the testing script and keep the session on track. This includes introducing others involved, explaining the purpose of the session, keeping the participant comfortable and guiding them as they complete a series of tasks.
In reality, facilitating a usability testing session involves a range of questioning and probing to get to the core of what the participant is thinking and feeling as they interact with the design.
As a facilitator, you should continually encourage your participant to think out loud, and address any hesitations and pauses. Non-verbal cues (such as facial expressions) can provide a lot of insight into what’s going on inside a person’s head, allowing you to reach deeper insights. Make sure you’re not influencing participants in any way by leading them through the test, asking loaded questions or suggesting answers.
While others observing the usability test may have questions they want to ask throughout the session, it’s up to the facilitator to decide on the best time to follow up with any subsequent questions.
It can be tempting for participants to ask the facilitator questions to verify they’re on the right track. To avoid this, it’s best to let the participant know upfront that you may defer answering any questions until the end to avoid influencing their actions.
Capturing observations as your session is taking place ensures you don’t miss the essential details as they arise. It also takes some of the load off after your session is complete, allowing you to quickly identify key findings.
Observations refer to any relevant things you see or hear during your usability testing sessions. In Reframer, you’ll capture your qualitative data in the form of individual observations.
Observations are things that you see or hear during your usability test.
If you have a dedicated notetaker, they can focus on taking notes while you facilitate the session.
If you’re on your own and don’t have a dedicated notetaker, it’s best to record your session and stay focused on your participant. You can listen back and take notes later.
When capturing observations, it’s important to focus not only on what your participant said but also what they did. There is often a difference between what people say they do, and what they actually end up doing. Capturing behavioral information can give your data an additional level of insight.
The role of a notetaker is not only to ensure key information is being recorded, but also to observe the session and make note of any behaviors and nonverbal cues (pauses, hesitations, looks of frustration or confusion and so on).
Take note of the important details — especially nonverbal cues.
Sometimes little things can have a big impact, so be sure to include smaller details in your observations. For example, a participant using an operating system they’re not familiar with may affect their ability to perform tasks.
If you’re a notetaker you can capture your observations directly in Reframer while your session is taking place. Observations can also be added after the session is complete (for example, from a recording). You can also import existing data into Reframer from elsewhere.
Capturing observations directly into Reframer is as easy as writing a note and hitting enter. The observation you’ve written will automatically save and appear in your list of observations. Each of your sessions can have an unlimited number of observations.
You may choose to capture your observations verbatim, or only make note of key information throughout your session. Including quotes is a great way to bring your observations to life.
To help focus on your participant during your session, you may want to refrain from tagging until after your session is over.
You can also:
Once your session is complete, it’s a good idea to conduct a debrief, allowing the team to come together and pull out key insights from what everyone saw and heard during the session.
Host this meeting as soon as possible after the session has finished so that the information is still fresh in your minds — booking in time immediately after each session is ideal!
If you had multiple notetakers and observers, encourage each person to list their most significant outtakes from the session to find patterns among what was observed across the team.
The debrief is an opportunity for the team to look through the session and review relevant findings in Reframer. Use the filter to bring up any starred observations so the team can discuss them together, such as why they were starred or if anything in particular made these observations stand out. You can even use this time together to tidy up some of the data and get rid of duplicates and errors before moving on to analysis.
In some circumstances, you may choose to record your sessions and make notes by reviewing the recordings.
You can use Reframer to make notes from your video or voice recordings in the same way a notetaker would capture observations during live in-person sessions.
This can be the case if you don’t have a dedicated notetaker, in which making notes on your own during the session could be disruptive.
If your notetaker has used a different method for taking notes other than Reframer, you can import this information directly into your individual sessions. If you have information on the time your observations were taken, you can include timestamps for individual observations.
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Focusing on your participant while capturing observations in real-time can be a challenge. The review tab allows you to check over what you’ve captured at the end of your session and tidy things up.
Take the time to get your data in good shape for analysis — fix any typos or incomplete observations, remove accidental duplicates and get rid of incorrectly used tags. This is also a good time to recall any significant observations or tag relevant findings.
The review stage gives you the opportunity to immerse yourself in the data from your session. Filtering comes in handy when you want to bring up all the observations relating to a particular keyword, making it easier to add tags to multiple observations at once.
For example, you can search for words such as ‘confusing’, ‘hard’ and ‘unsure’ to quickly identify places where your participant experienced issues with your design, or search for a particular moment in your session based on what your participant had said.
Here you can also view any starred observations that may have stood out to you or anyone else during your session.
You can also use the search box to filter observations by tag or author. This allows you to quickly see how a particular tag was used during your session, or review observations made by others in your team.
Tagging your observations helps you make sense of your data once you move on to analysis. There, you’ll be able to filter your observations based on tags and combine these to uncover themes. For example, observations tagged with #confusion and #check_out may suggest issues with that particular feature.
Use tags to link observations back to your research objectives, identify sentiment or help to mark any particularly significant findings.
You can add tags directly to your observations using hashtags (for example, #frustration), or using the tag list in the sidebar on the right. You can also multi-select your observations and add several tags at once.
The star feature allows you to quickly highlight key observations from your session, making them stand out from the rest of your findings.
Stars function in a similar way to tags, making it easy to filter key findings, quotes, or notes of interest when you begin to analyze your results across all your sessions.
Use this feature to mark any observations that are particularly significant, or those that provide great quotes or outtakes from your session.
Affinity mapping is a flexible and visual way to quickly group, organize and make sense of your data. Think in-person post-its and whiteboards, or digital whiteboard tools.
All your observations are stored post-session, you can group similar ones in the affinity map and give them labels, and easily see your tags and titles of your groups, or ‘themes’.
When you first land on the affinity map, the canvas will be blank and all your observations and their metadata (tags, segments) will be in the list on the left hand side.
How to read, edit and group observations
Your observations are shown in detail on the card list, allowing you to read and review all your observations one-by-one. As you go through them, you can move them from the card list to the board to form groups.
To edit an observation, simply click into it and make changes.
To make sense of your data, move observations that relate to a similar concept into the same group. This will create a visual representation of different themes within your data.
You don’t have to get everything perfect in one go. Affinity mapping is a very flexible method, and we encourage you to go through the grouping and refinement in iterations.
You can start by creating a few groups first, and then splitting them or combining a few of them later. However, tidying up the groups by adding titles and aligning them neatly as you go makes it easier for you to revisit your groups further down the track.
Use search and filter to narrow down relevant observations
Use the search functionality to quickly narrow down observations based on keywords and use filters to help you look at small sets of data. You can filter by segments, research questions/tasks, participants, or tags.
When filters are applied:
Collaborating with team members
The affinity map was built to support collaborative analysis. Your team members can edit, group, label and comment on all observations alongside you in real-time.
When there are multiple people in the affinity map, you’ll each see where the other’s cursor is and if you’re working on a particular group or observation, it will be locked until you finish with it, ensuring that there’s no double ups.
What is the table view?
Once you’ve grouped your observations on the affinity map, you can view them all on the Affinity map groups table to get a clear view of all the groups and sort them by relevant stats. The numbers under the ‘Sessions’, ‘Observations’ and ‘Tags’ columns help you quickly gauge how significant each group is, based on the frequency of unique sessions referenced, as well as total observations included.
You can click on an observation group to view more details, including the observations, sessions (usually the participants), tags and tasks that are associated with this group.
This makes it really easy for you to review the groups, and decide if the title reflects the observations they contain. It also makes it faster for you to retrieve an observation or quote related to a group.
Ordering groups and using filters
Affinity map groups are listed alphabetically by default, if you want to see how frequent a group was referenced by different participants, you can order the list of groups with the number of sessions/participants. Simply click on the arrows next to the Sessions label.
You can also sort them by other associated metadata, such as the number of observations, and number of tags as well.
You can use filters to narrow down groups to find the ones that are associated with certain attributes, such as a specific tag or segment.
The Theme builder tab enables you to explore relationships between your tagged observations, and then create themes based on these relationships. The data in the theme builder updates automatically whenever you create a session, add an observation or add a tag within that study.
However, you’ll get the most out of exploring trends in your data once you’ve tagged a substantial number of observations, so we suggest doing this first.
To get the most use out of the theme builder, it’s best to start by exploring themes that relate to your key research objectives. You may want to look at your data from the perspective of:
When exploring the relationships between different tags, pay attention to tags that have a strong relationship, as well as those that don’t. These can help you validate any assumptions you had prior to your research. For example, you may have had concerns that a new navigation will confuse users and find that, in fact, no one encountered any issues finding what they needed.
When your tag combinations start to reveal interesting patterns in your data, you can save these as a theme. Themes group together observations that relate to a similar topic. You can name your theme and add a description for additional context and help other team members understand what you’ve discovered.
There are 4 layers of filters you can use to ‘ask questions’ and spot patterns in your observations: session name, segments, tags, and tasks if you’ve set them up. You can use one or more of these types depending on what you are looking to discover.
Sessions lets you exclude entire sessions from the analysis. Got data from a participant that you’re not confident about? Leave it out for now until you can follow up.
Segments makes it simple to include or exclude multiple sessions based on the participant segment you created earlier. Only want to see data from people who matched a specific persona? You can do that here.
Tags help you spot connections across all your included sessions and segments, and is where the real power of Reframer lies. As an example, you might spot lots of observations that are tagged with both ‘frustrated’ and ‘login’, hinting that there’s a potential issue with your login screen.
Tasks lets you see all the observations that were captured during your pre-set usability testing tasks. For example, if you wanted to see how all your participants fared during Task 2, you can filter your observations by Task 2 only.
Select a tag in the Theme Builder to display all related observations on the right. As soon as you select a tag, you can create a theme by clicking ‘Save this theme’ and naming it.
In our example study, we’ve selected the tag ‘frustration’ because it’s one of the most common tags in the study so far. Now, only the 8 observations tagged with ‘frustration’ are shown in the right hand column.
We could create a theme with just this tag at this stage, or we can leave it unnamed for now and explore a few more relationships before committing.
The tag you’ve selected will influence part of what the Theme Builder shows you next. The list of tags will still display the number of observations each tag appears on, but it’ll now also display the number of observations each tag has in common with your selected tag.
You can refine your search by combining multiple tags. Once you have existing filters applied, the numbers next to the other tags will now indicate how many observations you might see if you combine them.
► indicates how many observations this tag may have in common with your existing filters, i.e. narrowing with this tag.
◀ indicates how many additional observations this tag will add to your existing filters, i.e. broadening with this tag.
The chord diagram is a visual way to easily explore the relationships between your tagged observations and find key themes. You can choose to view all your tags, or exclude the ones you may not be interested in exploring right then and there.
The core of your Reframer analysis lies in the theme builder, which allows you to create themes by grouping observations that all relate to a similar topic or finding.
Using the theme builder, you can filter your observations by combinations of sessions and tags. When you select a tag, you’ll see all observations in your study that have been assigned that specific tag. From here, you can see any other tags that were used in combination with the one you selected, and broaden or narrow your selection to reveal relationships between them.
When you narrow your tag selection you can reveal observations that contain the specific tags you selected (for example, ‘T1_Login’ AND ‘delight’).
Once you’ve identified the key themes from your research, it’s time to share your findings with others.
There’s no right or wrong way of sharing your findings with the rest of your team. While in some cases a simple session with your team and stakeholders to run through the themes you identified will suffice, in other cases you may have to compile supporting evidence such as quotes, audio or video clips to get others onboard.
The most important thing about usability testing is making sure the most significant findings are understood, prioritized and actioned.
Remember — usability testing is an iterative process. Each round should lead to improvements, and with time, lead to more confidence in you and your team, and a better experience for your users.