Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Interviews are an exploratory research technique aimed at uncovering people’s thoughts, behaviors, emotions and motivations. When done well, they can be an important source of insight for you and your team, allowing you to create better products and services for your users.
User interviews can be conducted one-on-one with your participant and any number of observers, and as semi or fully-structured sessions that follow a set of predetermined questions.
The goal is to understand your users on a deeper level, leading to more informed decisions. They can help you answer questions such as:
Conducting user interviews can act as support for:
Reframer is a qualitative research and analysis tool built specifically for user interviews.
It helps you:
Reframer streamlines the workflow of planning, running and analyzing a user interview. Here are a few tips to help you set up your Reframer study for success.
Before you jump straight into your research, we recommend taking the time to form a clear understanding of the goal of the research as well as the research objectives.
The goal is usually tied to the business goal for the overall work. So if the business goal is to increase conversion for the submission process by x%, then the research may be about identifying the mental models of users so you can organize the steps logically, or the goal might be to identify the overall success of the current and new structure.
Once you’ve decided on a goal, you need to set your objectives. These are the big questions you are hoping to get answered in the research. Writing them down can help you and the team get on the same page.
Objectives can be written as questions: Where are people dropping out of the conversion funnel?
But equally can be written as hypotheses if you have an inkling about what is happening: Users are unable to complete the form because we are asking for passport or other ID information that they are uncomfortable providing.
You want to aim for around 4-6 objectives. Any more than that and it’s hard to address them all in a single piece of research. For smaller projects, the goal and the objective are the same thing.
Ultimately, interviewing is the art of learning about another human through conversation.
However, in a user interview situation, you can’t directly ask participants “what’s wrong?” Getting honest, insightful answers requires strategic questions. But don’t worry, this is a skill that can be learnt with practice.
The key is ensuring your participants feel comfortable to share their experiences and honest opinions with you. Most of the time, even if your participants want to be honest with you, they aren’t always conscious of their own experiences and opinions of the subject matter you’re researching. The way you frame your questions or discussion guide needs to help them reflect and articulate their thoughts and feelings.
Creating your discussion guide
So what exactly are you going to ask your participants about? You’ve got an allotted time period with your participant and you want to make the most of it. Your discussion guide helps to provide structure and reference for your interview, acting as a reminder of your questions as you take notes during your session.
When you write your guide, the first thing to do is put your goal and objectives in front of you and think about how to address the questions within those.
Here are some key things to include in your discussion guide:
Using the Discussion Guide in Reframer
Once you’ve identified your interview questions, you can enter them into the Discussion Guide in Reframer.
When conducting your sessions, you’ll see the discussion guide in the ‘Capture’ tab. By moving through the discussion guide, your notes will be saved under the relevant questions, providing context for individual observations and making analysis easier.
Depending on how your sessions pan out, you might find it feels more natural to keep the order of your interview questions flexible and navigate through the topics as they crop up in conversation, instead of sticking to a rigid list.
This means you might want to keep the questions you write fairly high-level and use them as a guide rather than a script. Feel free to jump around the discussion guide, you can always tag your observations to specific questions or tasks further down the track.
Your participants should reflect the type of customer or audience you want to reach. For example, if you want insight into how doctors communicate and store patient data, you’d want to talk to medical professionals instead of members of the general public.
If you’re working with personas, interview different people from these personas to get multiple points of view and a wider range of information. This will also help you avoid making any decisions based on what one person said.
Use segments to reflect the cohorts or meaningful traits your participants have that are similar or different from each other. For example, demography, experience level, job title, etc.
Once you know the type of participants you want for your sessions, there are a few things to consider before you start recruiting:
When you start the recruitment process, you want to make sure that the potential participants you’re getting are the ones you want. It’s a good idea to create a screener survey that captures relevant information and allows you to ask specific qualification and segmentation questions.
You can do this easily using Questions by Optimal Workshop. Find out more about how to write a good screener survey here.
Once you have a pool of participants with relevant segmentation data, you can pick and choose the participants best suited to your research.
Set up sessions with your participants
Once you’ve booked in your participants, it’s time to set up sessions for each one in Reframer.
We recommend filling out the participant information in Reframer for each session to provide context about the participant for you and your notetakers. Use session information for any context or links that might be helpful for you or your notetakers and use segments for information on demographics, personas or user types.
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
You’ve done your prep, now it’s time to conduct your interview sessions!
Here are some tips on how to get the most out of your sessions:
User interviews are a meaningful and rewarding way to connect with and hear directly from your participants, not only for you as the researcher, but also for anyone who’s in the room observing or taking notes.
We tend to encourage researchers to bring on their team or stakeholders for note taking. This not only helps researchers capture data more effectively, it’s also a practical way to help your team develop empathy for your end-users, get buy-in from your stakeholders about the research and your findings, and build shared understanding with first-hand experience.
Prep yourself and your team
It’s always a good idea to do some prep work before you bring your team into an interview session.
We suggest holding a briefing session with your team members or notetakers and cover the following topics:
Inviting notetakers in Reframer
In Reframer, you can invite other people to your study as notetakers. If the person you want to invite already has a license to your Optimal Workshop account, you can simply invite them to the study in the ‘Study members’ tab. They can then take notes, tag and help analyze.
If they haven’t got a license, you can invite them to your study as a guest notetaker. They’ll be able to take notes and create or add tags to their own observations, however they won’t be able to edit your study or take part in analysis. Also, they will only have access to the specific study you invite them to.
Make sure you invite your notetakers before the session starts, so that they’re ready to go when you meet your participants.
Reframer’s ‘Capture’ feature is built for taking notes easily and effectively. There are many different ways to utilize it:
When you’re conducting an interview session, it’s crucial that you engage with your participant. You want to make sure you ask relevant questions and do so effectively, build rapport with your participant, and help them feel comfortable articulating their thoughts and feelings.
To do this successfully, we recommend either having a dedicated notetaker or recording your session and taking notes afterwards. This way you can give the participant your full presence and attention, as well as alleviate the pressure of having to take notes and moderate
Tips for effective note taking
Taking good notes takes a bit of practice. Here are a few tips on how to do it effectively for better analysis and record keeping:
Taking notes in Reframer
Reframer is designed for you to capture and tag your observations quickly as you move through your session. It encourages you to capture everything you see and hear.
Here’s how to capture observations efficiently:
Capturing observations: Type in your observation and hit ‘Enter’ to save it. Or you can hit ‘Shift+Enter’ to start a new line in the same observation. Keep your observations succinct; don’t use ‘Shift-Enter’ too much or it will make subsequent analysis harder.
Adding observations to a question or topic: Reframer’s discussion guide has a function called ‘tasks’ which, when conducting usability tests, allows you to pre-fill the tasks so that you can associate the participant’s responses with the task they’ve been assigned.
You can also use this function when conducting interviews, but instead of there being specific tasks the participant is being set, there will be key questions or topics that your questions cover that you can associate with the participants responses.
If you have pre-filled the questions or topics in Reframer’s tasks, you will see tasks in the discussion guide in the sidebar on the right. You can either click ‘Start tasks’ and add observations to each task as you move through, or click on the relevant task if you’re not moving through the guide linearly.
If you end up with any observations that aren’t assigned to a task, don’t worry, you can go through and do this later.
Tagging observations: If you’re taking notes, we recommend tagging your observations afterwards in the ‘Review and tag’ mode so you can focus wholly on note taking during the session.
However, sometimes you might want to quickly add tags for later reference while capturing observations.
To create tags during observations, place hashtags in front of keywords (for example, #focus). You can also create a ‘quote tag’ (indicating that this is something that the participant said, verbatim) automatically by adding speech marks to the observation (for example, “It’s hard working remotely”).
These tags will appear under the observation when you save it and will be added to the tag management area, giving you a complete picture of all the tags used by you and your team.
For a more in depth look at how to add and manage tags, check out our Help Center article.
Highlighting important observations: Highlight any important observations using the star icon. This is useful for identifying observations that may be particularly relevant, or quickly identifying observations you want to review later on.
To do this, simply click the star icon in the top right hand corner when capturing the observation.
Moderating an interview is essentially the “interviewing” itself and as you conduct more sessions, you’ll get a feel for what works for you and your participants. Perhaps for your first few interviews you feel more confident sticking to your prepared discussion guide or questions. But as you continue to do more, you’ll find it easier to naturally guide the conversation in different directions as you pick up on things the participant says.
Remember though, when you created your research objectives, that there were certain key notes you had to strike, or issues you had to tackle, otherwise your interview will be a ‘nice chat’ that doesn’t give you the facts and insights you are looking for.
Here are some tips to get you started with moderating a session:
For more tips on how to successfully moderate interview sessions, take a look at the following resources:
You’ve got through your first user interview, nice one! The work isn’t quite done yet, though.
A very important, but oftentimes ignored, part of user interviews is debriefing with your notetakers and observers immediately after the session. During the debrief, your observers and notetakers can share what they’ve seen or learned from the session while the context is fresh. This is helpful for two reasons:
Keep in mind though, some of the highlights that people share in a debrief session could be considered more as opinions and interpretations than actual observations. Keep a note of these as there could be valuable insights, but also make sure you can differentiate between these opinions or interpretations and observations further down the track.
If you don’t have notetakers then it’s equally valuable to have a quiet moment to reflect on what you just observed by yourself.
Here are some debrief questions we recommend:
You may want to consider a very simple template to capture these debrief moments. One way to do this with Reframer is to add a final ‘task(s)’ onto the guide that allows you to capture your debrief. Be sure to tag the comments here to ensure they don’t get confused with actual participant comments.
It’s also good practice for you to ask for any feedback on your moderation as well. Having your team members make suggestions on anything you can improve in the session (i.e. how you asked the questions or set up the environment) will help you become a better moderator.
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Once you’ve completed all your sessions, it’s time to start your analysis. Analyzing your qualitative data will give you the key insights you need to create better products and services for your users.
The analysis process is like distillation: you start with cleaning up and breaking down your raw materials, then refine the materials in a few rounds which gradually makes the materials more condensed, until you’re ready to distill them.
We know different researchers have their preferred ways of making sense of their data, which is why we built Reframer to accommodate the two main approaches to analysis.
For quick and easy analysis, jump into the affinity map, or for a more robust analysis of all the emerging trends and concepts from your data, tag them and do a thematic analysis and visualization with the theme builder. Or better yet, use both of these approaches together to really get down to the nitty gritty of your data.
The first step is to sift through your data and prepare it for analysis. In other words, read through all your observations and use tags to describe the key concepts in each observation. You’ll do this in the ‘Review and tag’ tab in Reframer.
This step can feel overwhelming and a bit tedious, but it’s absolutely crucial.
As you revisit the discussions and observations from each session, you’ll get a more coherent sense of the patterns emerging from your data. It also gives you the chance to read through any parts of the session your notetakers have observed that you may have missed while moderating.
Plus, tagging has the additional benefit of being able to quantify your data later on – it is a valuable step in turning your notes into themes and eventually into insights.
Editing observations
When you’re going through your observations, it’s a good time to fix up any typos or add any additional data that you may not have had time to include during your session. You can edit your observations by clicking directly into the observation. You can also delete an observation if it isn’t useful, or star it if it’s particularly significant.
Consolidating repeating observations
If you come across observations that make a similar point, you can consolidate them to save some analysis time and reduce false positives.
You can use keyword search to quickly find observations with similar wording and decide if some of them are repeats. To consolidate two observations, you can copy the text from one observation into another, and delete the duplicates.
If you want to quantify your qualitative data and identify the recurring patterns, concepts, topics or themes that appear from your participants, it’s vital to tag (or “code”) your data. Tagging is especially useful if you are doing a more robust analysis process along the lines of thematic analysis.
Tag one or more observations
Adding tags to your observations after your sessions will give you the time and flexibility to ensure they’re as relevant and useful as possible. The more accurate and refined your tags are, the more meaningful your results will be.
You can tag individual observations by selecting them and clicking on the relevant tags, or multi-select and tag several observations at once.
You can also remove a tag from an observation by clicking the ‘x’ icon to the right of the tag’s name.
Assigning a task to an observation
If your discussion guide has tasks (i.e. predefined questions or question topics) added to it, you can assign or reassign any participant observation to that task.
Assigning tasks works the same way as tagging. You can assign a task to individual observations by selecting them and clicking on the relevant tags, or multi-select and assign a task to several observations at once. You can also remove a task from an observation by selecting ‘No task assigned’.
Affinity mapping is a flexible and visual way to quickly group, organize and make sense of qualitative data. Think in-person post-its and whiteboards, or digital whiteboard tools.
In terms of user interviews, the process would typically involve the following steps:
Reframer makes all of the above easier. All your observations are stored post-session, you can group similar ones in the affinity map and give them labels, and easily see your tags and titles of your groups, or ‘themes’.
When you first land on the affinity map, the canvas will be blank and all your observations and their metadata (tags, segments) will be in the list on the left hand side.
How to read, edit and group observations
Your observations are shown in detail on the card list, allowing you to read and review all your observations one-by-one. As you go through them, you can move them from the card list to the board to form groups.
To edit an observation, simply click into it and make changes.
To make sense of your data, move observations that relate to a similar concept into the same group. This will create a visual representation of different themes within your data.
You don’t have to get everything perfect in one go. Affinity mapping is a very flexible method, and we encourage you to go through the grouping and refinement in iterations.
You can start by creating a few groups first, and then splitting them or combining a few of them later. However, tidying up the groups by adding titles and aligning them neatly as you go makes it easier for you to revisit your groups further down the track.
Use search and filter to narrow down relevant observations
Use the search functionality to quickly narrow down observations based on keywords and use filters to help you look at small sets of data. You can filter by segments, research questions/tasks, participants, or tags.
When filters are applied:
Collaborating with team members
The affinity map was built to support collaborative analysis. Your team members can edit, group, label and comment on all observations alongside you in real-time.
When there are multiple people in the affinity map, you’ll each see where the other’s cursor is and if you’re working on a particular group or observation, it will be locked until you finish with it, ensuring that there’s no double ups.
What is the table view?
Once you’ve grouped your observations on the affinity map, you can view them all on the Affinity map groups table to get a clear view of all the groups and sort them by relevant stats. The numbers under the ‘Sessions’, ‘Observations’ and ‘Tags’ columns help you quickly gauge how significant each group is, based on the frequency of unique sessions referenced, as well as total observations included.
You can click on an observation group to view more details, including the observations, sessions (usually the participants), tags and tasks that are associated with this group.
This makes it really easy for you to review the groups, and decide if the title reflects the observations they contain. It also makes it faster for you to retrieve an observation or quote related to a group.
Ordering groups and using filters
Affinity map groups are listed alphabetically by default, if you want to see how frequent a group was referenced by different participants, you can order the list of groups with the number of sessions/participants. Simply click on the arrows next to the Sessions label.
You can also sort them by other associated metadata, such as the number of observations, and number of tags as well.
You can use filters to narrow down groups to find the ones that are associated with certain attributes, such as a specific tag or segment.
The Theme builder tab enables you to explore relationships between your tagged observations, and then create themes based on these relationships. The data in the theme builder updates automatically whenever you create a session, add an observation or add a tag within that study.
However, you’ll get the most out of exploring trends in your data once you’ve tagged a substantial number of observations, so we suggest doing this first.
There are 4 layers of filters you can use to ‘ask questions’ and spot patterns in your observations: session name, segments, tags, and tasks if you’ve set them up. You can use one or more of these types depending on what you are looking to discover.
Sessions lets you exclude entire sessions from the analysis. Got data from a participant that you’re not confident about? Leave it out for now until you can follow up.
Segments makes it simple to include or exclude multiple sessions based on the participant segment you created earlier. Only want to see data from people who matched a specific persona or used an iPhone? You can do that here.
Tags help you spot connections across all your included sessions and segments, and is where the real power of Reframer lies. As an example, you might spot lots of observations that are tagged with both ‘frustrated’ and ‘login’, hinting that there’s a potential issue with your login screen.
Tasks lets you see all the observations that were captured during your pre-set usability testing tasks. For example, if you wanted to see how all your participants fared during Task 2, you can filter your observations by Task 2 only.
Select a tag in the Theme Builder to display all related observations on the right. As soon as you select a tag, you can create a theme by clicking ‘Save this theme’ and naming it.
In our example study, we’ve selected the tag ‘frustration’ because it’s one of the most common tags in the study so far. Now, only the 8 observations tagged with ‘frustration’ are shown in the right hand column.
We could create a theme with just this tag at this stage, or we can leave it unnamed for now and explore a few more relationships before committing.
The tag you’ve selected will influence part of what the Theme Builder shows you next. The list of tags will still display the number of observations each tag appears on, but it’ll now also display the number of observations each tag has in common with your selected tag.
You can refine your search by combining multiple tags. Once you have existing filters applied, the numbers next to the other tags will now indicate how many observations you might see if you combine them.
► indicates how many observations this tag may have in common with your existing filters, i.e. narrowing with this tag.
◀ indicates how many additional observations this tag will add to your existing filters, i.e. broadening with this tag.
The chord diagram is a visual way to easily explore the relationships between your tagged observations and find key themes. You can choose to view all your tags, or exclude the ones you may not be interested in exploring right then and there.
