Despite the abundance of user research methods, card sorting remains one of the best ways to get into the minds of your users and discover how they understand and categorize information.

Given that one of our most popular tools is an online card sorting tool called OptimalSort (You may have heard of it), we thought that we’d answer some common questions about online card sorting – the research method that OptimalSort uses.

Let’s begin!

1. What’s the difference between online and offline card sorting?

Traditional card sorting can be done using paper cards and hosted in-person, hence “offline”. Online card sorting is pretty much what it sounds like: a card sort hosted over the internet. But there’s a little more to it.

Primarily, running a card sort online as opposed to hosting one in person means that the process becomes much easier to facilitate. Instead of needing to schedule a time for your participants to come into an office, you can simply send them a link to your card sort. Then, they can complete the test in their own time.

Note that the very benefits of online card sorting mean that you can lose certain insights gained from an in-person card sort, like understanding why your participants sort cards in a certain way. There are ways around this, however. For example, you could pair your card sorting tool with an online video recording solution.

2. When should you run an online card sort?

Card sorting is best suited to answering specific, information-related questions. For example, maybe you want to rearrange the layout of your magazine? Or perhaps you need to add several new shopping categories to your website.

In the latter example, card sorting is the perfect technique to find out where people would commonly expect to find those categories on your website. In the card sort, you present participants with a list of cards containing the names of items within certain categories and task them with sorting those items into groups that make sense to them. The end result? You have a clear picture of how your users or customers would arrange the content on your website.

Card sorting is useful when you’ve got the information you need to organize, but you’re just not sure how to organize it.

3. Do I need to compensate participants for taking part in my card sort?

Compensation is tricky when it comes to online testing methods like card sorting. While there are no hard and fast rules, you may find that it’s the best way to incentivize people to take part in your study. Now, taking part in an online card sort is much easier than trekking across town to sit down for a user interview, so you may want to offer participants the chance to win a prize for taking part instead of compensating them directly.

Note: Offering a discount for your product or service is a great way to compensate users and encourage the use of your product. 

4. How do I make sense of the data?

Most card sorting tools offer powerful analysis functionality built right into the tool itself, so all you have to worry about is actually putting the card sort together, sending out the links and promoting it.

Using OptimalSort as an example, let’s take a look at some of the analysis functionality and why it’s useful. Other card sorting tools will likely have different analysis options available.

• Participants Table: Review all of the people who took part in your card sort and segment or exclude them.
• Participant-Centric Analysis (PCA): See the most popular grouping strategies as well as the alternatives among those people who disagreed with the first strategy.
• Dendrograms: Quickly spot popular groups of cards and get a sense of how similar or different your participant’s card sorts were.

5. Is online card sorting expensive?

Online card sorting tools can be expensive, but it’s all relative. As just one example of this, online research platforms mean that you’ll likely be gaining access to a whole host of other tools by signing up for an online card sorting tool.

There’s also the fact that it’s a cheaper exercise overall than in-person card sorting as you won’t have to pay as much for compensation, or even use as much of your own time. Time is money!

6. Can I still get qualitative insights from an online card sort?

You can draw qualitative insights directly from the results of an online card sort, but you can also use online card sorting tools alongside participant recording software to build a more holistic understanding. By using recording software, you’ll be able to watch participants as they complete a card sort, and ask them to talk through what they’re doing to learn why they placed cards in a certain way. 

7. How many participants do I need?

In a nutshell, a larger number than you’d probably bring in for a user interview. Aim for between 20 and 30 participants.

Card sorting (whether it’s performed online or offline) is what’s known as a generative user testing method. This means that you’re typically starting without a design, and you’re using the method to get an idea of how people think with regards to the problem you’re trying to solve. A good example of this would be that you’re building a new website, and are using card sorting to learn how people think the content should be grouped and arranged.

Here’s a great quote from Nielsen Norman Group: “There is great variability in different people’s mental models and in the vocabulary they use to describe the same concepts. We must collect data from a fair number of users before we can achieve a stable picture of the users’ preferred structure and determine how to accommodate differences among users”.

8. How many cards should I use?

We recommend aiming for between 30 and 60 cards, as per our comprehensive 101 guide. Why? Because:

• People will be more likely to complete your card sort.
• You’ll only be able to include the most relevant cards, and be forced to discard the rest.
• You’ll get enough useful data and insights to make informed decisions about your website, app or project.

9. What online card sorting tools are available?

There are a number of online card sorting tools available, including our very own OptimalSort, which is one of the tools included in our platform. OptimalSort has a number of useful features to make it easy to set up and run a card sort with participants based all over the world. Once you’ve gathered all of your responses, built-in analysis features can then help you make sense of the data.

Of course, there are other options available. Take a look at this tools map from User Interviews for a comprehensive overview of the major research tools.

10. What do I do after a card sort?

With your card sort done and dusted, it’s time to take that data and build a draft structure of your website or mobile app. Once you’ve put this rough structure together, you can use tree testing to to see how people navigate through it. We’ve got a guide for that too, which you can read here.

Wrap-up

So that’s 10 questions about online card sorting – answered! If you’re interested in diving straight into a card sort of your own, we obviously recommend giving OptimalSort a try (which you can do for free).

Happy testing!

Whether you’re designing a new website or redesigning an existing one, card sorting is a quick, reliable and inexpensive research tool that can significantly improve your information architecture. By improving your information architecture, you’re giving yourself the best chance at delivering a product that is accessible, usable and relevant.

So, what exactly is card sorting? In short, card sorting is a user research technique that helps you discover how people understand and categorize information. Since great information architecture is built on the premise of organizing and categorizing information, card sorting is a secret weapon for website and digital product designers around the world. Actually, the tool is super common, and for good reason.

In this article we’ll help you prepare and conduct card sorting research. We’ll also help you make sense of the data you find and how to apply it to design great information architecture.

Planning and Preparation

Card sorting delivers the best results when you clearly define your goals. The narrower your scope, the more insightful and practical your results will be. It’s important to focus on one goal at a time when planning a card sorting study. What part of your information structure do you need clarification on? Organizing FAQ’s, product categories in an online store, or submenus, are common examples of card sorting projects.

Next, how best can you feasibly recruit participants? Depending on your situation, you may prefer conducting remote card sorts or in-person. Card sorts in person allow you to read body language and you may be more comfortable asking qualitative “why” questions of your participants. Whereas the benefits of remote card sorts, like OptimalSort, is that you aren’t constrained by location or time - just set it up, share the link with participants, then quickly analyze the results. In either case, be sure to recruit participants that represent the demographics of your intended users.

The next step is to prepare the cards themselves. The cards will represent the elements/topics that you wish to organize. Typically, you should aim for between 30 and 50 cards in order to get enough useful data. It also forces you to include only the most relevant cards. Additionally, they should also be on the same conceptual level to avoid confusion and ambiguity.

Finally, decide if you’re asking participants to group the cards based on categories that you decide (closed card sorting), or if participants will be able to create their own groups for cards (open card sorting). You can also facilitate hybrid card sorting which starts off as a closed card sort, but gives participants the option to create additional categories themselves. When you’re deciding, think about your task list (how you’re asking using to sort the cards) and how open-ended you’re prepared for the answers to be. Closed card sorting will narrow your results, whereas open will broaden your results.

Conducting the Session

Now that you’ve done the preparation, it’s time for the fun part! How involved you’ll be depends on whether you’re conducting remote or in-person sessions. We’ll discuss in-person card sorting first, then we’ll point out how remote card sorting differs.

An overview of conducting in-person card sorts:

1. First, shuffle the cards and give them to your participant(s). Ask them to look at each card, then direct them to either organize into groups on their own (open) or into the groups you have provided (closed). It’s important to emphasize to the participant that there are no right or wrong answers. Remember, you’re looking for a real, unfiltered insight into how people organize your information. You can even ask them to think out loud while they’re sorting the cards to gain additional, qualitative insight. One benefit of group sessions is that they usually do this anyway via natural discussion.
2. Then, if you’re running an open card sort, ask your participant(s) to name the groups they have organized. This will help you to understand the rationale behind their decisions and will give you some pointers when you come to labeling information architecture.
3. Once the session is complete, ask participants some open questions. Did you find any cards difficult to place? Did some overlap? Were any left out entirely? This sort of questioning, along with your notes throughout the session, will prove invaluable when you come to analyze the results.
4. Carefully collect the cards and make a record of the groups - there’s nothing worse than clearing the table and messing up the cards before you do this!

Remote card sorts differ from remote sessions in that once you’ve set up the cards in a tool like OptimalSort, you’re good to go. No printing, no shuffling, no resetting. You simply send a link to your participants and ask them to complete the task within a defined timeframe. Online card sorts are generally quicker and less time consuming in this respect, and they may allow you to find more participants and therefore more data.

There are two key things to highlight when running a remote card sort session. Firstly, ensure your instructions are clear and concise. Unlike an in-person session you won’t get the opportunity to clarify any misunderstandings. Secondly, you may consider a follow up questionnaire to gather additional qualitative insights. Check out this facilitation guide for more pointers on remote card sorting.

Analyzing and Interpreting the Results

Now that you’ve got all of your juicy data, it’s time to analyze it! If you ran a remote card sort, there will be some manual processing of your results (usually translating data to excel) which can be time consuming, whereas online tools will generally have analysis tools built right in. This is great for getting quick insights and quick development of information architecture.

When analyzing results, you’re really looking for patterns by identifying similar groups and labels. Using a tool like OptimalSort, for example, you’ll be provided with a few reports that will help you identify patterns and themes:

• Participants Table: Review all of the people who took part in your card sort and segment or exclude them.
• Participant-Centric Analysis (PCA): See the most popular grouping strategies as well as the alternatives among those people who disagreed with the first strategy.
• Dendrograms: Quickly spot popular groups of cards and get a sense of how similar or different your participant’s card sorts were.

Strong patterns or themes that emerge from the data tell us that participants understood categories in a similar way. On the flipside, different or disperse patterns tell us that there was no clear consensus on how information should be categorized. Both insights support effective design of information architecture. The goal is to find common ground in order to create seamless user experiences.

So far we’ve discussed statistical analysis which is all about the hard numbers. But it’s important to infuse some of the qualitative data into your reporting too. If you find that there is confusion within your results or no clear themes, you need to understand why. This is where the interpretation of questionnaire feedback or notes from in-person sessions become so valuable.

Using a combination of your data and your insights, it’s helpful to pull a summary together of your findings in a report. This can be shared with the wider team who have influence on the design of information architecture. Check out this analysis guide for more information on interpreting your results.

Conclusion

Card sorting is a fairly quick and straightforward way to inform information architecture design. It allows us to put the user at the center of our decisions surrounding the categorisation and grouping of information. Why is this important? Because as designers we can often assume how things should be organized. It’s too easy to be influenced by internal factors, like organization structures and the status quo. Don’t fall into this trap - use card sorting to gather clear, unbiased feedback on your information architecture.

Effective card sorting has clear objectives and is best suited to answering specific, information-related questions. We recommend using it when you need clarification around specific information structure, such navigation, menus and product categorisation.

As we’ve discussed, there are a few different approaches to card sorting research. They all have their place, so hoose which one best suits your needs. There’s a lot of resources available if you want to learn more. A good place to start is our card sorting 101 article. Good luck and happy researching!

Are your visitors really getting the most out of your website? Tree testing (or sometimes referred to as reverse card sorting) takes away the guesswork by telling you how easily, or not, people can find information on your website. Discover why Treejack is the tool of choice for website architects.

‍

What’s tree testing and why does it matter? 🌲 👀

‍

Whether you’re building a website from scratch or improving an existing website, tree testing helps you design your website architecture with confidence. How? Tools like Treejack use analysis to help assess how findable your content is for people visiting your website. 

‍

It helps answer burning questions  like:

• Do my labels make sense?
• Is my content grouped logically?
• Can people find what they want easily and quickly?  If not, why not?

‍

Treejack provides invaluable intel for any Information Architect. Why? Knowing where and why people get lost trying to find your content, gives you a much better chance of fixing the actual problem. And the more easily people can find what they’re looking for, the better their experience which is ultimately better for everyone.

‍

How’s tree testing work? 🌲🌳🌿

‍

Tree testing can be broken down into two main parts: 

‍

• The Tree - Your tree is essentially your site map – a text-only version of your website structure.
• The Task - Your task is the activity you ask participants to complete by clicking through your tree and choosing the information they think is right. Tools like Treejack analyse the data generated from doing the task to build a picture of how people actually navigated your content in order to try and achieve your task.  It tells you if they got it right or wrong, the path they took and the time it took them.

‍

Whether you’re new to tree testing or already a convert, effective tree testing using Treejack has some key steps.

‍

Step 1.  The ‘ Why’:  Purpose and goals of tree testing

‍

Ask yourself what part of your information architecture needs improvement – is it your whole website or just parts of it? Also think about your audience, they’re the ones you’re trying to improve the website for so the more you know about their needs the better. 

‍

Tip:  Make the most of what tree testing offers to improve your website by building it into your overall design project plan

‍

Step 2.  The ‘How’:   Build your tree

‍

You can build your tree using two main approaches: 

‍

• Create your tree in spreadsheet and import it into Treejack or
• Build your tree in Treejack itself, using the labels and structure of your website.

‍

Tip:  Your category labels are known as ‘parent nodes’. Your information labels are known as ‘child nodes’.

‍

Step 3. The ‘What’: Write your tasks

‍

The quality of your tasks will be reflected in the usefulness of your data so it’s worth making sure you create tasks that really test what you want to improve.

‍

Tip:  Use plain language that feels natural and try to write your tasks in a way that reflects the way people who visit your website might actually think when they are trying to find information on your site.

‍

Step 4.  The ‘Who’:  Recruit participants

‍

The quality of your data will largely depend on the quality of your participants. You want people who are as close to your target audience as possible and with the right attitude - willing and committed to being involved.

‍

Tip:  Consider offering some kind of incentive to participants – it shows you value their involvement.

‍

Step 5.  The ‘insights’: Interpret your results

‍

Now for the fun part – making sense of the results. Treejack presents the data from your tree testing as a series of tables and visualizations. You can download them in a spreadsheet in their raw format or customized to your needs.

‍

Tip:  Use the results to gain quick, practical insights you can act on right away or as a starter to dive deeper into the data.

‍

When should I use tree testing? ⌛

‍

Tree testing is useful whenever you want to find out if your website content is labelled and organised in a way that’s easy to understand.  What’s more it can be applied for any website, big (10+ levels with 10000s of labels) or small (3 levels and 22 labels) and any size in between.  Our advice for using Treejack is simply this: test big, test small, test often.

Think about the last time you were visiting a friend or family member’s house and needed to find something. Like a spoon, for example. You start in the kitchen because that’s where we’ve all been trained to find things like spoons. But where do you look next? Some may look in the drawer near the dishwasher, while others may look to the drawer that’s by the cabinet that holds dishes and bowls. Point is, people use context clues and other points of reference (like familiarity and habits) to find things that they’re looking for. 

The same goes for navigating a webpage.

However, unlike the spoon example, there’s much more variation in website structure than kitchen layout. Websites vary from industry to industry, ranging in purpose from ecommerce to digital portfolios and everything in between. So when someone lands on your website to find something or complete a task, is it obvious where they should go to get things done? 

Your gut might tell you that it is. The people on your design team might tell you that it is too. But what really matters is what happens with actual users. Whenever you’re building a new website or optimizing an existing one, it’s important that you build it with quality user insights. That starts with user experience (UX) research.

And before you ask if we mean qualitative or quantitative research – the answer is both. Let’s explore a particular user research method that’s essential for gathering the information needed to build intuitive site navigation. It’s called tree testing.

How tree testing evaluates findability of topics on a website

For anyone unfamiliar with tree testing (sometimes referred to as ‘reverse card sorting’ or ‘card-based classification’), it’s a series of tests that help you understand where and why people get lost in your content.

It works by taking text-based versions of websites and stripping the influence of navigation aids or design elements to build a study. Participants are shown the text-only version of your website or app and are asked to indicate where they would find specific items or topics.

By removing everything but the labels and how they’re organized, you can work out just how effective your website structure is at enabling people to find what they need. If test participants consistently struggle to locate specific items, you’ll know there are issues with your structure. It can also help you visualize what paths participants take to get to specific site information.

Another point worth mentioning is that tree testing not only uncovers site navigation flaws, but it reveals word choice and microcopy that may not be clear to users. To reference back to the initial analogy, one person might say “where would I find your spoons,” while another may ask, “where do you keep your utensils?” So, tree testing can also be helpful for understanding which words work best in your site structure.

When should I conduct a tree test?

While it’s best to conduct a tree test early in the research phase of a project, there’s never a wrong time. (Unless that time is not at all). And if you have an existing website, this is a good time to establish a base of your existing site structure. The results you gather here can help to highlight any issues with your site’s structure and provide the data needed to compare to any improvements.

If you’re starting from scratch with a new website, you can run tree tests on different versions of your proposed site structure and then compare the results to determine which makes the most sense to your users.

How do I conduct a tree test?

Running a tree test is simple. You can use an online tool (like TreeJack) to collect all of the necessary quantitative data you need to ensure successful site navigation.

But what about the qualitative side of it all? We’re so glad you asked.

How to add qualitative data to your tree test

Running tree tests is a great way to gather quantitative data about your site navigation and topic findability, but it's not so good at providing you with qualitative insights, or why people are looking for information the way that they are.

Traditionally, you could get your gift cards ready and set up an in-person study where you watch people perform the tasks laid out in your tree test, and subsequently ask why they searched for things the way they did. But that takes a lot of time, coordination, and compensation. Alternatively, you could set up a remote usability test using a platform designed to record users as they complete your tree test.

This way, you’ll get the quantitative results you need to make decisions, backed by qualitative insights from real people. You’ll be able to set up your test to get answers to questions like: 

• “Why did people go to X category to find information on X?”
• “Why did it take so long to find X?”
• "Why did people take that path to find X?"

Combine quantitative and qualitative research solutions to paint the whole picture

Teams that solely rely on quantitative data are only getting half the story. By pairing tree testing tools, like Optimal Workshop’s Treejack, with UserTesting’s Human Insight Platform, researchers are able to not only see how people search for information on a website but get to the bottom of why. When used together, quantitative and qualitative data reveal the most valuable insights for tree testing.