Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
If the first click that someone makes on a website is ‘correct’ (i.e. takes them towards their goal), then they are two to three times as likely to find what they’re looking for than if their first click was in the wrong direction. First-click testing results can tell you if your customers are able to find what they’re looking for quickly and easily when they land on your webpage, and if not, where they click instead.
To create a first-click test, you’ll upload screenshots, sketches, or wireframes of the pages you want to test, and invite people to complete tasks by clicking where they think they’d find the right information.
First-click testing results enable you to make user-centered decisions for your webpages about:
Here’s an example task from start to finish to show you what first-click testing is and what it can tell you.
For this task, we decided to test the Bank of America’s homepage (please note that this case study and all of its data is for example purposes only). We wanted to find out if most people relied on Search to find information, or if they would rely on looking around the homepage.
Participants were presented with this task:
To complete the task, participants clicked on the screenshot where they thought they’d find the right information. Once they had clicked they were moved on to the next task:
Overall, 38 people completed this task, taking an average 2.09 seconds.
Chalkmark results are presented as various clickmaps that display where people clicked, with each map being customizable depending on the data that matters the most to you — you might want to see individual clicks, or large sections showing percentages only, for example.
For this task, we wanted to compare the ‘Search’ to the ‘find ATM’ button, so we increased the grid size so that the clicks around the ‘find closest ATM’ button were grouped together.
In answer to this question, we found that 18% of people went to the search bar, and 82% clicked on the ‘find closest ATM’ button lower on the page.
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
First-click testing can help you make informed decisions at any stage of a web design or market research project. The key to getting useful data is to be clear why you need it, and exactly what it can help you with.
Gather evidence for the ‘why’ from things like:
Every task you set will answer variations on the following questions about your webpage, so keep these in mind as you’re figuring out what you want to achieve. The answers to these questions (your first-click test data) will help you make most user-focused decisions about your designs.
When you come to create a first-click test, keep the goal in mind: you want to know if people can correctly find the answer to your task with something on the screen presented.
If we are presenting images of webpages, we know that the first click they make in their journey to their eventual outcome will determine the likelihood of success: if the first click is correct then they are two, to three-times more likely to be successful than if their first choice was a misstep.
Some ideas for things to test include:
Generally, throughout this guide, you will have noticed that we have referred to ‘webpages’, when talking about the ‘stimulus’ that you are uploading and testing in your first click test. Most first click tests are website tests, but all the same principles apply, whatever the stimulus.
If you want the ability to segment your participants for in-depth analysis, you can choose to ask pre and post study questions. This can be really useful if you see a split in where people have clicked; it may be due to a specific difference in your audience which you may be able to identify through applying filters based upon questionnaire answers.
You can upload JPEG (jpg) or PNG files of any size or dimension. If you upload high-density ‘retina images’, they’ll be scaled down to fit a normal-sized screen. No other image types will be automatically resized to fit. You might want all your images to be consistent across the whole test. If so, resize them before you upload.
The ‘preview’ feature is your best friend when creating a first-click test. Once you’ve uploaded your images and added tasks, preview the test on the devices your participants will use to complete the test. For example, if you are asking participants to complete the activity on a mobile device, preview your study on a mobile device.
For each task, click the ‘Set correct areas’ button.
On the image, draw the hotspots that correspond to the ‘correct areas’. You can name the areas, so that when you review the results, the scores will be reported against each named area.
What to mark as ‘correct’ and when
As with Treejack, it is possible to nominate your ‘correct’ answers after the results have come in. Sometimes you may feel you have to do that as you will realize that you have missed off a correct answer, for example, leaving off a footer link that corresponds to a valid correct click. We generally caution against doing this as you are literally falling into Texas Sharpshooter fallacy: shoot the barn door and then draw the target on afterwards to show you always intended to make that bullet grouping. There will be occasions when you’ll see clicks just outside the rectangle you have drawn; again the temptation is to stretch the rectangle to include these. Control your impulses!
Remember what you are really trying to do with a first click test: work out whether the labels and their locations are triggering the correct response from users when they are presented with a task. Are users clicking the thing you want them to click? Or are you seeing users finding sneaky ‘backdoor’ ways of answering the task? Or perhaps those off-target clicks are telling you your click target is not clear?
With a click test, you are not trying to get to a perfect score, you are trying to make navigation as painless as possible for the majority of users, so maybe including that ‘correct’ footer link in your results is not really what you should be doing. Instead focus on how to get users to click on the intended link through better, clearer design.
As noted, you can click test anything. We have click tested wireframes with no text at all, simply to understand from the page ‘shape’ where people expect things to be. Strangely revealing.
It’s totally acceptable to click test ‘bits’ of a site too; we’ve click tested just the mega-menus, with no page content and we’ve chopped the middle out of pages so that the top of the page and the footer can be seen without scrolling.
You can leave things out of page designs when testing, but beware then reporting ‘90% of people clicked in the right area’, as you are not acknowledging what might happen in a real world situation. Again, remember the purpose of a click test is not to get a perfect score, it is to learn what you can improve in your designs to better support users’ navigation choices.
Here are some examples of things you could test:
The screen and remote control for a video-streaming service:
An About Us page from a company:
A sketched wireframe for a website design:
A fancier version of the same wireframe:
Wireframe of a website at mobile resolution:
There’s a knack for getting user research tasks right, and it’s about understanding how people naturally think and behave. First-click tests happen away from a live website or app: people receive an invitation, read the instructions, and complete the tasks knowing it’s a test and not the real thing. We know this for certain: in general, if someone is given a test, they’ll want to pass.
You have two main weapons to combat this instinctive behavior: your instructions, and your tasks. The default instructions in Chalkmark make clear that the page is being tested, not the person. And you can get your tasks right by taking care with the language you use.
The data you get from a first-click test is about the link between task and success.
People tend to be action-oriented when they visit a website: everything they do, from scanning the page to clicking on a menu, is in pursuit of a goal. This may seem an obvious point for large service-based websites, such as government or university websites, but even people ‘just browsing’ are on the hunt for something, and primarily scan until they find it. The same goes when using an app, or when reviewing the cover of the magazine; people are working out what to click on or pay close attention to in order to reach their goal.
You want your tasks to mimic the thought a person might have when they visit your website. So write in a natural, plain English style, and introduce a true-to-life scenario for people to bring to mind.
Instead of writing ‘Find our office location’ you could write ‘You’ve booked a meeting with one of our staff, and you want to find out where to go.’ Keeping the tone conversational will make the task less ‘Click the thing’ and more ‘Find the information you need to get something done.’
In light of what we know about people — that they’re inclined to want to get it ‘right’ — they’ll scan the wording of your tasks searching for clues to help them do just that. If you set a task that includes “Find out about career options at our company”, and the word “Careers” appears on the webpage, your participants will probably play a game of ‘Snap’ instead of reacting to the task more naturally. Humans are very good at pattern matching and if the opportunity is presented to short-cut a problem or life task with a quick bit of pattern matching, we will all take it.
The first solution to avoiding pattern-matching tasks is to craft your tasks with the webpages out of sight. Though you might find it useful to use synonyms of your webpage words — a good place to start to get ideas flowing — it’s risky. Once a word or phrase gets in your head, it can be hard to shake, and this may limit the variations you could come up with. If you hide the webpage and brainstorm a bunch of ideas based on your knowledge of the website and your users, your tasks have a better shot at authenticity.
For example, let’s say you want to find out if people can find a funding application form on your website. Instead of writing a task that includes the verb ‘apply’, picture a user finding the link to the form and thinking ‘This is it, I found it.’ Then brainstorm a bunch of ideas for what they might have had in mind for what they might have had in mind, like:
In the following example, we wanted to find out whether or not people could find how to sign up for a bank account, and if they could, which link would they select (because there were three!). So we worded our task thus:
“You want to sign up to start banking with the Bank of America, where do you start this process?”
After 38 people had completed the task, we found out that 37% of people clicked on ‘Enrol’ and another 37% on ‘Open an account’. 16% clicked on the Call to Action (CTA) at the bottom of the page ‘Open a checking account – Get started now’, and the remaining 10% were scattered around ‘Schedule an appointment’ and ‘About us’.
If we had used ‘Enrol’ or ‘Open an account’ in our task, instead of ‘Sign up’, we’d have more ‘correct’ answers but it would be quite useless data, as users would most likely have been pattern matching the words in the task with words they saw on screen.
Chalkmark offers you the option to randomise the order of the pages shown to users. We recommend that you do this to minimise the effects of order on your results. Obviously, if you intend to test a sequence of pages, this won’t really be possible, but that sequence will reflect the user’s real life experience in any case.
How many tasks depends upon a few factors:
Generally, we feel 8 – 10 tasks is a reasonable limit, but this depends on the noted factors.
Whilst it’s possible to conduct lots of tests with a single page, we don’t really recommend it. The purpose of a click test is to test the findability of things on the webpage presented. If you present the same image to users repeatedly, they become so familiar with it that you cease to test findability and instead, test recall. Generally, never show the same page more than two, three or four times in a study and don’t show the same page in different sequential tasks – intersperse the same webpage with others.
Recruiting quality participants is important. You want participants who are as close to your intended user demographic as possible, and who are willing to put the time and thought into giving authentic responses.
If you have access to a pool of participants (like employees if you’re working on an internal product, or your customer mailing list) then these people are the best first choice. In fact, building up a loyal customer base that is willing to participate in user research is the best thing you can do to ensure quality data over time. Make use of various channels when inviting these people: email, website pop ups, social media, and so on.
You can also make use of reputable recruitment panels, which can be effective if you want trustworthy data with minimal effort. You can recruit participants from quite specific demographics, and be confident that the participants will take your study seriously (they are getting paid, after all). You can choose to do this within Chalkmark from your account using our in-app recruitment service.
The more people feel they personally get something from an activity, the more likely they’ll be to take it seriously. Here’s some insights into the psychology of study respondents that you’ll find useful (and that you can read more about on our blog):
As Scott Smith wrote in his insightful piece on why people participate in studies, we can learn a lot from theories of human behavior.
Aim for 50+ quality participants. Aiming for 50–100 completed first-click tests will give you reliable data that you can use to make informed decisions.
From what we’ve seen, the trends in participant responses should start to come clear with around 50 participants. The more participants you recruit from this point, the more confident you can be that the data is representative of your users.
Don’t fret if timing or budget means you can’t get 50+ completed first-click tests. If you can get between 30 and 50 participants, you’ll still see patterns that will lead you in the right direction and kick-start important conversations with stakeholders and designers. And fewer than 30? You’ll be surprised at how much you find out (plus data from a first-click test with 10 participants is way more useful than no first-click test at all).
Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.
Donna Spencer (the creator of the card sorting methodology
Card sorting is a research technique that helps you discover how people understand and categorize information, and ensures you create an information architecture that matches users’ expectations. In a card sort, participants sort labeled cards into groups. You can then use the results of your participants’ card sorts to give you ideas about how to group and label the information on your website* in a way that makes the most sense to your audience.
Card sorting is useful when you want to:
*We’ve used the word ‘website’ here and throughout this guide, but you could be organizing information in an app, an intranet, a TV program guide, a form, a board game or anything where information might be organized in a structure to make sense.
Card sorting involves creating a set of cards that each represent a concept or item, and asking people to group the cards in a way that makes sense to them.
Let’s say you’re working on redesigning a city council website and you want to understand how your users categorize the different content and information that will be on the site. You’ll add in a bunch of cards (these could be text or images, but more on that later) which will look like this in setup:
And look like this to your study participants:
There are three approaches to card sorting: open, closed and hybrid. Which approach you use will depend on what you want to find out. We’ll go into further detail on each method below, but here’s a high-level introduction to how they work:
Card sorting is most useful when you’ve already got the information or content you need to organize, but you’re just not sure exactly how to organize it.
Using the city council website as an example, you want to redesign how information is grouped together across the entire site. Card sorting will help you discover where people would commonly expect to find a category on your website.
You simply present them with a list of cards containing the names of items, concepts or labels and have users sort them into groups that make sense to them.
While card sorting is typically used in the early stages of the design process, when there’s no fixed information architecture (IA), it’s also common to use the technique to make changes to an IA, later down the line.
The three card sorting techniques — open, closed, and hybrid — will each tell you something different about how people understand and group your information. Choosing the right technique at the right time is key to gathering high-quality, relevant data to inform your design decisions.
It’s also the best place to start. Let’s take a deep dive into each of the card sorting methods…
First-click testing results are presented in clickmaps you can poke and prod and adjust for a whole bunch of different views and insights. You’ll also get detailed participant information, the results of your questionnaire, and links for sharing and downloading your results.
Overall, how you approach the results will depend on your style (you could call it your data personality, perhaps). You might perform a data smash-n-grab: dive in, pull out the quantitative data that’ll convince your client their website needs work, throw it up on a slide, and run. Or you might stroll through it, with clear questions in mind, but with an exploratory and creative approach — chances are you’ll want to do a bit of both.
The results Overview tells you all the need-to-know information about your first-click test. It updates in real time so you can see relevant dates, the number of people who’ve completed the test so far (and number who’ve abandoned), and the time taken.
The Participants table displays useful information about every participant who started your first-click test, and can be used to narrow and broaden the range of data you want to analyze.
You can use the participants table to:
You can do any of these things any time during your analysis, but we certainly recommend weeding out the ‘off’ or suspicious responses before conducting your analysis. If you plan to analyze data from different demographics separately, then you’d begin each demographic analysis at the Participant table.
If you’re interested in segmenting your results in the participants table based on answers to these questions, visiting the Questionnaire results first will help you spot the potentially significant and interesting patterns.
Here’s where you can see how participants responded to your pre and post study questions overall. If you’ve asked questions with multi-choice answers, you’ll see a bar graph, percentages of participants, and number of participants who selected each answer.
In our study we were interested in comparing the responses of people who were students (29%) to those who weren’t (71%).
To do this, we segmented the participants by whether or not they were a student, starting with those who answered ‘yes’. We selected them using the filter and then reloaded the results. Once we’d analyzed that group, we went back and reloaded the results to show the group that answered ‘no’.
To compare like with like, we looked at the same task, and used the same clickmap type, the selection view. We found there were a few small differences between the two groups, 16% of students clicked on ‘student banking’ rather than ‘explore checking solutions’ or ‘Bank of America advantage banking’ but mostly the two groups were very similar:
Students:
Non students:
First-click testing measures just that: individual clicks in response to tasks. How you measure things like success will depend on how you’ve set your test up and what your objectives are. You may not be interested in success rates at all, but instead just be curious about where people clicked, and thus how they perceive your content.
Bring to mind — and to hand — the questions you set out to ask at the start, like the following:
Our main question: On the Bank of America website, when people need to calculate their home loan payments, will they find and click on the ‘correct’ button that will get them the answers they are looking for (‘Calculate your monthly mortgage payment’), or will they look somewhere else?
Number of completed responses: 38
Number of successful clicks: 35
Average time taken: 21.7 seconds
Our webpage and results (Three different views of the same data):
Heatmap:
Grid:
Custom selection:
To use the selection view, use your cursor to drag out and select an area of your clickmap to view the percentage and number of clicks in that area. This can be helpful when you have a large button or thumbnail where the clicks are spread out but you want to count them as one group.
Our analysis: 92% of people (35 clicks) did go to the ‘Calculate your monthly mortgage payments- Try it out’ button. 8% of people (3 clicks) clicked elsewhere on the page, either on ‘5 steps toward your new home loan’ or ‘Get the latest mortgage rates’. Perhaps the biggest insight is how long it took people to click on average: 21.7 seconds! That’s like weeks on the web!
Finally, you have the option to download the analysis that Chalkmark has conducted for you. You can get heatmap images to include in your reports, spreadsheet summaries of the correct/incorrect clicks or the entire raw data as a spreadsheet.