How to get started with tree testing 🌱

Are your visitors really getting the most out of your website? Tree testing (or sometimes referred to as reverse card sorting) takes away the guesswork by telling you how easily, or not, people can find information on your website. Discover why Treejack is the tool of choice for website architects.

‍

What’s tree testing and why does it matter? 🌲 👀

‍

Whether you’re building a website from scratch or improving an existing website, tree testing helps you design your website architecture with confidence. How? Tools like Treejack use analysis to help assess how findable your content is for people visiting your website. 

‍

It helps answer burning questions  like:

• Do my labels make sense?
• Is my content grouped logically?
• Can people find what they want easily and quickly?  If not, why not?

‍

Treejack provides invaluable intel for any Information Architect. Why? Knowing where and why people get lost trying to find your content, gives you a much better chance of fixing the actual problem. And the more easily people can find what they’re looking for, the better their experience which is ultimately better for everyone.

‍

How’s tree testing work? 🌲🌳🌿

‍

Tree testing can be broken down into two main parts: 

‍

• The Tree - Your tree is essentially your site map – a text-only version of your website structure.
• The Task - Your task is the activity you ask participants to complete by clicking through your tree and choosing the information they think is right. Tools like Treejack analyse the data generated from doing the task to build a picture of how people actually navigated your content in order to try and achieve your task.  It tells you if they got it right or wrong, the path they took and the time it took them.

‍

Whether you’re new to tree testing or already a convert, effective tree testing using Treejack has some key steps.

‍

Step 1.  The ‘ Why’:  Purpose and goals of tree testing

‍

Ask yourself what part of your information architecture needs improvement – is it your whole website or just parts of it? Also think about your audience, they’re the ones you’re trying to improve the website for so the more you know about their needs the better. 

‍

Tip:  Make the most of what tree testing offers to improve your website by building it into your overall design project plan

‍

Step 2.  The ‘How’:   Build your tree

‍

You can build your tree using two main approaches: 

‍

• Create your tree in spreadsheet and import it into Treejack or
• Build your tree in Treejack itself, using the labels and structure of your website.

‍

Tip:  Your category labels are known as ‘parent nodes’. Your information labels are known as ‘child nodes’.

‍

Step 3. The ‘What’: Write your tasks

‍

The quality of your tasks will be reflected in the usefulness of your data so it’s worth making sure you create tasks that really test what you want to improve.

‍

Tip:  Use plain language that feels natural and try to write your tasks in a way that reflects the way people who visit your website might actually think when they are trying to find information on your site.

‍

Step 4.  The ‘Who’:  Recruit participants

‍

The quality of your data will largely depend on the quality of your participants. You want people who are as close to your target audience as possible and with the right attitude - willing and committed to being involved.

‍

Tip:  Consider offering some kind of incentive to participants – it shows you value their involvement.

‍

Step 5.  The ‘insights’: Interpret your results

‍

Now for the fun part – making sense of the results. Treejack presents the data from your tree testing as a series of tables and visualizations. You can download them in a spreadsheet in their raw format or customized to your needs.

‍

Tip:  Use the results to gain quick, practical insights you can act on right away or as a starter to dive deeper into the data.

‍

When should I use tree testing? ⌛

‍

Tree testing is useful whenever you want to find out if your website content is labelled and organised in a way that’s easy to understand.  What’s more it can be applied for any website, big (10+ levels with 10000s of labels) or small (3 levels and 22 labels) and any size in between.  Our advice for using Treejack is simply this: test big, test small, test often.

In 2009, Bob Bailey and Cari Wolfson published published findings that changed how we approach first click testing and usability testing. They analyzed 12 scenario-based user tests and found that if someone gets their first click right, they're about twice as likely to complete their task successfully. This finding was so compelling that we built First Click Testing (formerly Chalkmark) specifically to help teams test this.  But we'd never actually validated their research using our own data, until now.

‍

Turns out, we're sitting on one of the world's largest databases of tree testing results. So we analyzed millions of task responses to see if the "first click predicts success" hypothesis holds up.

‍

It does. Convincingly.

‍

Users who get their first click correct are nearly three times more likely to complete their task successfully (70% vs 24% success rate).

‍

Here's how we validated the original study, what our data shows, and why first clicks matter more than you might think.

‍

‍

‍

Original first click testing study: 87% task success rate

‍

‍

Bob and Cari analyzed data from twelve usability studies on websites and products with varying amounts and types of content, a range of subject matter complexity, and distinct user interfaces. They found that people were about twice as likely to complete a task successfully if they got their first click right, than if they got it wrong:

‍

If the first click was correct, the chances of getting the entire scenario correct was 87% if the first click was incorrect, the chances of eventually getting the scenario correct was only 46%.

‍

‍

‍

Our Tree Testing data: First clicks predict 70% task success rate

‍

‍

We analyzed millions of tree testing responses in our database. We've found that people who get the first click correct are almost three times as likely to complete a task successfully:

‍

If the first click was correct, the chances of getting the entire scenario correct was 70% if the first click was incorrect, the chances of eventually getting the scenario correct was 24%

‍

To give you another perspective on the same data, here's the inverse:

‍

If the first click was correct, the chances of getting the entire scenario incorrect was 30% if the first click was incorrect, the chances of getting the whole scenario incorrect was 76%

‍

‍

‍

How Tree Testing measures first click success and task completion

‍

‍

Bob and Cari proved the usefulness of the methodology by linking two key metrics in scenario-based usability studies: first clicks and task success. First Click Testing doesn't measure task success — it's up to the researcher to determine as they're setting up the study what constitutes 'success', and then to interpret the results accordingly. Tree Testing (formerly Treejack) does measure task success — and first clicks.

‍

In a tree test, participants are asked to complete a task by clicking though a text-only version of a website hierarchy, and then clicking 'I'd find it here' when they've chosen an answer. Each task in a tree test has a pre-determined correct answer — as was the case in Bob and Cari's usability studies — and every click is recorded, so we can see participant paths in detail.

‍

Thus, every single time a person completes an individual tree testing task, we record both their first click and whether they are successful or not. When we came to test the 'correct first click leads to task success' hypothesis, we could therefore mine data from millions of task.

‍

To illustrate this, have a look at the results for one task. The overall Task result, you see a score for success and directness, and a breakdown of whether each Success, Fail, or Skip was direct (they went straight to an answer), or indirect (they went back up the tree before they selected an answer):

‍

‍

‍

‍

In the pie tree for the same task, you can look in more detail at how many people went the wrong way from a label (each label representing one page of your website):

‍

‍

‍

‍

In the First Click tab, you get a percentage breakdown of which label people clicked first to complete the task:

‍

‍

‍

‍

And in the Paths tab, you can view individual participant paths in detail (including first clicks), and can filter the table by direct and indirect success, fails, and skips (this table is only displaying direct success and direct fail paths):

‍

‍

‍

‍

‍

How to run first click tests: Best practices for usability testing

‍

‍

First click analysis is one of the most predictive metrics in usability testing. Whether you're testing wireframes, landing pages, or information architecture, measuring first click success gives you early insight into whether your design will work.

‍

This analysis reinforces something we already knew: first clicks matter. It is worth your time to get that first impression right. You have plenty of options for measuring the link between first clicks and task success in your scenario-based usability tests. From simply noting where your participants go during observations, to gathering quantitative first click data via online tools, you'll win either way. And if you want quantitative first click data, Optimal has you covered. First Click Testing works for wireframes and landing pages, while Tree Testing validates your information architecture.

‍

To finish, here are a few invaluable insights from other researchers on getting the most from first click testing:

‍

• Jeff Sauro details a useful approach to running a first click test, and shares the findings from a test he ran on 13 people.
• An article on Neoinsight describes three common usability problems that first click testing can solve.
• Gianna LaPin describes a first click test she ran on Netflix, VUDU, and Hulu Plus.

‍

‍

About this study

‍

This analysis was conducted in 2015 using millions of task responses from Optimal’s First Click and Tree Testing tools. While the dataset predates recent UI trends, the underlying behavioral principle, that a correct first click strongly predicts task success, remains consistent with modern usability research.

"Dear Optimal Worshop

I have two huge content structures I would like to A/B test. Do you think Treejack would be appropriate?"

— Mike

‍

Hi Mike (and excellent question)!

‍

Firstly, yes, Treejack is great for testing more than one content structure. It’s easy to run two separate Treejack studies — even more than two. It’ll help you decide which structure you and your team should run with, and it won’t take you long to set them up.

‍

When you’re creating the two tree tests with your two different content structures, include the same tasks in both tests. Using the same tasks will give an accurate measure of which structure performs best. I’ve done it before and I found that the visual presentation of the results — especially the detailed path analysis pietrees — made it really easy to compare Test A with Test B.

‍

Plus (and this is a big plus), if you need to convince stakeholders or teammates of which structure is the most effective, you can’t go past quantitative data, especially when its presented clearly — it’s hard to argue with hard evidence!

‍

Here’s two example of the kinds of results visualizations you could compare in your A/B test: the pietree, which shows correct and incorrect paths, and where people ended up:

‍

‍

‍

And the overall Task result, which breaks down success and directness scores, and has plenty of information worth comparing between two tests:

‍

‍

Keep in mind that running an A/B tree test will affect how you recruit participants — it may not be the best idea to have the same participants complete both tests in one go. But it’s an easy fix — you could either recruit two different groups from the same demographic, or test one group and have a gap (of at least a day) between the two tests.

‍

I’ve one more quick question: why are your two content structures ‘huge’?

‍

I understand that sometimes these things are unavoidable — you potentially work for a government organization, or a university, and you have to include all of the things. But if not, and if you haven’t already, you could run an open card sort to come up with another structure to test (think of it as an A/B/C test!), and to confirm that the categories you’re proposing work for people.

‍

You could even run a closed card sort to establish which content is more important to people than others (your categories could go from ‘Very important’ to ‘Unimportant’, or ‘Use everyday’ to ‘Never use’, for example). You might be able to make your content structure a bit smaller, and still keep its usefulness. Just a thought... and of course, you could try to get this information from your analytics (if available) but just be cautious of this because of course analytics can only tell you what people did and not what they wanted to do.

‍

All the best Mike!