March 29, 2016
3

Which comes first: card sorting or tree testing?

“Dear Optimal Workshop,I want to test the structure of a university website (well certain sections anyway). My gut instinct is that it's pretty 'broken'. Lots of sections feel like they're in the wrong place. I want to test my hypotheses before proposing a new structure. I'm definitely going to do some card sorting, and was planning a mixture of online and offline. My question is about when to bring in tree testing. Should I do this first to test the existing IA? Or is card sorting sufficient? I do intend to tree test my new proposed IA in order to validate it, but is it worth doing it upfront too?" — Matt

Dear Matt,

Ah, the classic chicken or the egg scenario: Which should come first — tree testing or card sorting?

It’s a question that many researchers often ask themselves, but I’m here to help clear the air!You should always use both methods when changing up your information architecture (IA) in order to capture the most information.

Tree testing and card sorting, when used together, can give you fantastic insight into the way your users interact with your site. First of all, I’ll run through some of the benefits of each testing method.

What is card sorting and why should I use it?

Card sorting is a great method to gauge the way in which your users organize the content on your site. It helps you figure out which things go together and which things don’t. There are two main types of card sorting: open and closed.

Closed card sorting involves providing participants with pre-defined categories into which they sort their cards. For example, you might be reorganizing the categories for your online clothing store for women. Your cards would have all the names of your products (e.g., “socks”, “skirts” and “singlets”) and you also provide the categories (e.g.,“outerwear”, “tops” and “bottoms”).

Open card sorting involves providing participants with cards and leaving them to organize the content in a way that makes sense to them. It’s the opposite to closed card sorting, in that participants dictate the categories themselves and also label them. This means you’d provide them with the cards only — no categories.

Card sorting, whether open or closed, is very user focused. It involves a lot of thought, input, and evaluation from each participant, helping you to form the structure of your new IA.

What is tree testing and why should I use it?

Tree testing is a fantastic way to determine how your users are navigating your site and how they’re finding information. Your site is organised into a tree structure, sorted into topics and subtopics, and participants are provided with some tasks that they need to perform. The results will show you how your participants performed those tasks, if they were successful or unsuccessful, and which route they took to complete the tasks. This data is extremely useful for creating a new and improved IA.

Tree testing is an activity that requires participants to seek information, which is quite the contrast to card sorting — an activity that requires participants to sort and organize information. Each activity requires users to behave in different ways, so each method will give its own valuable results.

Should you run a card or tree test first?

In this scenario, I’d recommend running a tree test first in order to find out how your existing IA currently performs. You said your gut instinct is telling you that your existing IA is pretty “broken”, but it’s good to have the data that proves this and shows you where your users get lost.

An initial tree test will give you a benchmark to work with — after all, how will you know your shiny, new IA is performing better if you don’t have any stats to compare it with? Your results from your first tree test will also show you which parts of your current IA are the biggest pain points and from there you can work on fixing them. Make sure you keep these tasks on hand — you’ll need them later!

Once your initial tree test is done, you can start your card sort, based on the results from your tree test. Here, I recommend conducting an open card sort so you can understand how your users organize the content in a way that makes sense to them. This will also show you the language your participants use to name categories, which will help you when you’re creating your new IA.

Finally, once your card sort is done you can conduct another tree test on your new, proposed IA. By using the same (or very similar) tasks from your initial tree test, you will be able to see that any changes in the results can be directly attributed to your new and improved IA.

Once your test has concluded, you can use this data to compare the performance from the tree test for your original information architecture — hopefully it is much better now!

Share this article
Author
Optimal
Workshop

Related articles

View all blog articles
Learn more
1 min read

UX workshop recap: experts from Meta, Netflix & Google share insights to elevate your career

Recently, Optimal Workshop partnered with Eniola Abioye, Lead UX Researcher at Meta and UXR Career Coach at UX Outloud to host a career masterclass featuring practical advice and guidance on how to: revamp and build a portfolio, emphasize the impact of your projects and showcase valuable collaborations. It also included panel discussions with experts from a variety of roles (UX, product management, engineering, career coaching and content design) talking about their journeys to becoming UX leaders. 

Keep reading to get key takeaways from the discussion on:

  • How to show the impact of your UX work
  • Common blockers in UX work
  • How to collaborate with cross-functional UX stakeholders 
  • How to build a resume and portfolio that uses industry language to present your experience

How to show the impact of your UX 💥

At a time when businesses are reducing costs to focus on profitability - proving the value of your work is more important than ever. Unfortunately, measuring the impact of UX isn’t as straightforward as tracking sales or marketing metrics. With this in mind, Eniola asked the panelists - how do you show the impact of UX in your work?Providing insights is simply not enough. “As a product manager, what I really care about is insights plus recommendations, because recommendations make my life easier,” said Kwame Odame. 

Auset Parris added her perspective on this topic as a Growth Content Designer, “the biggest thing for me to be impactful in my space [Content Design] is to consistently document the changes that I’ve made and share them with the team along with recommendations.” Auset also offered her perspective regarding recommendations, “recommendations are not always going to lead to the actual product executions, but recommendations are meant to guide us.” When it comes to deciding which recommendation to proceed with (if any) it's important to consider whether or not they are aligned with the overarching goal. 

Blockers in UX work 🚧

As UXR becomes more democratized in many businesses and the number of stakeholders increases, the ability to gain cross-functional buy-in for the role and outcomes of UXR is a key way to help keep research a priority. 

In his past experience, Kwame has realized that the role of a user experience researcher is just as important as that of a product manager, data scientist, engineer, or designer. However, one of the biggest blockers for him as a product manager is how the role of a UX researcher is often overlooked. “Just because I’m the product manager doesn’t mean that I’m owning every aspect of the product. I don’t have a magic wand right? We all work as a team.” Furthermore, Kwame notes that a user researcher is an incredibly hard role and a very important one, and I think we need to invest more in the UX space.

Auset also shared her perspective on the topic, “I wouldn’t say this is a blocker, but I do think this is a challenging piece of working in a team - there are so many stakeholders.” Although it would be ideal for each of the different departments to work seamlessly together at all times, that’s not always the case. Auset spoke about a time where the data scientists and user researchers were in disagreement. Her role as a Growth Content Designer is to create content that enhances the user experience. “But if I’m seeing two different experiences, how do I move forward? That’s when I have to ask everyone - come on let’s dig deeper. Are we looking at the right things?” If team members are seeing different results, or having different opinions, then maybe they are not asking the right questions and it's time to dig deeper. 

How to collaborate with cross-functional UX stakeholders 🫱🏽🫲🏻

The number and type of roles that now engage with research are increasing. As they do, the ability to collaborate and manage stakeholders in research projects has become essential. 

Kwame discussed how he sets up a meeting for the team to discuss their goals for the next 6 months. Then, he meets with the team on a weekly basis to ensure alignment. The main point of the meeting is to ensure everyone is leaving with their questions answered and blockers addressed. It's important to ensure everyone is moving in the right direction. 

Auset added that she thinks documentation is key to ensuring alignment. “One thing that has been helpful for me is having the documentation in the form of a product brief or content brief.” The brief can include the overarching goal, strategy, and indicate what each member of the team is working on. Team members can always look back at this document to ensure they are on the right track. 

Career advice: documenting the value you bring 💼

One of the participants asked the panel, “how do you secure the stability of your UX career?” 

Eniola took this opportunity to share some invaluable advice as a career coach, “I think the biggest thing that comes to mind is value proposition. It's important to be very clear about the value and impact you bring to the team. It used to be enough to just be really, really good at research and just do research and provide recommendations. Now that’s not enough. Now you have to take your teams through the process, integrate your recommendations into the product, and focus on driving impact.” 

Companies aren’t looking to hire someone who can perform a laundry list of tasks, they’re looking for UX professionals who can drive results. Think about the metrics you can track, to help showcase the impact of your work. For example, if you’re a UX designer - how much less time did the user spend on the task with your new design? Did the abandonment or error rate decrease significantly as a result of your work? How much did the overall customer satisfaction score rise, after you implemented your project? Before starting your project, decide on several metrics to track (make sure they align with your organization’s goals), and reflect on these after each project. 

Fatimah Richmond offered another piece of golden career advice. She encourages UX researchers to create an ongoing impact tracker. She’ll create a document where she lists the company's objectives, the projects she worked on, and the specific impact she made on the companies objectives. It's much easier to keep track of the wins as they happen, and jot a few notes about the impact you’ve made with each project, then scrambling to think of all the impact you’ve made when writing your resume. It's also important to note the impact your work has made on the different departments - product, marketing, sales, etc.

She also advises UX researchers to frequently share their science insights with their colleagues as the project progresses. Instead of waiting until the very end of the project and providing a “perfectly polished” deck, be transparent with the team about what you are working on and the impact it's having throughout the duration of the project.

Another participant asked - what if you need help determining the value you bring? Auset recommends asking for actionable feedback from coworkers. These people work with you every single day, so they know your contributions you are making to the team. 

Documenting the tangible impact you make as a UX professional is crucial - not only will it help create greater stability for your career, but it will also help organizations recognize the importance of a UX research. As Kwame discussed in the “blockers” section, one of the biggest challenges he faces as a product manager is the perception of the UX role as less important than the more traditional product manager, Engineer, and Designer roles. 

About Eniola Abioye

Eniola helps UX researchers improve their research practice. Whether you’re seasoned and looking to level up or a new researcher looking to get your bearings in UX, Eniola can help you focus and apply your skillset. She is a UX Researcher and Founder of UX Outloud. As a career coach, she guides her clients through short and long term SMART goals and then works with them to build a strategic plan of attack. She is innately curious, a self-starter, adaptable, and communicative with a knack for storytelling.

Learn more about UX Outloud.

Connect with Eniola on Linkedin.

About the panelists 🧑🏽🤝🧑🏽

The panel was comprised of talented professionals from a variety of fields including UX research, content strategy, product management & engineering, and career coaching. Their diverse perspectives led to an insightful and informative panel session. Keep reading to get to know each of the amazing panelists: 

Growth Content Designer: Auset Parris is a growth content designer at Meta. She has spent 7 years navigating the ever-evolving landscape of content strategy. She is passionate about the role of user research in shaping content strategies. Furthermore, Auset believes that understanding user behavior and preferences is fundamental to creating content that not only meets but exceeds user expectations. 

Senior UX Researcher: Jasmine Williams, Ph.D. is a senior researcher with over a decade of experience conducting youth-focused research. She has deep expertise in qualitative methods, child and adolescent development, and social and emotional well-being. Jasmine is currently a user experience researcher at Meta and her work focuses on teen safety and wellbeing. 

Product Manager: Kwame Odame has over 7 years of high-tech experience working in product management and software engineering. At Meta, Kwame is currently responsible for building the product management direction for Fan Engagement on Facebook. Kwame has also helped build Mastercard’s SaaS authentication platform, enabling cardholders to quickly confirm their identity when a suspicious transaction occurred, leveraging biometric technology. 

UX Researcher (UXR): Fatimah Richmond is a well-rounded UX researcher with over 15 years of experience, having influenced enterprise products across leading tech giants like Google, SAP, Linkedin, and Microsoft. Fatimah has led strategy for research, programs and operations that have significantly impacted the UXR landscape, from clinician engagement strategist to reshaping Linkedin Recruiter and Jobs. As a forward thinker, she’s here to challenge our assumptions and the status quo on how research gets planned, communicated, and measured.

Career Coach: An Xia spent the first decade of her professional life in consulting and Big Tech data science (Netflix, Meta). As a career coach, An has supported clients in gaining clarity on their career goals, navigating challenges of career growth, and making successful transitions. As a somatic coach, An has helped clients tap into the wisdom of their soma to reconnect with what truly matters to them. 

UX Strategist: Natalie Gauvin is an experienced professional with a demonstrated history of purpose-driven work in agile software development industries and higher education. Skilled in various research methodologies. Doctor of Philosophy (Ph.D.) Candidate in Learning Design and Technology from the University of Hawaii at Manoa, focused on empathy in user experience through personas

Level up your UXR capabilities (for free!) with the Optimal Academy 📚

Here at Optimal we really care about helping UX researchers level up their career. This is why we’ve developed the Optimal Academy, to help you master your Optimal Workshop skills and learn more about user research and information architecture.

Check out some of our free courses here: https://academy.optimalworkshop.com/

Learn more
1 min read

Empowering UX Careers: Designlab Joins Forces with Optimal Workshop

Optimal Workshop is thrilled to welcome Designlab as our newest education partner. This collaboration merges our strengths to provide innovative learning opportunities for UX professionals looking to sharpen their design skills and elevate their careers. 

The Power of a Design-First Education Partner

What makes Designlab unique is its exclusive focus on design education. For more than a decade, they have dedicated themselves to providing hands-on learning experiences that  combine asynchronous, online lessons and projects with synchronous group sessions and expert mentorship. With a robust catalog of industry-relevant courses and an alumni network of over 20,000 professionals, Designlab is committed to empowering designers to make an impact at both individual and team levels.

What Designlab Offers for Experienced Designers

Designlab offers a range of advanced programs that support ongoing professional development. Some courses that might be interesting for our audience include:

  • Data-Driven Design: Gain confidence in your ability to collect and interpret data, justify design decisions with business impact, and win over stakeholders. 
  • Advanced Figma: Accelerate your design workflow and become a more efficient Figma user by learning tools like components, auto-layout, and design tokens. 
  • Strategic Business Acumen for Designers: Learn the foundational business knowledge and frameworks you need to influence strategy and get your design career to the next level.  
  • Advanced Usability and Accessibility: Strengthen your usability and accessibility skills, integrate universal design principles into your work, and improve advocacy for inclusivity in design.  

These courses ensure that experienced designers can enhance their technical and strategic skills to solve complex problems, lead projects, and design user-centered experiences.

Solutions for Design Teams

Designlab also offers solutions for design teams looking to upskill together. These solutions can range from multi-seat enrollments to their courses to custom facilitation and training programs, perfectly tailored to your teams’ needs. By partnering with Designlab, companies ensure their teams are equipped with practical skills and a forward-thinking mindset to tackle design challenges effectively.

READ: Designing for Accessibility with The Home Depot

Special Offer for the Optimal Workshop Community

To celebrate this partnership, Optimal Workshop users can take advantage of a special discount—$100 off any Designlab course with the code OPTIMAL. Whether you’re looking to refine your skills or explore new areas of expertise, Designlab’s programs offer the perfect opportunity to invest in your professional growth.

Explore how Designlab’s offerings can help you level up your design career—whether it’s through mastering advanced tools, leveraging data more, or becoming a more strategic thinker. With continuous learning at the heart of success in UX and product design, there’s no better time to start your journey with Designlab.

Unlock your potential and discover new possibilities with Designlab’s courses today. Use code OPTIMAL to save $100 on your next course and take the next step in your design career.

Learn more
1 min read

Ready for take-off: Best practices for creating and launching remote user research studies

"Hi Optimal Work,I was wondering if there are some best practices you stick to when creating or sending out different UX research studies (i.e. Card sorts, Prototyye Test studies, etc)? Thank you! Mary"

Indeed I do! Over the years I’ve learned a lot about creating remote research studies and engaging participants. That experience has taught me a lot about what works, what doesn’t and what leaves me refreshing my results screen eagerly anticipating participant responses and getting absolute zip. Here are my top tips for remote research study creation and launch success!

Creating remote research studies

Use screener questions and post-study questions wisely

Screener questions are really useful for eliminating participants who may not fit the criteria you’re looking for but you can’t exactly stop them from being less than truthful in their responses. Now, I’m not saying all participants lie on the screener so they can get to the activity (and potentially claim an incentive) but I am saying it’s something you can’t control. To help manage this, I like to use the post-study questions to provide additional context and structure to the research.

Depending on the study, I might ask questions to which the answers might confirm or exclude specific participants from a specific group. For example, if I’m doing research on people who live in a specific town or area, I’ll include a location based question after the study. Any participant who says they live somewhere else is getting excluded via that handy toggle option in the results section. Post-study questions are also great for capturing additional ideas and feedback after participants complete the activity as remote research limits your capacity to get those — you’re not there with them so you can’t just ask. Post-study questions can really help bridge this gap. Use no more than five post-study questions at a time and consider not making them compulsory.

Do a practice run

No matter how careful I am, I always miss something! A typo, a card with a label in the wrong case, forgetting to update a new version of an information architecture after a change was made — stupid mistakes that we all make. By launching a practice version of your study and sharing it with your team or client, you can stop those errors dead in their tracks. It’s also a great way to get feedback from the team on your work before the real deal goes live. If you find an error, all you have to do is duplicate the study, fix the error and then launch. Just keep an eye on the naming conventions used for your studies to prevent the practice version and the final version from getting mixed up!

Sending out remote research studies

Manage expectations about how long the study will be open for

Something that has come back to bite me more than once is failing to clearly explain when the study will close. Understandably, participants can be left feeling pretty annoyed when they mentally commit to complete a study only to find it’s no longer available. There does come a point when you need to shut the study down to accurately report on quantitative data and you’re not going to be able to prevent every instance of this, but providing that information upfront will go a long way.

Provide contact details and be open to questions

You may think you’re setting yourself up to be bombarded with emails, but I’ve found that isn’t necessarily the case. I’ve noticed I get around 1-3 participants contacting me per study. Sometimes they just want to tell me they completed it and potentially provide additional information and sometimes they have a question about the project itself. I’ve also found that sometimes they have something even more interesting to share such as the contact details of someone I may benefit from connecting with — or something else entirely! You never know what surprises they have up their sleeves and it’s important to be open to it. Providing an email address or social media contact details could open up a world of possibilities.

Don’t forget to include the link!

It might seem really obvious, but I can’t tell you how many emails I received (and have been guilty of sending out) that are missing the damn link to the study. It happens! You’re so focused on getting that delivery right and it becomes really easy to miss that final yet crucial piece of information.

To avoid this irritating mishap, I always complete a checklist before hitting send:

  • Have I checked my spelling and grammar?
  • Have I replaced all the template placeholder content with the correct information?
  • Have I mentioned when the study will close?
  • Have I included contact details?
  • Have I launched my study and received confirmation that it is live?
  • Have I included the link to the study in my communications to participants?
  • Does the link work? (yep, I’ve broken it before)

General tips for both creating and sending out remote research studies

Know your audience

First and foremost, before you create or disseminate a remote research study, you need to understand who it’s going to and how they best receive this type of content. Posting it out when none of your followers are in your user group may not be the best approach. Do a quick brainstorm about the best way to reach them. For example if your users are internal staff, there might be an internal communications channel such as an all-staff newsletter, intranet or social media site that you can share the link and approach content to.

Keep it brief

And by that I’m talking about both the engagement mechanism and the study itself. I learned this one the hard way. Time is everything and no matter your intentions, no one wants to spend more time than they have to. Even more so in situations where you’re unable to provide incentives (yep, I’ve been there). As a rule, I always stick to no more than 10 questions in a remote research study and for card sorts, I’ll never include more than 60 cards. Anything more than that will see a spike in abandonment rates and of course only serve to annoy and frustrate your participants. You need to ensure that you’re balancing your need to gain insights with their time constraints.

As for the accompanying approach content, short and snappy equals happy! In the case of an email, website, other social media post, newsletter, carrier pigeon etc, keep your approach spiel to no more than a paragraph. Use an audience appropriate tone and stick to the basics such as: a high level sentence on what you’re doing, roughly how long the study will take participants to complete, details of any incentives on offer and of course don’t forget to thank them.

Set clear instructions

The default instructions in Optimal Workshop’s suite of tools are really well designed and I’ve learned to borrow from them for my approach content when sending the link out. There’s no need for wheel reinvention and it usually just needs a slight tweak to suit the specific study. This also helps provide participants with a consistent experience and minimizes confusion allowing them to focus on sharing those valuable insights!

Create a template

When you’re on to something that works — turn it into a template! Every time I create a study or send one out, I save it for future use. It still needs minor tweaks each time, but I use them to iterate my template.What are your top tips for creating and sending out remote user research studies? Comment below!

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.