Optimal Blog

Whether you’re designing a new website or redesigning an existing one, card sorting is a quick, reliable and inexpensive research tool that can significantly improve your information architecture. By improving your information architecture, you’re giving yourself the best chance at delivering a product that is accessible, usable and relevant.

So, what exactly is card sorting? In short, card sorting is a user research technique that helps you discover how people understand and categorize information. Since great information architecture is built on the premise of organizing and categorizing information, card sorting is a secret weapon for website and digital product designers around the world. Actually, the tool is super common, and for good reason.

In this article we’ll help you prepare and conduct card sorting research. We’ll also help you make sense of the data you find and how to apply it to design great information architecture.

Planning and Preparation

Card sorting delivers the best results when you clearly define your goals. The narrower your scope, the more insightful and practical your results will be. It’s important to focus on one goal at a time when planning a card sorting study. What part of your information structure do you need clarification on? Organizing FAQ’s, product categories in an online store, or submenus, are common examples of card sorting projects.

Next, how best can you feasibly recruit participants? Depending on your situation, you may prefer conducting remote card sorts or in-person. Card sorts in person allow you to read body language and you may be more comfortable asking qualitative “why” questions of your participants. Whereas the benefits of remote card sorts, like OptimalSort, is that you aren’t constrained by location or time - just set it up, share the link with participants, then quickly analyze the results. In either case, be sure to recruit participants that represent the demographics of your intended users.

The next step is to prepare the cards themselves. The cards will represent the elements/topics that you wish to organize. Typically, you should aim for between 30 and 50 cards in order to get enough useful data. It also forces you to include only the most relevant cards. Additionally, they should also be on the same conceptual level to avoid confusion and ambiguity.

Finally, decide if you’re asking participants to group the cards based on categories that you decide (closed card sorting), or if participants will be able to create their own groups for cards (open card sorting). You can also facilitate hybrid card sorting which starts off as a closed card sort, but gives participants the option to create additional categories themselves. When you’re deciding, think about your task list (how you’re asking using to sort the cards) and how open-ended you’re prepared for the answers to be. Closed card sorting will narrow your results, whereas open will broaden your results.

Conducting the Session

Now that you’ve done the preparation, it’s time for the fun part! How involved you’ll be depends on whether you’re conducting remote or in-person sessions. We’ll discuss in-person card sorting first, then we’ll point out how remote card sorting differs.

An overview of conducting in-person card sorts:

1. First, shuffle the cards and give them to your participant(s). Ask them to look at each card, then direct them to either organize into groups on their own (open) or into the groups you have provided (closed). It’s important to emphasize to the participant that there are no right or wrong answers. Remember, you’re looking for a real, unfiltered insight into how people organize your information. You can even ask them to think out loud while they’re sorting the cards to gain additional, qualitative insight. One benefit of group sessions is that they usually do this anyway via natural discussion.
2. Then, if you’re running an open card sort, ask your participant(s) to name the groups they have organized. This will help you to understand the rationale behind their decisions and will give you some pointers when you come to labeling information architecture.
3. Once the session is complete, ask participants some open questions. Did you find any cards difficult to place? Did some overlap? Were any left out entirely? This sort of questioning, along with your notes throughout the session, will prove invaluable when you come to analyze the results.
4. Carefully collect the cards and make a record of the groups - there’s nothing worse than clearing the table and messing up the cards before you do this!

Remote card sorts differ from remote sessions in that once you’ve set up the cards in a tool like OptimalSort, you’re good to go. No printing, no shuffling, no resetting. You simply send a link to your participants and ask them to complete the task within a defined timeframe. Online card sorts are generally quicker and less time consuming in this respect, and they may allow you to find more participants and therefore more data.

There are two key things to highlight when running a remote card sort session. Firstly, ensure your instructions are clear and concise. Unlike an in-person session you won’t get the opportunity to clarify any misunderstandings. Secondly, you may consider a follow up questionnaire to gather additional qualitative insights. Check out this facilitation guide for more pointers on remote card sorting.

Analyzing and Interpreting the Results

Now that you’ve got all of your juicy data, it’s time to analyze it! If you ran a remote card sort, there will be some manual processing of your results (usually translating data to excel) which can be time consuming, whereas online tools will generally have analysis tools built right in. This is great for getting quick insights and quick development of information architecture.

When analyzing results, you’re really looking for patterns by identifying similar groups and labels. Using a tool like OptimalSort, for example, you’ll be provided with a few reports that will help you identify patterns and themes:

• Participants Table: Review all of the people who took part in your card sort and segment or exclude them.
• Participant-Centric Analysis (PCA): See the most popular grouping strategies as well as the alternatives among those people who disagreed with the first strategy.
• Dendrograms: Quickly spot popular groups of cards and get a sense of how similar or different your participant’s card sorts were.

Strong patterns or themes that emerge from the data tell us that participants understood categories in a similar way. On the flipside, different or disperse patterns tell us that there was no clear consensus on how information should be categorized. Both insights support effective design of information architecture. The goal is to find common ground in order to create seamless user experiences.

So far we’ve discussed statistical analysis which is all about the hard numbers. But it’s important to infuse some of the qualitative data into your reporting too. If you find that there is confusion within your results or no clear themes, you need to understand why. This is where the interpretation of questionnaire feedback or notes from in-person sessions become so valuable.

Using a combination of your data and your insights, it’s helpful to pull a summary together of your findings in a report. This can be shared with the wider team who have influence on the design of information architecture. Check out this analysis guide for more information on interpreting your results.

Conclusion

Card sorting is a fairly quick and straightforward way to inform information architecture design. It allows us to put the user at the center of our decisions surrounding the categorisation and grouping of information. Why is this important? Because as designers we can often assume how things should be organized. It’s too easy to be influenced by internal factors, like organization structures and the status quo. Don’t fall into this trap - use card sorting to gather clear, unbiased feedback on your information architecture.

Effective card sorting has clear objectives and is best suited to answering specific, information-related questions. We recommend using it when you need clarification around specific information structure, such navigation, menus and product categorisation.

As we’ve discussed, there are a few different approaches to card sorting research. They all have their place, so hoose which one best suits your needs. There’s a lot of resources available if you want to learn more. A good place to start is our card sorting 101 article. Good luck and happy researching!

Thanks to an ever-growing digital world, businesses are spoiled for information and data. The more complex the business, the more information there is and the more complicated the business requirements are. But where there are challenges, there is opportunity. That’s where information architecture and data architecture come in.

‍

Information and data architecture both seek to make sense of the plethora of information a business handles. However, the two have different roles to play in the way businesses use, move, maintain, and present data - both to internal and external stakeholders. So what are they and why should businesses take note?

‍

Defining Information Architecture 🗺️

‍

Information architecture is the structure used to organize and label content on websites, mobile applications and other digital environments. Its primary purpose is to enhance user experience by ensuring information is structured in an accessible, usable and relevant way.

‍

Information architecture seeks to understand user needs and goals by analyzing both existing and required information, then building an information framework in a logical and user-friendly way. It deals with three main components:

‍

• Labels: How information is represented
• Navigation: How users make their way through the information
• Search: How users look for information

‍

Whilst this information sits in the background, it’s the layer upon which you build the design of your digital products.

‍

Information architects bring data from file systems and databases to life by building meaningful narratives and stories. Outputs can include site-mapping, information architecture diagrams and content inventories. These outputs are supported by user research techniques such as card sorting, tree testing, user surveys and first-click testing.

‍

Defining Data Architecture 💻

‍

Data architecture bridges the gap between business needs, goals, and system requirements related to data handling. It sets out a framework for managing data assets, the flow of data and the maintenance of data systems. As such, it has a slightly more macro view than information architecture and concerns itself with emerging technologies such as artificial intelligence, machine learning, and blockchain.

‍

Where information architecture centers around the end-user interaction, data architecture centers around practical handling and operation of data processes i.e. collection through to transformation, distribution, and consumption. Because of this, data architecture must take into account the businesses ability to scale operations, integrate with third party systems, support real-time data processes and the reduction of operating costs. Modern data architecture may point to artificial intelligence to tackle some of these challenges.

‍

The Importance of Enterprise Architects in Information and Data Architecture 🏗

‍

Enterprise architects are big-picture people. Data architecture and information architecture both fall within their remit, and they often oversee other data management job specialities within an IT department.

‍

As a leader (and often, visionary) within a business, enterprise architects shoulder the responsibility of ‘mission critical’ projects. As a result, they tend to have several years experience with IT systems, backed by a bachelor’s degree in computer science, IT management, data science or similar. Many will hold a master’s degree and specialty certifications.

‍

The role involves collaborating with senior business leaders, solution-delivery teams and external stakeholders, and requires creative problem solving and excellent communication skills. Therefore, enterprise architects very much steer the ship when it comes to information and data architecture. Combining high-level business strategy with knowledge of ‘the nuts and bolts’ of IT data systems and processes, they command an annual salary in New Zealand between $150,000 and $200,000 per annum..

‍

Continuous improvement within any business that has substantial IT infrastructure calls for serious investment in enterprise architecture.

‍

Designing and Implementing an Effective Information and Data Architecture 𝞹📈🧠📚

‍

Once overarching business goals are aligned with the scope of data and system requirements, information and data architecture design (or redesign) can begin.

‍

Crucial to the design and implementation process is developing an architecture framework. This is a set of guidelines that lays out principles, practices, tools and approaches required to complete the design. It supports system design decisions, assigns key tasks and provides project guidance throughout the design process. The framework essentially aims to unite disparate teams and maintain business and IT alignment.

‍

The choice of architecture design is also critical. It should consider scalability, performance, maintainability and adaptability to emerging technology. Which is why cloud platforms feature so heavily in modern data architecture. Cloud architects will navigate the architecture design and technical requirements of cloud-based delivery models, which offer the solution to those scalability and adaptability challenges. They are responsible for bridging the gaps between complex business problems and solutions in the cloud. Modern data architecture tends to involve some form of cloud delivery component.

‍

Throughout implementation, data and information architects will work closely with designers and engineers until testable architecture is ready. User research and testing will be carried out, and a feedback loop will commence until requirements are met. Users, as always, should be at the center of your digital product.

‍

Summing Up the Complexities of Information and Data Architecture 🧮

‍

Whilst the difference between information and data architecture can appear nuanced on the surface, they hold unique roles when delivering a cohesive, user-friendly digital product.

‍

Think of a sliding scale where business operations sit at one end, and users sit at the other. Data architecture addresses challenges closer to the business: aligning business requirements and goals with how data flows through the system. On the other hand, information architecture addresses the challenges related to how this data is organized and interpreted for the end user.

‍

At the end of the day, both information and data architecture need to work in harmony to satisfy the user and the business.

The term ‘usability’ captures sentiments of how usable, useful, enjoyable, and intuitive a website or app is perceived by users. By its very nature, usability is somewhat subjective. But what we’re really looking for when we talk about usability is how well a website can be used to achieve a specific task or goal. Using this definition we can analyze usability metrics (standard units of measurement) to understand how well user experience design is performing.

‍

Usability metrics provide helpful insights before and after any digital product is launched. They help us form a deeper understanding of how we can design with the user front of mind. This user-centered design approach is considered the best-practice in building effective information architecture and user experiences that help websites, apps, and software meet and exceed users' needs.

‍

In this article, we’ll highlight key usability metrics, how to measure and understand them, and how you can apply them to improve user experience.

‍

Understanding Usability Metrics

‍

Usability metrics aim to understand three core elements of usability, namely: effectiveness, efficiency, and satisfaction. A variety of research techniques offer designers an avenue for quantifying usability. Quantifying usability is key because we want to measure and understand it objectively, rather than making assumptions.

‍

Types of Usability Metrics
‍

There are a few key metrics that we can measure directly if we’re looking to quantify effectiveness, efficiency, and satisfaction. Here are four common examples:

‍

• Success rate: Also known as ‘completion rate’, success rate is the percentage of users who were able to successfully complete the tasks.
• Time-based efficiency: Also known as ‘time on task’, time-based efficiency measures how much time a user needs to complete a certain task.
• Number of errors: Sounds like what it is! It measures the average number of times where an error occurred per user when performing a given task.
• Post-task satisfaction: Measures a user's general impression or satisfaction after completing (or not completing) a given task.

‍

How to Collect Usability Metrics

‍
Usability metrics are outputs from research techniques deployed when conducting usability testing. Usability testing in web design, for example, involves assessing how a user interacts with the website by observing (and listening to) users completing defined tasks, such as purchasing a product or signing up for newsletters.

‍

Conducting usability testing and collecting usability metrics usually involves:

‍

• Defining a set of tasks that you want to test
• Recruitment of test participants
• Observing participants (remotely or in-person)
• Recording detailed observations
• Follow-up satisfaction survey or questionnaire

‍

Tools such Reframer are helpful in conducting usability tests remotely, and they enable live collaboration of multiple team members. It is extremely handy when trying to record and organize those insightful observations! Using paper prototypes is an inexpensive way to test usability early in the design process.

‍

The Importance of Usability Metrics in User-Centered Design

‍

User-centered design challenges designers to put user needs first. This means in order to deploy user-centered design, you need to understand your user. This is where usability testing and metrics add value to website and app performance; they provide direct, objective insight into user behavior, needs, and frustrations. If your user isn’t getting what they want or expect, they’ll simply leave and look elsewhere.

‍

Usability metrics identify which parts of your design aren’t hitting the mark. Recognizing where users might be having trouble completing certain actions, or where users are regularly making errors, are vital insights when implementing user-centered design. In short, user-centered design relies on data-driven user insight.

‍

But why hark on about usability metrics and user-centered design? Because at the heart of most successful businesses is a well-solved user problem. Take Spotify, for example, which solved the problem of dodgy, pirated digital files being so unreliable. People liked access to free digital music, but they had to battle viruses and fake files to get it. With Spotify, for a small monthly fee, or the cost of listening to a few ads, users have the best of both worlds. The same principle applies to user experience - identify recurring problems, then solve them.

‍

Best Practices for Using Usability Metrics

‍

Usability metrics should be analyzed by design teams of every size. However, there are some things to bear in mind when using usability metrics to inform design decisions:

‍

• Defining success: Usability metrics are only valuable if they are being measured against clearly defined benchmarks. Many tasks and processes are unique to each business, so use appropriate comparisons and targets; usually in the form of an ‘optimized’ user (a user with high efficiency).
• Real user metrics: Be sure to test with participants that represent your final user base. For example, there’s little point in testing your team, who will likely be intimately aware of your business structure, terminology, and internal workflows.
• Test early: Usability testing and subsequent usability metrics provide the most impact early on in the design process. This usually means testing an early prototype or even a paper prototype. Early testing helps to avoid any significant, unforeseen challenges that could be difficult to rewind in your information architecture.
• Regular testing: Usability metrics can change over time as user behavior and familiarity with digital products evolve. You should also test and analyze the usability of new feature releases on your website or app.

‍

Remember, data analysis is only as good as the data itself. Give yourself the best chance of designing exceptional user experiences by collecting, researching, and analyzing meaningful and accurate usability metrics.

‍

Conclusion

‍

Usability metrics are a guiding light when it comes to user experience. As the old saying goes, “you can’t manage what you can’t measure”. By including usability metrics in your design process, you invite direct user feedback into your product. This is ideal because we want to leave any assumptions or guesswork about user experience at the door.

‍

User-centered design inherently relies on constant user research. Usability metrics such as success rate, time-based efficiency, number of errors, and post-task satisfaction will highlight potential shortcomings in your design. Subsequently, they identify where improvements can be made, AND they lay down a benchmark to check whether any resulting updates addressed the issues.

‍

Ready to start collecting and analyzing usability metrics? Check out our guide to planning and running effective usability tests to get a head start!

Your cards have been sorted, and now you have lots of amazing data and insight to help improve your information architecture. So how do you interpret the results? 

‍

Never fear, our product ninjas Alex and Aidan are here to help. In our latest live training session they take you on a walk-through of card sort analysis using OptimalSort.
‍

‍
What they cover:

‍

• Use cases for open, closed and hybrid card sort methodologies
• How, when and why to standardize categories
• How to interpret 3D cluster views, dendrograms, and similarity matrix
• Tips on turning those results into actionable insights

‍

‍

In our latest live training session product experts, Pete and Caitlin, take us on a deep dive into the new and improved qualitative analysis tool Reframer.

‍

The session is loaded with tips and demo’s on how to save time and streamline your qualitative research all within one tool.  They also discuss best practices for setting up and conducting user interviews, and how to get the most out of your analysis.

‍

Every month we have fun and informative “bite sized” presentations to add some inspiration to your lunch break.  These virtual events allow us to partner with amazing speakers, community groups and organizations to share their insights and hot takes on a variety of topics impacting our industry. 

‍

Join us at the end of every month for Lunch n' Learn.

‍

‍

Susanna Carman

‍

Leading design processes amidst a world in transition requires all practitioners to continuously invest in their own development. One aspect worth investing in, is an ability to integrate holistic thinking into our design leadership practice. This includes re-evaluating our own biases and how that bias is reflected in the tools we choose to work with when understanding and designing for/within complex systems.

‍

Recently our guest Susanna Carman, Strategic Designer and founder of Transition Leadership LAB, introduced us to a holistic approach to qualitative design research using Ken Wilber’s 4 Quadrant Model. Susanna explained the fundamental principles underpinning the framework, and showed how it can be used to ensure a multi-perspectival harvest of critical qualitative and quantitative data on any design project.  

‍

‍

‍

Speaker bio

‍

Susanna Carman is a Strategic Designer and research-practitioner who helps people solve complex problems, the types of problems that have to do with services, systems and human interactions. Specialising in design, leadership and learning, Susanna brings a high value toolkit and herself as Thinking Partner to design, leadership and change practitioners who are tasked with delivering sustainable solutions amidst disruptive conditions. 

‍

Susanna holds a Masters of Design Futures degree from RMIT University, and has over a decade of combined experience delivering business performance, cultural alignment and leadership development outcomes to the education, health, community development and financial services sectors. She is also the founder and host of Transition Leadership Lab, a 9-week learning lab for design, leadership and change practitioners who already have a sophisticated set of tools and mindsets, but still feel these are insufficient to meet the challenge of leading change in a rapidly transforming world.

‍

Grab your lunch, invite your colleagues and we hope to see you at our next Lunch n' Learn 🌯🍱🍜🍲