How to interpret your card sort results Part 1: open and hybrid card sorts

You’ve just finished running your card sort. The study has closed and the data is waiting to be analyzed. It’s time to take a look at the analysis side of card sorting, specifically in our tool OptimalSort. Let’s get started.

‍

A note on analysis 📌

‍

When it comes to analysis, there are essentially two types. There’s exploratory analysis (when you look through data to get impressions, pull out useful ideas and be creative) and statistical analysis (which really just comes down to the numbers). These two types of analysis also go by qualitative and quantitative, respectively.

‍

You’re able to get fantastic insights from both forms.

‍

“Remember that you are the one who is doing the thinking, not the technique… you are the one who puts it all together into a great solution. Follow your instincts, take some risks, and try new approaches.” Donna Spencer, Maadmob.

‍

Getting started with analysis 🏁

‍

Whenever you wrap up a study using our card sorting tool, you’ll want to kick off your analysis by heading to the Results Overview section. It’s here that you’ll be able to see how many people actually took part in the study, the average time taken and general statistics about the study itself.

‍

This is useful data to include in presentations to interested stakeholders, just to give them a more holistic view of your research.

‍

Digging into your participant data ⛏

‍

With the Results Overview section out of the way, you can make your way over to the Participants Table. This is where you can find information about the individual people who took part in your card sort. You can also start to filter your data here.

‍

Here are just a few of the different actions that you can take:

‍

• Review your participants, and include or exclude certain individuals based on their card sorts. This is a useful tool if you want to use your data in different ways.
• Segment and reload your results. This function can allow you to view data from individuals or groups of your choosing.
• Add additional card sorts. If you also decided to run manual (in-person) card sorts using printed cards, you can add this data here.

‍

Analysing open and hybrid card sort data 🕵️♂

‍

The Categories tab is the best place to go for open and hybrid card sort results. Take some time to scan the categories people came up with and you’ll be able to quickly build up a good understanding of their ‘mental models’, or how they perceived the theme of your cards.

‍

Consider how different the categories might look for cards containing food items, for example. Some participants might create categories reflecting supermarket aisles, while others might create categories reflecting food groups.

‍

A good place to get started here is by refining your data. Standardize any categories that have similar labels (whether that’s wording, spelling or capitalizations etc). Hybrid card sorts have some set categories, and these will already be standardized.

‍

Note: Before you start throwing categories with similar labels together, take a closer look to see if people had the same conceptual approach. Here’s an example from our card sorting 101 guide:

‍

Of the 15 groups with the word ‘Animal’ in the label, 13 had a similar set of cards, but two participants had labeled their categories slightly differently (Animals and Environment’ and ‘Animals and Nature’) and had thus included extra cards the others didn’t have (‘Glaciers melting faster than previously thought’, for example).

‍

Reviewing the Similarity Matrix 🤔

‍

One really useful tool for understanding how your participants think is the Similarity Matrix. This view shows you the percentage of people who grouped 2 cards together.

The most closely related pairings are clustered along the right edge. Higher agreement between participants on which cards go together equates to darker and larger clusters.

‍

‍

There are a few different ways to use the insights from the Similarity Matrix:

‍

• Put together a draft website structure based on the clusters you see on the right.
• Identify which card pairings are most common (and as a result should probably go together on your website).
• Identify which card pairings are least common so you don’t need to waste time considering how they might work on your website.

‍

Spotting popular card groupings 🔍

‍

Dendrograms are a tool to enable you to spot popular groups of cards, as well to get a general feel of how similar or different your participants’ card sorts were to each other.

‍

There are two dendrograms to explore:

‍

• More than 30 card sort participants: The Actual Agreement Method (AAM) dendrogram gives you the data straight: “X% of participants agree with this exact grouping”.
• Fewer than 30 card sort participants: The Best Merge Method (BMM) tells you “X% of participants agree with parts of this grouping”, and so enables you to extract as much as you can from the data.

‍

Looking for alternative approaches 👀

‍

The Participant-Centric Analysis (PCA) view can be useful when you have a lot of results. It’s quite simple. Basically, it aims to find the most popular grouping strategy, and then find two more popular alternatives among participants who agreed with the first strategy.

‍

This approach is called Participant-Centric Analysis because every response (from every participant) is treated as a potential solution, and then ranked for similarity with other responses. What this is telling you is that if you see a card sort with a 11/43 agreement score, this means 10 other participants sorted their cards into groups similar to these ones. 

‍

Taking the next step: Run a card sort and try analysis for yourself 🃏

‍

Now that we’ve taken a bit of a deep dive into the analysis side of card sorting in OptimalSort, it’s time to take the tool for a spin and start generating your own data.

‍

Getting started is easy. If you haven’t already, simply sign up for a free account (you don’t need a credit card) and start a card sort. You can also practice by creating a card sort and sending it out to your coworkers, friends or family. Once you start to see results trickling in, you can start to make sense of the data.

‍

For more information, check out the card sorting 101 guide that we’ve put together, or our introduction to card sorting on the Optimal Workshop Blog.

‍

Happy testing! 

We’re excited to announce our first six project templates are now available in Optimal Workshop! We understand that not everyone knows where to start with customer research, so these ready-made templates have been created with UX industry experts to give you the confidence to quickly launch studies and get the results you need to make data-driven decisions.  

‍

Templates aren’t only a great solution for people who need guidance with which study type to use and when; our detailed templates also give you the tools to develop your IA thinking, compare the performance of studies over time, and get detailed project plans to guide you through your information architecture. 

‍

‍

How do templates work?

‍

On the dashboard, you’ll see a new button called Browse Templates. From the templates menu, you can select a template that matches your use case, e.g. ‘I need to organise content into categories’. The templates are a helpful starting point, for you to adapt to suit your research goals. 

‍

Let’s take a look at some of our favourite project templates. 

‍

‍

Organize content into categories

‍

‍

This template helps you design the best categories to organize your information based on how your  users think. It's useful for designing your product, website, or knowledge base experience, as well as re-evaluating any part of it. In this template, we will first conduct an open card sort, and then use that information to design a navigation structure that will be tested on end users.

‍

‍

1. First up run a card sort with OptimalSort

‍

During this study, users will organise all information presented to them into categories they create themselves using an open card sort. This method is great for generating category ideas based on how users process this information allowing you to better design an experience in a more user-focused way. To find out more on how to set up your card sort, refer to our card sorting 101 guide.

‍

‍

2. Test your navigation structure with a Treejack

‍

Based on the groupings that were produced from the card sort, you can now generate a hierarchy for your users to test using Treejack. Users search for the information you’ve categorised and represented as a hierarchy, which is valuable because it helps to confirm whether information placed within your hierarchy is findable and understandable.  
‍

To learn more about tree testing, refer to our tree testing 101 guide.

‍

Evaluate an existing navigation experience 

‍

‍

Regularly evaluating an existing navigation experience is a good way to monitor the health and performance of your website and product. This template is useful for both redesigning your experience and for re-evaluating part of it by helping you design ideal categories to organize your information based on how your target users think and improve findability and task completion. 

‍

‍

1. Start by identifying your top tasks using Reframer

‍

Using Reframer, conduct interviews with various stakeholders in your business to evaluate and theme which tasks your organization believes are the most important within your existing environment. This is a solid first step towards building a list of top tasks for testing. Reframer allows you to easily visualize and group your observations by proximity using the affinity map.  

‍

‍

2. Survey users to understand their top tasks

‍

Next, survey users to confirm their top tasks and identify any existing issues with our survey tool Questions. This will provide insight into what users believe are their top tasks and whether anything is getting in their way to achieve them. This step helps to ensure all design work is informed by up-to-date user tasks.

‍

‍

3. Design and test your current experience in Treejack  

‍

Using the prioritised top tasks create a tree test using Treejack to test your navigation experience with your users. For example “How would you open a home loan” or “How would you upgrade your broadband plan” This will enable you to see how your users navigate your website in order to achieve the most business critical tasks in your organization. This is a valuable step that helps to identify information and design problems to solve early in the design process. 

‍

‍

More templates from our community

‍

This is just the beginning of templates in Optimal Workshop and while we continue to add value and build up our collection, we’d love your input! If there are templates that you regularly use and think the community could benefit from, let us know at hello@optimalworkshop.com.

Taylor Swift's music has captivated millions, but what do her fans really think about her extensive catalog? We've crunched the numbers, analyzed the data, and uncovered some fascinating insights into how Swifties perceive and categorize their favorite artist's work. Let's dive in!

‍

The great debate: openers, encores, and everything in between ⋆.˚✮🎧✮˚.⋆

‍

Our study asked fans to categorize Swift's songs into potential opening numbers, encores, and songs they'd rather not hear (affectionately dubbed "Nah" songs). The results? As diverse as Swift's discography itself!

‍

Opening with a bang 💥

‍

Swifties seem to agree that high-energy tracks make for the best concert openers, but the results are more nuanced than previously suggested. "Shake It Off" emerged as the clear favorite for opening a concert, with 17 votes. "Love Story" follows closely behind with 14 votes, showing that nostalgia indeed plays a significant role. Interestingly, both "Cruel Summer" and "Blank Space" tied for third place with 13 votes each.

‍

This mix of songs from different eras of Swift's career suggests that fans appreciate both her newer hits and classic favorites when it comes to kicking off a show. The strong showing for "Love Story" does indeed speak to the power of nostalgia in concert experiences. It's worth noting that "...Ready for It?", while a popular song, received fewer votes (9) for the opening slot than might have been expected.

‍

‍

Encore extravaganza 🎤

‍

When it comes to encores, fans seem to favor a diverse mix of Taylor Swift's discography, with a surprising tie at the top. "Slut!" (Taylor's Version), "exile", "Guilty as Sin?", and "Bad Blood (Remix)" all received the highest number of votes with 13 each. This variety showcases the breadth of Swift's career and the different aspects of her artistry that resonate with fans for a memorable show finale.

‍

Close behind are "evermore", "Wildest Dreams", "ME!", "Love Story", and "Lavender Haze", each garnering 12 votes. It's particularly interesting to see both newer tracks and classic hits like "Love Story" maintaining strong popularity for the encore slot. This balance suggests that Swifties appreciate both nostalgia and Swift's artistic evolution when it comes to closing out a concert experience.

‍

‍

The "Nah" list 😒

‍

Interestingly, some of Taylor Swift's tracks found themselves on the "Nah" list, indicating that fans might prefer not to hear them in a concert setting. "Clara Bow" tops this category with 13 votes, closely followed by "You're On Your Own, Kid", "You're Losing Me", and "Delicate", each receiving 12 votes.

‍

This doesn't necessarily mean fans dislike these songs - they might just feel they're not well-suited for live performances or don't fit as well into a concert setlist. It's particularly surprising to see "Delicate" on this list, given its popularity. The presence of both newer tracks like "Clara Bow" and older ones like "Delicate" suggests that the "Nah" list isn't tied to a specific era of Swift's career, but rather to individual song preferences in a live concert context.

‍

It's worth noting that even popular songs can end up on this list, highlighting the complex relationship fans have with different tracks in various contexts. This data provides an interesting insight into how Swifties perceive songs differently when considering them for a live performance versus general listening.

‍

‍

‍

The Similarity Matrix: set list synergies ⚡

‍

Our similarity matrix revealed fascinating insights into how fans envision Taylor Swift's songs fitting together in a concert set list:

‍

1. The "Midnights" Connection: Songs from "Midnights" like "Midnight Rain", "The Black Dog", and "The Tortured Poets Department" showed high similarity in set list placement. This suggests fans see these tracks working well in similar parts of a concert, perhaps as a cohesive segment showcasing the album's distinct sound.

‍

2. Cross-album transitions: There's an intriguing connection between "Guilty as Sin?" and "exile", with a high similarity percentage. This indicates fans see these songs from different albums as complementary in a live setting, potentially suggesting a smooth transition point in the set list that bridges different eras of Swift's career.

‍

3. The show-stoppers: "Shake It Off" stands out as dissimilar to most other songs in terms of placement. This likely reflects its perceived role as a high-energy, statement piece that occupies a unique position in the set list, perhaps as an opener, closer, or peak moment.

‍

4. Set list evolution: There's a noticeable pattern of higher similarity between songs from the same or adjacent eras, suggesting fans envision distinct segments for different periods of Swift's career within the concert. This could indicate a preference for a chronological journey through her discography or strategic placement of different styles throughout the show.

‍

5. Thematic groupings: Some songs from different albums showed higher similarity, such as "Is It Over Now? (Taylor's Version)" and "You're On Your Own, Kid". This suggests fans see them working well together in the set list based on thematic or emotional connections rather than just album cohesion.

‍

What does it all mean?! 💃🏼📊

‍

This card sort data paints a picture of an artist who continually evolves while maintaining certain core elements that define her work. Swift's ability to create cohesive album experiences, make bold stylistic shifts, and maintain thematic threads throughout her career is reflected in how fans perceive and categorize her songs. Moreover, the diversity of opinions on song categorization - with 59 different songs suggested as potential openers - speaks to the depth and breadth of Swift's discography. It also highlights the personal nature of music appreciation; what one fan sees as the perfect opener, another might categorize as a "Nah".

‍

In the end, this analysis gives us a fascinating glimpse into the complex web of associations in Swift's discography. It shows us not just how Swift has evolved as an artist, but how her fans have evolved with her, creating deep and sometimes unexpected connections between songs across her entire career. Whether you're a die-hard Swiftie or a casual listener, or a weirdo who just loves a good card sort, one thing is clear: Taylor Swift's music is rich, complex, and deeply meaningful to her fans. And with each new album, she continues to surprise, delight, and challenge our expectations.

‍

Conclusion: shaking up our understanding 🥤🤔

‍

This deep dive into the Swiftie psyche through a card sort reveals the complexity of Taylor Swift's discography and fans' relationship with it. From strategic song placement in a dream setlist to unexpected cross-era connections, we've uncovered layers of meaning that showcase Swift's artistry and her fans' engagement. The exercise demonstrates how a song can be a potential opener, mid-show energy boost, poignant closer, or a skip-worthy track, highlighting Swift's ability to create diverse, emotionally resonant music that serves various roles in the listening experience.

‍

The analysis underscores Swift's evolving career, with distinct album clusters alongside surprising connections, painting a picture of an artist who reinvents herself while maintaining a core essence. It also demonstrates how fan-driven analyses like card sorting can be insightful and engaging, offering a unique window into music fandom and reminding us that in Swift's discography, there's always more to discover. This exercise proves valuable whether you're a die-hard Swiftie, casual listener, or someone who loves to analyze pop culture phenomena.

‍

‍