At its heart, PCA is a powerful way of seeing trends in your data, based upon the card sorts conducted by individual people in your study. You get to see the card sorts – the IAs – that other IAs are most like. So the ones shown in PCA are the ‘strongest’ in terms of agreement.
Behind the scenes this is what is happening:
The card parings that one person made are compared with the card pairings that another person made. You can pair a card more than once, of course – if there are three cards in a group you make then there are three parings (AB, AC, CB), so there’s usually a lot of pairings to compare.
When two different people’s card sorts contain 50% or more of the same card pairings, – regardless of the group they’ve placed the cards in – then the two IA ‘support’ each other.
By repeatedly checking these pairing matchings for all the people who did your study we can work out which of the IAs that participants made are the most ‘supported’.
The labels that people apply to their groupings are ignored when comparing the pairings, but they are separately analyzed and offered as suggestions for the categories against the most popular IAs.
In the example below, our first IA says ‘Similar IAs 37/50’., This means 37 participants out of a total of 50 paired the same cards together at least 50% of the time. The PCA is showing us this particular IA because it’s the one with the most ‘support’ from the other 36 participants. We could comfortably base our initial draft IAs for our website on a PCA result with this level of agreement. If you want to go and check the full IA of the most supported person, you can see their participant number (#16). Go to the Participants tab and check them out!
The other two IAs shown work in the same way. The three IAs are distinct from each other because they don’t support each other (as in, fewer than 50% of card pairings in one IA matches the card pairings in the other two.) If you have good agreement across each of the IAs, the PCA gives you three different ‘views’ on your data; three ideas of how you could arrange it in your eventual IA. A good way to think of this is; three possible trees to test with a tree test!
The PCA is therefore a very powerful way to get to the most compelling mental models at play and in combination with the similarity matrix can show the strength of opinion on card pairings.
If you see low levels of agreement for the 3 IAs (for example, 1/15 participant sorts were similar to this IA) this shows that none of the participants’ sorts are similar to each other. That is, each of the 15 participants have come up with different categories and grouped their cards in different ways.
To address this, your first approach might be to recruit more participants to get a better sample size and see if more people come up with similar card sorts. Otherwise, you can focus your analysis on the other visualizations, like the similarity matrix and the actual agreement method (AAM) dendrogram, but ultimately a lack of agreement might indicate that your cards were unsortable, i.e. they did not fit any sensible recognisable pattern for users. Remember, an important consideration for any card sort, according to Donna Spencer, is that the cards are sortable!