Before we start: Terminology used in this article

Card: An individual card, e.g. “Bananas”

Category: A group of cards that a participant has categorised together:

• {Apples, Bananas, Grapes}

Pair: A category with two cards only:

• {Apples, Bananas}

Subset: A category that is wholly contained by another category, e.g:

• {Apples , Bananas}
• {Apples , Grapes}
• {Bananas, Grapes}

Are all subsets of:

• {Apples, Bananas, Grapes}

What do the OptimalSort dendrograms do?

Each of the OptimalSort dendrograms analyses participant responses and provides an interpretation of how the cards were categorised together. How the results are generated depends on the algorithm that is used. We currently provide two algorithms:

• Actual Agreement Method (AAM)
• Best Merge Method (BMM)

The example dendrograms below are rendered from the same data set.

Example AAM:

Example BMM:

Why does the percentage score decrease as the categories become clearer?

Generally, the bigger the category, the more people will disagree with it. That is to say, as the number of cards in a category gets larger, it is less and less likely that multiple participants will have created that exact combination of cards. However, while many users can agree on very small categories, this is not a very useful result.
Providing the full range of viable categories along with scores allows you to make an informed compromise between practical requirements and what the participants are telling you.

How does the Actual Agreement Method work? (Warning: Nerdy!)

The Actual Agreement Method algorithm counts each instance of a complete category from every participant. Each category with a non-zero score (a "real category") inherits the base score (i.e. Before inheritance) of all superset categories. The category with the highest score is taken, and all conflicting categories are eliminated.

When should I use the AAM?

This algorithm works best with a lot of participants, but it generally provides better results than the BMM algorithm. The scores that it provides tells you “X% of participants agree with this grouping”.

How does Best Merge work? (Warning: Nerdy!)

The Best Merge Method algorithm breaks each instance of a category from every participant down into their base pairs. The pair with the highest score is locked in. This repeats, and where the pair being locked in intersects with an existing locked category, it is agglomerated with this category. All subsets of this new category are eliminated.

When should I use BMM?

This algorithm can make the most out of limited participants. The scores that it provides tells you “X% of participants agree with parts of this grouping”. This is an important distinction (see below) and care is required when interpreting the results.

So, what's the most important difference between AAM and BMM?

If 10 participants sorted their cards {A,B},{C} and another 10 participants {A},{B,C}, a AAM algorithm will tell you that {A,B,C} is not a very good category because nobody created that exact combination of cards, while the BMM algorithm will tell you that {A,B,C} is quite a good category because every participant found it partially acceptable.

The two algorithms generate different insights into the results, but broadly speaking, BMM's ability to compromise and extrapolate helps you squeeze the most out of small or incomplete responses, while AAM gives you more definitive results overall, and also provides more useful results when you have a higher number of participants.