Optimal Sort - any limitation on num of responses?
Support Staff2 Posted by William on 11 Jul, 2011 11:32 PM
While we don't force an upper limit on Optimal Sort responses,
there are a few things to keep in mind:
The first thing I should point out is that in an open card
sorting exercise where participants are naming their own
categories, the more responses you have the harder it is to analyze
your results once the survey is finished.
Of course, this is going to depend on the number of cards you
have, and is somewhat mitigated in the case of a closed sort. But
if you make the assumption that participants will create between 5
and 10 categories in an open sort, that's potentially 20,000
category names to analyze.
We personally recommend testing between 50-200 people
(especially for your first study). Statistically, there's good
that the correlation between how your tested population groups the
cards and how the 'entire' population would group the cards quickly
approaches an upper limit where no further insight is necessarily
The other caveat I should mention is that while we have surveys
with 3000 participants in our database, they're very much the
exception rather than the rule. For the sake of safety, I'd
appreciate it if you could 'throttle' the users you send the survey
to, at least initially. We've never had 3000 users attempt to
complete a study at the same time, so I can't make any guarantees
about the load that might put on our system.
The number of responses you get will also affect how long the
results will take to load, but we've recently tested this
functionality and if you're using a modern browser I'm confident
you won't run into any problems loading your survey results.