From our experience with remote user testing and analysis, and from talking with our customers, we make the following recommendations for Optimal Workshop survey sizes.

These aren't technical limitations, but rather figures that will make your surveys manageable to analyse while providing the insights that you are looking for.

OptimalSort

Open card sort:

Minimum: 20
Optimum: 40
Maximum: 60

Closed card sort:

Minimum: 40
Optimum: 80
Maximum: 150

Treejack

Minimum: 40
Optimum: 120
Maximum: 400

For Treejack, we suggest an upper limit of 50 responses per task. If you have 30 tasks but you're only showing 15 to each participant, you'll need twice as many participants than if you only had 15 tasks total.

On that note, 15 is a good upper limit for the number of tasks to show each participant. In our experience, the more tasks you show, the more of the tree your participants will remember, which may cloud your results - they'll look better than they should because your participants have become familiar with your tree as they progressed.

Chalkmark

Minimum: 40
Optimum: 100
Maximum: 250

Inviting participants

As a rule of thumb, start out by inviting the recommended 'maximum' number. You'll find that some participants abandon before starting the survey, and you are likely to end up near the Optimum level. It's better to invite too few participants and then progressively send out more invites as you need them to push up the response numbers. This technique gives you the ability to create more surveys later (to iteratively improve your Treejack tree let's say), and you will still have an untapped pool of fresh participants that you can invite to the subsequent surveys.