Your 101 guide to running effective usability tests
Set up your study
Conducting your test
Analysing your data
Jump to a section
- Decide how many participants to recruit
- Source your own participants
- Source participants through recruiters and panels
- Incentivize participants
When you recruit participants, it’s important to find people who are representative of your end users to ensure the tasks and scenarios make sense to them. For example, if you’re testing the usability of a new baby monitor, you’ll get more reliable results testing with soon-to-be or existing parents, rather than members of the general public who may or may not have children of their own.
You can recruit participants in a variety of different ways, and how you do so will depend on a few different factors such as your budget, company policy, resources and time.
Ask yourself the following questions before recruiting:
- How will you keep track of your participants?
- How will you protect participant privacy?
- How will you handle their personal data?
There’s no correct answer to these questions — it will depend on your company’s policies and how your company handles data. As a rule of thumb, only collect the essential information you need to get your research done.
Decide how many participants to recruit
The number of people you recruit will largely depend on the time and resources available to you, as well as the availability of your target participants.
Aim for a number that will allow you to start seeing patterns in your data. Avoid making any decisions based on what one person said or did, unless they’ve come across a bug or usability issue that blocks people from completing a task successfully.
Try and include at least five participants in each round of testing. Add more if you continue to discover issues that are unique to each participant, until you start to see patterns emerge.
If you’re working with personas and have several different types of tasks that each user type may perform, you’ll want to recruit five participants from each user group to get reliable results.
Source your own participants
If you have access to a pool of participants (like employees if you’re working on an internal product, or your customer mailing list) then reaching out to potential participants via email can be an easy way to source them.
Similarly, you can recruit participants by advertising on social media channels or by running banners or pop ups on your website.
When sourcing participants for usability tests, you may want to direct them to a screener, depending on your needs. This is to ensure you’re reaching people who meet the specific criteria you’re looking for. What you decide to include in your screener will vary depending on your objectives.
Some useful things to include may be demographic information, previous experiences using your product or service, frequency of use, and anything else that will help you match your candidates against any target-user characteristics.
Source participants through recruiters and panels
You can also make use of recruitment providers, which can be a good option if you don’t have easy access to the people you’d like to target. Dedicated recruiters will generally have access to a wide variety of potential participants, which can make this a fast and easy way to source relevant people with minimal effort.
In some cases, they’ll also oversee logistics, such as scheduling sessions and paying out incentives on your behalf. Keep in mind that recruiters and panels come at a higher price than sourcing participants on your own.
Make sure you provide your recruiter with a clear understanding of the participant characteristics you’re looking for – similar to a screening questionnaire.
An alternative to using recruiting agencies is sourcing participants on your own from online panels. These self-service panels contain databases of potential participants, allowing you to filter, match and submit orders based on your own criteria without relying on a dedicated recruiter.
To encourage potential participants to take part in your research, you’ll want to consider how you incentivize them. While you can get away with little to no incentive for internal participants (for example, existing users within your organization), external participants may be less obligated to participate if they’re not reimbursed for their time.
Incentives can be monetary in the form of cash or gift cards, or take other forms, such as company merchandise or free access to your product or service.
The amount you decide to pay your participants will depend on a few different factors. Consider the length of the session, the complexity of the tasks and whether you need participants with specific experience or skills. For example, if you’re testing a specialist medical interface with doctors, the incentive you offer should be higher than if you’re testing a banking app that has a wider range of users.
An incentive should be high enough to motivate people to participate in your study. A good starting point is to consider how much a participant would usually receive for an hour of their time and add in extra to cover the additional effort required for participating.