What gear do I need for qualitative user testing?

Summary: The equipment and tools you use to run your user testing sessions can make your life a lot easier. Here’s a quick guide.

‍

It’s that time again. You’ve done the initial scoping, development and internal testing, and now you need to take the prototype of your new design and get some qualitative data on how it works and what needs to be improved before release. It’s time for the user testing to begin.

‍

But the prospect of user testing raises an important question, and it’s one that many new user researchers often deliberate over: What gear or equipment should I take with me? Well, never fear. We’re going to break down everything you need to consider in terms of equipment, from video recording through to qualitative note-taking.

‍

Recording: Audio, screens and video

‍

The ability to easily record usability tests and user interviews means that even if you miss something important during a session, you can go back later and see what you’ve missed. There are 3 types of recording to keep in mind when it comes to user research: audio, video and screen recording. Below, we’ve put together a list of how you can capture each. You shouldn’t have to buy any expensive gear – free alternatives and software you can run on your phone and laptop should suffice.

‍

• Audio – Forget dedicated sound recorders; recording apps for smartphones (iOS and Android) allow you to record user interviews and usability tests with ease and upload the recordings to Google Drive or your computer. Good options include Sony’s recording app for Android and the built-in Apple recording app on iOS.
• Transcription – Once you’ve created a recording, you’ll no doubt want a text copy to work with. For this, you’ll need transcription software to take the audio and turn it into text. There are companies that will make transcriptions for you, but software like Transcribe means you can carry out the process yourself.
• Screen recording – Very useful during remote usability tests, screen recording software can show you exactly how participants react to the tasks you set out for them, even if you’re not in the room. OBS Studio is a good option for both Mac and Windows users. You can also use Quicktime (free) if you’re running the test in person.
• Video – Recording your participants as they make their way through the various tasks in a usability test can provide useful reference material at the end of your testing sessions. You can refer back to specific points in a video to capture any detail you may have missed, and you can share video with stakeholders to demonstrate a point. If you don’t have access to a dedicated camera, consider mounting your smartphone on a tripod and recording that way.

‍

Taking (and making use of) notes

‍

Notetaking and qualitative user testing go hand in hand. For most user researchers, notetaking during a research session means busting out the Post-it notes and Sharpie pens, rushing to take down every observation and insight and then having to arduously transcribe these notes after the session – or spend hours in workshops trying to identify themes and patterns. This approach still has merit, as it’s often one of the best ways to get people who aren’t too familiar with user research involved in the process. With physical notes, you can gather people around a whiteboard and discuss what you’re looking at. What’s more, you can get them to engage with the material directly.

‍

But there are digital alternatives. Qualitative notetaking software (like our very own Reframer) means you can bring a laptop into a user interview and take down observations directly in a secure environment. Even better, you can ask someone else to sit in as your notetaker, freeing you up to focus on running the session. Then, once you’ve run your tests, you can use the software for theme and pattern analysis, instead of having to schedule yet another full day workshop.

‍

Scheduling your user tests

‍

Ah, participant scheduling. Perhaps one of the most time-consuming parts of the user testing process. Thankfully, software can drastically reduce the logistical burden.

Here are some useful pieces of software:

‍

Dedicated scheduling tool Calendly is one of the most popular options for participant scheduling in the UX community. It’s really hands-off, in that you basically let the tool know when you’re available, share the Calendly link with your prospective participants, and then they select a time (from your available slots) that works for them. There are also a host of other useful features that make it a popular option for researchers, like integrations and smart timezones.

‍

If you’re already using the Optimal Workshop platform, you can use our  survey tool Questions as a fairly robust scheduling tool. Simply set up a study and add in prospective time slots. You can then use the multi-choice field option to have people select when they’re available to attend. You can also capture other data and avoid the usual email back and forth.

‍

Storing your findings
‍

One of the biggest challenges for user researchers is effectively storing and cataloging all of the research data that they start to build up. Whether it’s video recordings of usability tests, audio recordings or even transcripts of user interviews, you need to ensure that your data is A) easily accessible after the fact, and B) stored securely to ensure you’re protecting your participants.

‍

Here are some things to ask yourself when you store any piece of customer or user data:

‍

• Who will have access to this data?
• How long do I plan to keep this data?
• Will this data be anonymized?
• If I’m keeping physical data on hand, where will it be stored?

‍

Don’t make the mistake of thinking user data is ‘secure enough’, whether that’s on a company server that anyone can access, or even in an unlocked filing cabinet beneath your desk. Data privacy and security should always be at the top of your list of considerations. We won’t dive into best practices for participant data protection in this article, but instead, just mention that you need to be vigilant. Wherever you end up storing information, make sure you understand who has access.

‍

Wrap up

‍

Hopefully, this guide has given you an overview of some of the tools and software you can use before you start your next user test. We’ve also got a number of other interesting articles that you can read right here on our blog.

Every month we have fun and informative “bite sized” presentations to add some inspiration to your lunch break.  These virtual events allow us to partner with amazing speakers, community groups and organizations to share their insights and hot takes on a variety of topics impacting our industry. 

‍

Join us at the end of every month for Lunch n' Learn.

‍

‍

Susanna Carman

‍

Leading design processes amidst a world in transition requires all practitioners to continuously invest in their own development. One aspect worth investing in, is an ability to integrate holistic thinking into our design leadership practice. This includes re-evaluating our own biases and how that bias is reflected in the tools we choose to work with when understanding and designing for/within complex systems.

‍

Recently our guest Susanna Carman, Strategic Designer and founder of Transition Leadership LAB, introduced us to a holistic approach to qualitative design research using Ken Wilber’s 4 Quadrant Model. Susanna explained the fundamental principles underpinning the framework, and showed how it can be used to ensure a multi-perspectival harvest of critical qualitative and quantitative data on any design project.  

‍

‍

‍

Speaker bio

‍

Susanna Carman is a Strategic Designer and research-practitioner who helps people solve complex problems, the types of problems that have to do with services, systems and human interactions. Specialising in design, leadership and learning, Susanna brings a high value toolkit and herself as Thinking Partner to design, leadership and change practitioners who are tasked with delivering sustainable solutions amidst disruptive conditions. 

‍

Susanna holds a Masters of Design Futures degree from RMIT University, and has over a decade of combined experience delivering business performance, cultural alignment and leadership development outcomes to the education, health, community development and financial services sectors. She is also the founder and host of Transition Leadership Lab, a 9-week learning lab for design, leadership and change practitioners who already have a sophisticated set of tools and mindsets, but still feel these are insufficient to meet the challenge of leading change in a rapidly transforming world.

‍

Grab your lunch, invite your colleagues and we hope to see you at our next Lunch n' Learn 🌯🍱🍜🍲

If there’s one universal truth in user research, it’s that at some point you’re going to need to find people to actually take part in your studies. Be it a large number of participants for quantitative research or a select number for in-depth, in-person user interviews. Finding the right people (and number) of people can be a hurdle.

‍

With the right strategy, you can source exactly the right participants for your next research project.

‍

We share a practical step-by-step guide on how to find participants for user experience research.

‍

The difficulties/challenges of user research recruiting 🏋️

‍

It has to be acknowledged that there are challenges when recruiting research participants. You may recognize some of these:

‍

• There are so many channels and methods you can use to find participants, different channels will work better for different projects.
• Repeatedly using the same channels and methods will result in diminishing returns (i.e. burning out participants).
• It’s a lengthy and complex process, and some projects don’t have the luxury of time.
• Offering the right incentives and distributing them is time-consuming.
• It’s hard to manage participants during long-term or recurring studies, such as customer research projects.

‍

We’ll simplify the process, talk about who the right participants are, and unpack some of the best ways to find them. Removing these blocks can be the easiest way to move forward.

‍

Who are the right participants for different types of research? 🤔

‍

1. The first step to a successful participant recruitment strategy is clarifying the goals of your user research and which methods you intend to use. Ask yourself:

• What is the purpose of our research?
• How do we plan to understand that?

‍

2. Define who your ideal research participant is. Who is going to have the answers to your questions?

‍

3. Work out your research recruitment strategy. That starts by understanding the differences between recruiting for qualitative and quantitative research.

‍

Recruiting for qualitative vs. quantitative research 🙋🏻

‍

Quantitative research recruiting is a numbers game. For your data analysis to be meaningful and statistically significant, you need a lot of data. This means you need to do a lot of research with a lot of people. When recruiting for quantitative research, you first have to define the population (the entire group you want to study). From there, you choose a sampling method that allows you to create a sample—a randomly selected subset of the population who will participate in your study.

‍

Qualitative recruiting involves far fewer participants, but you do need to find a selection of ‘perfect’ participants. Those that fit neatly into your specific demographic, geographic, psychographic, and behavioral criteria relevant to your study. Recruiting quality participants for qualitative studies involves non-random sampling, screening, and plenty of communication.

‍

How many participants do you need? 👱🏻👩👩🏻👧🏽👧🏾

‍

How many participants to include in a qualitative research study is one of the most heavily discussed topics in user research circles. In most cases, you can get away with 5 people – that’s the short answer. With 5 people, you’ll uncover most of the main issues with the thing you’re testing. Depending on your research project there could be as many as 50 participants, but with each additional person, there is an additional cost (money and time).

‍

Quantitative research is obviously quite different. With studies like card sorts and tree tests, you need higher participant numbers to get statistically meaningful results. Anywhere from 20 - 500 participants, again coming back to the purpose of your test and your research budget. These are usually easier and quicker to implement therefore the additional cost is lower.

‍

User research recruitment - step by step 👟

‍

Let’s get into your research recruitment strategy to find the best participants for your research project. There are 5 clear steps to get you through to the research stage:

‍

1. Identify your ideal participants

‍

Who are they? What do they do? How old are they? Do they already use your product? Where do they live? These are all great questions to get you thinking about who exactly you need to answer your research questions. The demographic and geographic detail of your participants are important to the quality of your research results.

‍

2. Screen participants

‍

Screening participants will weed out those that may not be suitable for your specific project. This can be as simple as asking if the participants have used a product similar to yours. Or coming back to your key identified demographic requirements and removing anyone that doesn’t fit these criteria.

‍

3. Find prospective participants

‍

This is important and can be time-consuming. For qualitative research projects, you can look within your organization or ask over social media for willing participants. Or if you’re short on time look at a participant recruitment service, which takes your requirements and has a catalog of available persons to call on. There’s a cost involved, but the time saving can negate this. For qualitative surveys, a great option can be a live intercept on your website or app that interrupts users and asks them to complete a short questionnaire.

‍

4. Research incentives

‍

In some cases you will need to provide incentives. This could be offering a prize or discount for those who complete online qualitative surveys. Or a fixed sum for those that take part in longer format quantitative studies.

‍

5. Scheduling with participants

‍

Once you have waded through the emails, options, and communication from your inquiries make a list of appropriate participants. Schedule time to do the research, either in person or remotely. Be clear about expectations and how long it will take. And what the incentive to take part is.

‍

Tips to avoid participant burnout 📛

‍

You’ve got your participants sorted and have a great pool of people to call on. If you keep hitting the same group of people time and time again, you will experience the law of diminishing returns. Constantly returning to the same pool of participants will eventually lead to fatigue. And this will impact the quality of your research because it’s based on interviewing the same people with the same views.

‍

There are 2 ways to avoid this problem:

‍

1. Use a huge database of potential participant targets.
2. Use a mixture of different recruitment strategies and channels.

‍

Of course, it might be unavoidable to hit the same audience repeatedly when you’re testing your product development among your customer base.

‍

Wrap up 🌯

‍

Understanding your UX research recruitment strategy is crucial to recruiting quality participants. A clear idea of your purpose, who your ideal participants are, and how to find them takes time and experience. 

‍

And to make life easier you can always leave your participant recruitment with us. With a huge catalog of quality participants all at your fingertips on our app, we can recruit the right people quickly.

‍

Check out more here.