Optimal Blog

If there’s one universal truth in user research, it’s that at some point you’re going to need to find people to actually take part in your studies. Be it a large number of participants for quantitative research or a select number for in-depth, in-person user interviews. Finding the right people (and number) of people can be a hurdle.

‍

With the right strategy, you can source exactly the right participants for your next research project.

‍

We share a practical step-by-step guide on how to find participants for user experience research.

‍

The difficulties/challenges of user research recruiting 🏋️

‍

It has to be acknowledged that there are challenges when recruiting research participants. You may recognize some of these:

‍

• There are so many channels and methods you can use to find participants, different channels will work better for different projects.
• Repeatedly using the same channels and methods will result in diminishing returns (i.e. burning out participants).
• It’s a lengthy and complex process, and some projects don’t have the luxury of time.
• Offering the right incentives and distributing them is time-consuming.
• It’s hard to manage participants during long-term or recurring studies, such as customer research projects.

‍

We’ll simplify the process, talk about who the right participants are, and unpack some of the best ways to find them. Removing these blocks can be the easiest way to move forward.

‍

Who are the right participants for different types of research? 🤔

‍

1. The first step to a successful participant recruitment strategy is clarifying the goals of your user research and which methods you intend to use. Ask yourself:

• What is the purpose of our research?
• How do we plan to understand that?

‍

2. Define who your ideal research participant is. Who is going to have the answers to your questions?

‍

3. Work out your research recruitment strategy. That starts by understanding the differences between recruiting for qualitative and quantitative research.

‍

Recruiting for qualitative vs. quantitative research 🙋🏻

‍

Quantitative research recruiting is a numbers game. For your data analysis to be meaningful and statistically significant, you need a lot of data. This means you need to do a lot of research with a lot of people. When recruiting for quantitative research, you first have to define the population (the entire group you want to study). From there, you choose a sampling method that allows you to create a sample—a randomly selected subset of the population who will participate in your study.

‍

Qualitative recruiting involves far fewer participants, but you do need to find a selection of ‘perfect’ participants. Those that fit neatly into your specific demographic, geographic, psychographic, and behavioral criteria relevant to your study. Recruiting quality participants for qualitative studies involves non-random sampling, screening, and plenty of communication.

‍

How many participants do you need? 👱🏻👩👩🏻👧🏽👧🏾

‍

How many participants to include in a qualitative research study is one of the most heavily discussed topics in user research circles. In most cases, you can get away with 5 people – that’s the short answer. With 5 people, you’ll uncover most of the main issues with the thing you’re testing. Depending on your research project there could be as many as 50 participants, but with each additional person, there is an additional cost (money and time).

‍

Quantitative research is obviously quite different. With studies like card sorts and tree tests, you need higher participant numbers to get statistically meaningful results. Anywhere from 20 - 500 participants, again coming back to the purpose of your test and your research budget. These are usually easier and quicker to implement therefore the additional cost is lower.

‍

User research recruitment - step by step 👟

‍

Let’s get into your research recruitment strategy to find the best participants for your research project. There are 5 clear steps to get you through to the research stage:

‍

1. Identify your ideal participants

‍

Who are they? What do they do? How old are they? Do they already use your product? Where do they live? These are all great questions to get you thinking about who exactly you need to answer your research questions. The demographic and geographic detail of your participants are important to the quality of your research results.

‍

2. Screen participants

‍

Screening participants will weed out those that may not be suitable for your specific project. This can be as simple as asking if the participants have used a product similar to yours. Or coming back to your key identified demographic requirements and removing anyone that doesn’t fit these criteria.

‍

3. Find prospective participants

‍

This is important and can be time-consuming. For qualitative research projects, you can look within your organization or ask over social media for willing participants. Or if you’re short on time look at a participant recruitment service, which takes your requirements and has a catalog of available persons to call on. There’s a cost involved, but the time saving can negate this. For qualitative surveys, a great option can be a live intercept on your website or app that interrupts users and asks them to complete a short questionnaire.

‍

4. Research incentives

‍

In some cases you will need to provide incentives. This could be offering a prize or discount for those who complete online qualitative surveys. Or a fixed sum for those that take part in longer format quantitative studies.

‍

5. Scheduling with participants

‍

Once you have waded through the emails, options, and communication from your inquiries make a list of appropriate participants. Schedule time to do the research, either in person or remotely. Be clear about expectations and how long it will take. And what the incentive to take part is.

‍

Tips to avoid participant burnout 📛

‍

You’ve got your participants sorted and have a great pool of people to call on. If you keep hitting the same group of people time and time again, you will experience the law of diminishing returns. Constantly returning to the same pool of participants will eventually lead to fatigue. And this will impact the quality of your research because it’s based on interviewing the same people with the same views.

‍

There are 2 ways to avoid this problem:

‍

1. Use a huge database of potential participant targets.
2. Use a mixture of different recruitment strategies and channels.

‍

Of course, it might be unavoidable to hit the same audience repeatedly when you’re testing your product development among your customer base.

‍

Wrap up 🌯

‍

Understanding your UX research recruitment strategy is crucial to recruiting quality participants. A clear idea of your purpose, who your ideal participants are, and how to find them takes time and experience. 

‍

And to make life easier you can always leave your participant recruitment with us. With a huge catalog of quality participants all at your fingertips on our app, we can recruit the right people quickly.

‍

Check out more here.

Knowing and understanding why and how your users use your product can be invaluable for getting to the nitty gritty of usability. Where they get stuck and where they fly through. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative.

‍

Usability testing can be done in several ways, each way has its benefits. Put super simply, usability testing literally is testing how useable your product is for your users. If your product isn't useable users will not stick around or very often complete their task, let alone come back for more.

‍

‍

What is usability testing? 🔦

‍

Usability testing is a research method used to evaluate how easy something is to use by testing it with representative users.

‍

These tests typically involve observing a participant as they work through a series of tasks involving the product being tested. Having conducted several usability tests, you can analyze your observations to identify the most common issues.

‍

We go into the three main methods of usability testing:

1. Moderated and unmoderated
2. Remote or in person
3. Explorative, assessment or comparative

‍

‍

1. Moderated or unmoderated usability testing 👉👩🏻💻

‍

Moderated usability testing is done in-person or remotely by a researcher who introduces the test to participants, answers their queries, and asks follow-up questions. Often these tests are done in real time with participants and can involve other research stakeholders. Moderated testing usually produces more in-depth results thanks to the direct interaction between researchers and test participants. However, this can be expensive to organize and run.

‍

Top tip: Use moderated testing to investigate the reasoning behind user behavior.

‍

Unmoderated usability testing is done without direct supervision; likely participants are in their own homes and/or using their own devices to browse the website that is being tested. And often at their own pace.  The cost of unmoderated testing is lower, though participant answers can remain superficial and making follow-up questions can be difficult.

‍

Top tip: Use unmoderated testing to test a very specific question or observe and measure behavior patterns.

‍

‍

2. Research or in-person usability testing 🕵

‍

Remote usability testing is done over the internet or by phone. Allowing the participants to have the time and space to work in their own environment and at their own pace. This however doesn’t give the researcher much in the way of contextual data because you’re unable to ask questions around intention or probe deeper if the participant makes a particular decision. Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources.

‍

Top tip: Use remote testing when a large group of participants are needed and the questions asked can be direct and unambiguous.

‍

In-person usability testing, as the name suggests, is done in the presence of a researcher. In-person testing does provide contextual data as researchers can observe and analyze body language and facial expressions. You’re also often able to converse with participants and find out more about why they do something. However, in-person testing can be expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and often pay) participants.

‍

Top tip: In-person testing gives researchers more time and insight into motivation for decisions.

‍

‍

3. Explorative, Assessment or comparative testing 🔍

‍

These three usability testing methods generate different types of information:

‍

Explorative testing is open-ended. Participants are asked to brainstorm, give opinions, and express emotional impressions about ideas and concepts. The information is typically collected in the early stages of product development and helps researchers pinpoint gaps in the market, identify potential new features, and workshop new ideas.

‍

Assessment research is used to test a user's satisfaction with a product and how well they are able to use it. It's used to evaluate general functionality.

‍

Comparative research methods involve asking users to choose which of two solutions they prefer, and they may be used to compare a product with its competitors.

‍

Top tip: Depending on what research is being done, and how much qualitative or quantitative data is wanted.

‍

‍

Which method is right for you? 🧐

‍

Whether the testing is done in-person, remote, moderated or unmoderated will depend on your purpose, what you want out of the testing, and to some extent your budget. 

‍

Depending on what you are testing, each of the usability testing methods we explored here can offer an answer. If you are at the development stage of a product it can be useful to conduct a usability test on the entire product. Checking the intuitive usability of your website, to ensure users can make the best decisions, quickly. Or adding, changing or upgrading a product can also be the moment to check on a specific question around usability. Planning and understanding your objectives are key to selecting the right usability testing option for your project.

‍

Let's take a look at a couple of examples of usability testing.

‍

‍

1. Lab based, in-person moderated testing - mid-life website

‍

Imagine you have a website that sells sports equipment. Over time your site has become cluttered and disorganized, much like a bricks and mortar store may. You’ve noticed a drop in sales in certain areas. How do you find out what is going wrong or where users are getting lost? Having an in-person, lab (or other controlled environment), moderated usability test with users you can set tasks, watch (and record) what they do.

‍

The researcher can literally be standing or sitting next to the participant throughout, recording contextual information such as how they interacted with the mouse, laptop or even the seat. Watching for cues as to the comfort of the participant and asking questions about why they make decisions can provide richer insights. Maybe they wanted purple yoga pants, but couldn’t find the ‘yoga’ section which was listed under gym rather than a clothing section.

‍

Meaning you can look at how your stock is organised, or even investigate undertaking a card sort. This provides robust and fully rounded feedback on users behaviours, expectations and experiences. Providing data that can directly be turned into actionable directives when redeveloping the website. 

‍

‍

2. Remote, moderated assessment testing - app product development

‍

You are looking at launching an app for parents to access for information and updates for the school. It’s still in development stage and at this point you want to know how easy the app is to use. Setting some very specific set tasks for participants to complete the app can be sent to them and they can be left to complete (or not). Providing feedback and comments around the usability.

‍

The next step may be to use first click testing to see how and where the interface is clicked and where participants may be spending time, or becoming lost. Whilst the feedback and data gathered from this testing can be light, it will be very direct to the questions asked. And will provide data to back up (or possibly not) what assumptions were made.

‍

‍

3. Moderated, In-person, explorative testing - new product development

‍

You’re right at the start of the development process. The idea is new and fresh and the basics are being considered. What better way to get an understanding of what your users’ truly want than an explorative study.

‍

Open-ended questions with participants in a one-on-one environment (or possibly in groups) can provide rich data and insights for the development team. Imagine you have an exciting new promotional app that you are developing for a client. There are similar apps on the market but none as exciting as what your team has dreamt up. By putting it (and possibly the competitors) to participants they can give direct feedback on what they like, love and loathe.

‍

They can also help brainstorm ideas or better ways to make the app work, or improve the interface. All of this done, before there is money sunk in development.

‍

‍

Wrap up 🌯

‍

Key objectives will dictate which usability testing method will deliver the answers to your questions.

‍

Whether it’s in-person, remote, moderated or comparative with a bit of planning you can gather data around your users very real experience of your product. Identify issues, successes and failures. Addressing your user experience with real data, and knowledge can but lead to a more intuitive product.

There is an enormous amount of information on the internet. Every page you load is crammed with it. With so much information around, it’s incredibly important to remember that for human minds to be able to actually process and make sense of it, considered, strategic, ordered information architecture is needed.

‍

Information architecture is everywhere digitally, but where does it actually live? IA sits in behind all of the digital interfaces we humans use (and even out in the real world). It ideally has been considered, researched, and implemented with humans in mind, though as we all know, this isn’t always the case.  

‍

Information architecture always has a role to play, and here we’ll focus on 4 key digital spaces where information architecture is most prominent, and likely most familiar.

‍

1. Websites
2. Apps
3. Company intranet
4. Social media software

‍

To find out more about putting information architecture into action check out our blog.

‍

1. Websites 🌐

‍

‍

Information architecture for websites can often be confused with navigation. While site navigation is super important, (think of it as the ‘skin’ that sits above the information), that’s only part of the story. Great website architecture considers and organizes user requirements, organizing the content and structure of the website with labels, search and navigation that simply makes sense and is easy to use. Done well, it will  feel simple and straightforward, guiding users through smoothly.

‍

Hubspot has a great article on IA for websites and how they can be structured. From a complex library system, to travel booking systems. Each has its own needs and can be quite deep in layers of information. The trick is to create a website structure, based on user needs, that makes sense of the piles of information and a simple navigation that sits above, making the journey to task completion as quick as possible. Working together they make any user experience feel quick, simple, and intuitive! 

‍

2. Apps 📱

‍

‍

An organization's app acts as the first point of contact. It needs to be super simple, clean, and quick to interact with. A well thought out, thorough, researched, and organized information architecture plays a big part in this.

‍

Information architecture for mobile use has a different set of rules than websites. The key consideration here is around the ease of use across a smaller screen. Navigation that makes sense for a laptop or desktop computer can be clunky in a mobile app.

‍

An app's interface needs to have fewer options, and fewer clicks to complete the task. Researching and designing an app’s IA with just the right amount of information is key. Some retail apps are simplified websites, whereas other apps exist on their own merits, with no need for a website.

‍

Bringing it back to the humans that will interact with the app is key to creating a product that delivers on user requirements and increases interaction. 

‍

3. Intranet 🖥

‍

‍

An organizational intranet is possibly less of a priority than a website or an app but is vitally important to the success of an organization. And in these times of remote or hybrid working, intranets have proven to be more valuable than ever. An intranet is more than where to stick newsletters for staff, it is an interface that can make or break the productivity or even the wellbeing of an organization. 

‍

Access to files, information, messaging platforms, and corporate requirements wherever and whenever people are working is more important than ever.  A well researched and designed information architecture can build an intranet that meets users’ requirements, increase communication and interaction and ultimately boost productivity. 

‍

Conducting UX research with staff on what they need to access, when, and how will help inform the intranet IA far more intuitively. The information available (and needed) can be huge, keeping it simple and human focused is key.

‍

Have you ever thought about how poor internal information architecture might be hurting your business?

‍

4. Social media 🤳

‍

‍

Social media software is complex in terms of ecosystem and display of information. Each social media platform has developed over time, think back to Facebook and how it looked when it initially launched. And through the uptake of users, gathering of information over time, and continual research and testing, it has evolved into what we see today. And will continue to evolve with users’ needs.

‍

Every user has a different experience based on the individuals, groups, organizations and even retailers that they choose to interact with (or haven’t chosen via advertising). The piles of information that sit behind and are brought to the interface for an individual through their choice, associations, labels, tags, interests, age, etc will present them with a unique feed. 

‍

The interface of a social media platform needs to be considered, tested, tested again, and occasionally tested or changed once launched as the interaction of users is vitally important.

‍

The information architecture sitting behind is huge in order to enable the agility to pull the right information forward in a dynamic and coherent way. Continuously learning, testing and requesting interaction from users through options to ‘hide’ posts that aren’t appropriate or respond to direct queries about what they do and don’t want to see are just some instances of continuous user research.

‍

Social media continues to be a sophisticated information architecture that is constantly updating and changing with user needs.

‍

Wrap Up 🫔

‍

Information architecture lives wherever there is information needing to be found by humans. Successful information architecture is sorted, organized and labeled in a way that is simple, intuitive, and considered. Making interaction and life simple, which in a world where there are an increasing number of websites, apps, and tools to choose from – intuitive information architecture has never been more important for your business and your customers.

Making a difference through UX was a shared passion among an impressive line-up of 7 researchers, strategists, and designers from the global UX community at this year’s 100% virtual 3-day UX New Zealand conference.

‍

‍

1. From bombs to bots: the evolving landscape of frontline research

‍

These days Darya Pilram, Senior Researcher at Twitter, spends her days trying to understand the motivation and techniques of groups who ‘hire’ technology to spread harmful narratives.  The desert of Mogadishu and the urban conflicts of South Africa are just some of the unlikely places she’s leveraged the power of frontline research to create change.

‍

"I realized the only way to influence change was by bringing folks along with me - and so I did.  I bought them right into the field with me."

‍

‍

‍

‍

2. Beautifully accessible: why embracing inclusive design shouldn’t hold back your creativity

‍

Experience Designer Beth McPhail refuses to buy into the mindset that ‘accessibility is a creativity killer’. She challenges her peers to view accessibility as an opportunity to grow creatively while making technology more inclusive.

‍

“Accessibility is making it possible for someone to attend the party…and lose themselves in the music.”

‍

‍

‍

‍

3. Innovating within the Justice sector | Part 2: For a fairer start - design’s role in shaping mana enhancing social & systemic change.

‍

Kelsey Gee is back challenging designers across all levels to think differently about how design can be used across different mediums and constraints to generate meaningful experiences and meaningful change.  In this session, she explores design’s role in creating empowering experiences that break both cycles of crime and institutional racism. (If you missed Part 1 from Mini Con head over here)

‍

"I truly believe that our superpower lies in our ability to redesign society, especially for our whanau and our most vulnerable communities…and once again explore design’s role in creating equal opportunities across safe, seamless, and healing public services."

‍

‍

‍

‍

4. First do no harm: make your designs more trauma-informed and survivor sensitive

‍

In 1985, a researcher botched an interview question which led to a new understanding of trauma and its long-term effects. It grew awareness of the need to be trauma-informed in your work but what’s it actually mean?  UX Researcher Melissa Eggleston explores what it means to be trauma-informed and shares practical advice on how to achieve it.

‍

"Trauma is everywhere and something for us to think about…regardless of whether we’re working with people we know are dealing with traumatic events…it’s really all over the place."

‍

‍

‍

5. Changing the way we design high-risk products to make meaningful impact

‍

One in five people experiences “mental illness or significant mental distress” in New Zealand.  It’s a problem the Government knows needs to be addressed but how? In her powerful presentation, Rachael Reeves reveals what’s involved in balancing the complexities of Government with the need to rethink the way we design health products.

‍

"Be warned you can’t please everyone and it can be tough to keep product vision aligned when you’re talking about serious consequences for people."

‍

‍

‍

6. Remote research with new internet users (yes you can!)

‍

One billion new internet users (NIU) will come online for the first time over the next 5 years. These NIU's are using their first smartphones, with most of their online activities focused on communication, maintaining social connections, and entertainment. Tiane Lee, UX Research Lead at Google outlines the challenges and considerations behind adapting research for varying levels of digital literacy, including practical ideas for planning and conducting remote research with NIU.

‍

"NIU’s are typically less digitally literate, they may show lower confidence in digital capability, and they may perceive lower value of the internet for things like chatting and entertainment.”

‍

‍

‍

‍

7. Conditions Design: weaving the invisible threads of service design, value orchestration, and culture building  

‍

Michael Tam introduces us to the niche field of conditions design and cites a purpose built high diving board on Wellington city’s busy waterfront in New Zealand as a good example of conditions design.  Find out why in this fascinating talk.

‍

"What really impressed me here…hats off to the council because they didn’t design an experience that would discourage people from doing it. It’s designed for people to have fun (vs Hong Kong where public spaces are designed for Tai Chi not fun like this). The design allows it to happen by influencing human behavior to stay safe but encouraging fun and exploration.”

‍

‍

‍

‍

For a taste of what even more speakers from UX New Zealand 2022 had to share, head over to our highlights reel

In the crowded and complex world we live in, great user experiences can make all the difference to the way we navigate day-to-day life.

‍

CRUX#5 reveals the many faces of information architecture at work and its positive impact in the fields of business, entertainment, education, and more. Our enthusiastic and knowledgeable contributors generously paint a picture of the challenges, opportunities and benefits of harnessing the power of IA in their world.

‍

In this issue

‍

• Author/teacher and games user research expert Steve Bromley reveals some fascinating, even surprising insights from the niche world of UX for video games.

• Product Designer Katie Del Angel from Shopify shares some of her favorite ways to adopt an IA mindset when designing digital products.

• Digital interactive designer Kate Berry talks information architecture according to the world of visitor experience

• Chantal Schweizer from Earley Information Science dives into the all important area of taxonomies and the merits of following best practice to help drive success in the world of eCommerce

• And UX Researcher Rebecca Klee shares five approaches to increase the alignment and impact of your IA research.

‍

As always we're grateful to all our contributors for helping shine a light on the fascinating, even addictive world of UX.  They certainly captured our imagination and attention - we hope they inspire and even surprise you too. 

‍

Welcome to CRUX#5

‍

P.S.  We’re already on the lookout for contributors for our next edition of CRUX for 2022.  To find out more please drop us a line.

Return on investment (ROI) is often the term on everyone’s lips when starting a big project or even when reviewing a website. It’s especially popular with those that hold the purse strings.  As UX researchers it is important to consider the ROI of the work we do and understand how to measure this. 

‍

We’ve lined up 5 key ways to measure ROI for UX research to help you get the conversation underway with stakeholders so you can show real and tangible benefits to your organization. 

‍

‍

1. Meet and exceed user expectations

‍

Put simply, a product that meets and exceeds user expectations leads to increased revenue. When potential buyers are able to find and purchase what they’re looking for, easily, they’ll complete their purchase, and are far more likely to come back. The simple fact that users can finish their task will increase sales and improve overall customer satisfaction which has an influence on their loyalty. Repeat business means repeat sales. Means increased revenue.

‍

Creating, developing and maintaining a usable website is more important than you might think. And this is measurable! Tracking and analyzing website performance prior to the UX research and after can be insightful and directly influenced by changes made based on UX research.

‍

Measurable: review the website (product) performance prior to UX research and after changes have been made. The increase in clicks, completed tasks and/or baskets will tell the story.

‍

‍

2. Reduce development time

‍

UX research done at the initial stages of a project can lead to a reduction in development time of by 33% to 50%! And reduced time developing, means reduced costs (people and overheads) and a speedier to market date. What’s not to love? 

‍

Measurable: This one is a little more tricky as you have saved time (and cost) up front. Aiding in speed to market and performance prior to execution. Internal stakeholder research may be of value post the live date to understand how the project went.

‍

‍

3. Ongoing development costs

‍

And the double hitter? Creating a product that has the user in mind up front, reduces the need to rehash or revisit as quickly. Reducing ongoing costs. Early UX research can help with the detection of errors early on in the development process. Fixing errors after development costs a company up to 100 times more than dealing with the same error before development.

‍

Measureable: Again, as UX research has saved time and money up front this one can be difficult to track. Though depending on your organization and previous projects you could conduct internal research to understand how the project compares and the time and cost savings.

‍

‍

4. Meeting user requirements

‍

Did you know that 70% of projects fail due to the lack of user acceptance? This is often because project managers fail to understand the user requirements properly. Thanks to UX research early on, gaining insights into users and only spending time developing the functions users actually want, saving time and reducing development costs. Make sure you get confirmation on those requirements by iterative testing. As always, fail early, fail often. Robust testing up front means that in the end, you’ll have a product that will meet the needs of the user.

‍

Measurable: Where is the product currently? How does it perform? Set a benchmark up front and review post UX research. The deliverables should make the ROI obvious.

‍

‍

5. Investing in UX research leads to an essential competitive advantage.

‍

Thanks to UX research you can find out exactly what your customers want, need and expect from you. This gives you a competitive advantage over other companies in your market. But you should be aware that more and more companies are investing in UX while customers are ever more demanding, their expectations continue to grow and they don’t tolerate bad experiences. And going elsewhere is an easy decision to make.

‍

Measurable: Murky this one, but no less important. Knowing, understanding and responding to competitors can help keep you in the lead, and developing products that meet and exceed those user expectations.

‍

‍

Wrap up

‍

Showing the ROI on the work we do is an essential part of getting key stakeholders on board with our research. It can be challenging to talk the same language, ultimately we all want the same outcome…a product that works well for our users, and delivers additional revenue.

‍

For some continued reading (or watching in this case), Anna Bek, Product and Delivery Manager at Xplor explored the same concept of "How to measure experience" during her UX New Zealand 2020 – watch it here as she shares a perspective on UX ROI.