Create a user research plan with these steps

In-person moderated user testing is a valuable part of any research project. Meaning you can see first-hand how your users interact with your prototypes and products. But in-person isn’t always a viable option. What to do if your project needs user testing but it’s just not possible to get in front of your users personally? 

‍

Let’s talk unmoderated user testing. This approach sidesteps the need to meet your participants face-to-face as it’s done entirely remotely, over the internet. By it’s very nature there are also considerable benefits to unmoderated user testing.

‍

What is unmoderated user testing? 💻👀

‍

In the most basic sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants complete the testing activity by themselves and in their own time. For the most part, everything else stays the same.

‍

The key differences are:

‍

• You can’t ask follow-up questions
• You can’t use low-fidelity prototypes
• You can’t support participants (beyond the initial instructions you send them).

‍

Is unmoderated user testing right for your research project?

‍

By nature, unmoderated user research does not include any direct interaction between the researcher and the study participants. This is really the biggest benefit and also the biggest drawback. 

‍

Benefits of unmoderated usability testing 👩🏻💻

‍

• Speed and turnaround  - As there is no need to schedule meetings with each participant, unmoderated testing is usually much faster to initiate and complete. Depending on the study, it may be possible to launch a study and receive results in just a few hours.

• Size of study (participant numbers) - Unmoderated user testing also allows you to collect feedback from dozens or even hundreds of users at the same time.

• Location (local and/or international) -Testing online removes reliance on participants being physically present for the testing which broadens the ability to make contact with participants within your country or across the globe. 

‍

If you’d like to know more about the benefits of unmoderated usability testing, take a look at our article five reasons you should consider unmoderated user testing.

‍

Limitations of unmoderated usability testing 🚧

‍

• Early-prototype testing is difficult without a moderator to explain and help participants recover from errors or limitations of the prototype.

• Participant behavior - Without a moderator, participants tend to be less engaged and behave less realistically in tasks that depend on imagination, decision-making, or emotional responses.

• Inability to ask follow-up questions - by not being in the testing with the participant, the facilitator can’t ask further questions to get a deeper understanding of the participant’s reasoning. As you can’t rely on human judgment through a moderator being in the room with the participants and the ability to adjust the test in the moment, unmoderated usability testing needs thorough up front planning.

‍

Because of these limitations unmoderated usability testing usually works best for evaluating live websites and apps or highly functional prototypes.  It’s great for testing activities that don’t require a lot of imagination or emotion from participants. Such as testing functionality or answering direct queries to do with your product.

‍

What’s involved when setting up unmoderated usability testing? 🤔💭

‍

1. Define testing goals

With any usability testing, it pays to define your goals before getting underway with setting up the software. What do you want to know from the participants? Goals vary from test to test. Understanding your goals upfront will help you to make the correct tool choice.

‍

2. Define your demographic

With a clear understanding of your goal, now it’s time to consider which participants are right for your study. Think about who they are, their demographic, and where they live. Are they new users or existing? Are they experts or novices?

‍

3. Selecting testing software

As unmoderated studies, are done remotely, the software used to faciliate the study plays a key role in ensuring you get useful results. Without a facilitator, the software must guide the participants through the session and record what happens. Take the time to test software and select one that is right for your study.

‍

4. Write your own tasks and questions

Think through your goals and what you want to achieve from the testing. Many of the unmoderated testing services include study templates with generic example tasks. Remember they are templates, and your tasks and questions should be specific to your particular study. Any task instructions guiding the participants should be clear and directive.

‍

5. Trial session

You’ve done all of the upfront work, now it’s time to test that it works, the software does what you expect and the instructions you have written can be followed. Doing a test run is crucial, especially with unmoderated usability testing, as there won’t be a facilitator in the testing to fix any problems.

‍

6. Recruit participants

Having defined your target audience and demographic, now is the time to recruit participants. Ensuring you have some control over the recruitment process is important, either through screening questions or recruiting your own. There are services that  recruit from a pool of willing participants. Thiscan be a great way to get a wide range of users.

‍

7. Analyze results

You are likely to accumulate a lot of data from your unmoderated testing. You’ll need a way to organize and analyze the data to derive insights that are valuable. Depending on the type of usability testing you do will vary the type of results. Quantitative testing gives data-driven results and direct answers. Whereas qualitative testing through audio or video recordings of participants’ actions or comments will need time to analyze and look at behavioral observations. 

‍

Wrap Up 🌯

‍

Unmoderated usability testing can be a good option for your study. It may not be right for all of your studies all of the time. While it can be quick to implement and often cheaper than moderated usability testing, it still requires time and planning to ensure you get the data insights you are looking for. Following a checklist can be a great way to ensure you approach your research methodically.

‍

Many organizations are aware that staying relevant essential for their success. This can mean a lot of things to different organizations. What it often means is coming up with plenty of new, innovative ideas and products to keep pace with the demands and needs of the marketplace. It also means keeping up with the expectations and needs of your users, which often means  shorter and shorter product development life cycle times.  While maintaining this pace can be daunting, it can also be seen as a strength, tightening up your processes and cutting out unnecessary steps.

‍

A vital part of developing new (or tweaking existing) products is considering the end user first. There really is no point in creating anything new if it isn’t meeting a need or filling a gap in the market. How can you make sure you are hitting the right mark? Ask your users.  We look into some of the key user research methods available to help you in your product development process.

‍

If you want to know more about how to fit research into your product development process, take a read here.

‍

What is user research? 👨🏻💻

‍

User experience (UX) research, or user research as it’s commonly referred to, is an important part of the product development process. Primarily, UX research involves using different research methods to gather qualitative and quantitative data and insights about how your users interact with your product. It is an essential part of developing, building, and launching a product that truly meets the needs, desires, and requirements of your users. 

‍

At its simplest, user research is talking to your users and understanding what they want and why. And using this to deliver what they need.

‍

How does user research fit into the product development process? 🧩🧩

‍

User research is an essential part of the product development process. By asking questions of your users about how your product works and what place it fills in the market, you can create a product that delivers what the market needs to those who need it. 

‍

Without user research, you could literally be firing arrows in the dark, or at the very best, working from a very internal organizational view based on assuming that what you believe users need is what they want. With user research, you can collect qualitative and quantitative data that clearly tells you where and what users would like to see and how they would use it.

‍

Investing in user research right at the start of the product development process can save the team and the organization heavy investment in time and money. With detailed data responses, your brand-new product can leapfrog many development hurdles, delivering a final product that users love and want to keep using. Firing arrows to hit a bullseye.

‍

What user research methods should we use? 🥺

‍

Qualitative ResearchMethods

Qualitative research is about exploration. It focuses on discovering things we cannot measure with numbers and typically involves getting to know users directly through interviews or observation.

‍

Usability Testing – Observational

One of the best ways to learn about your users and how they interact with your new product is to observe them in their own environment. Watch how they accomplish tasks, the order they do things, what frustrates them, and what makes the task easier and/or more enjoyable for your subject. The data can be collated to inform the usability of your product, improving intuitive design and what resonates with your users.

‍

Competitive Analysis

Reviewing products already on the market can be a great start to the product development process. Why are your competitors’ products successful? And how well do they behave for users? Learn from their successes, and even better, build on where they may not be performing as well and find where your product fills the gap in the market.

‍

Quantitative Research Methods

Quantitative research is about measurement. It focuses on gathering data and then turning this data into usable statistics.

‍

Surveys

Surveys are a popular user research method for gathering information from a wide range of people. In most cases, a survey will feature a set of questions designed to assess someone’s thoughts on a particular aspect of your new product. They’re useful for getting feedback or understanding attitudes, and you can use the learnings from your survey of a subset of users to draw conclusions about a larger population of users.

‍

Wrap Up 🌯

‍

Gathering information on your users during the product development process and before you invest time and money can be hugely beneficial to the entire process. Collating robust data and insights to guide the new product development and respond directly to user needs, and filling that all-important niche. Undertaking user experience research shouldn’t stop at product development but throughout each and every step of your product life cycle. If you want to find out more about UX research throughout the life cycle of your product, take a read of our article UX research for each product phase.

Knowing and understanding why and how your users use your product can be invaluable for getting to the nitty gritty of usability. Where they get stuck and where they fly through. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative.

‍

Usability testing can be done in several ways, each way has its benefits. Put super simply, usability testing literally is testing how useable your product is for your users. If your product isn't useable users will not stick around or very often complete their task, let alone come back for more.

‍

‍

What is usability testing? 🔦

‍

Usability testing is a research method used to evaluate how easy something is to use by testing it with representative users.

‍

These tests typically involve observing a participant as they work through a series of tasks involving the product being tested. Having conducted several usability tests, you can analyze your observations to identify the most common issues.

‍

We go into the three main methods of usability testing:

1. Moderated and unmoderated
2. Remote or in person
3. Explorative, assessment or comparative

‍

‍

1. Moderated or unmoderated usability testing 👉👩🏻💻

‍

Moderated usability testing is done in-person or remotely by a researcher who introduces the test to participants, answers their queries, and asks follow-up questions. Often these tests are done in real time with participants and can involve other research stakeholders. Moderated testing usually produces more in-depth results thanks to the direct interaction between researchers and test participants. However, this can be expensive to organize and run.

‍

Top tip: Use moderated testing to investigate the reasoning behind user behavior.

‍

Unmoderated usability testing is done without direct supervision; likely participants are in their own homes and/or using their own devices to browse the website that is being tested. And often at their own pace.  The cost of unmoderated testing is lower, though participant answers can remain superficial and making follow-up questions can be difficult.

‍

Top tip: Use unmoderated testing to test a very specific question or observe and measure behavior patterns.

‍

‍

2. Research or in-person usability testing 🕵

‍

Remote usability testing is done over the internet or by phone. Allowing the participants to have the time and space to work in their own environment and at their own pace. This however doesn’t give the researcher much in the way of contextual data because you’re unable to ask questions around intention or probe deeper if the participant makes a particular decision. Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources.

‍

Top tip: Use remote testing when a large group of participants are needed and the questions asked can be direct and unambiguous.

‍

In-person usability testing, as the name suggests, is done in the presence of a researcher. In-person testing does provide contextual data as researchers can observe and analyze body language and facial expressions. You’re also often able to converse with participants and find out more about why they do something. However, in-person testing can be expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and often pay) participants.

‍

Top tip: In-person testing gives researchers more time and insight into motivation for decisions.

‍

‍

3. Explorative, Assessment or comparative testing 🔍

‍

These three usability testing methods generate different types of information:

‍

Explorative testing is open-ended. Participants are asked to brainstorm, give opinions, and express emotional impressions about ideas and concepts. The information is typically collected in the early stages of product development and helps researchers pinpoint gaps in the market, identify potential new features, and workshop new ideas.

‍

Assessment research is used to test a user's satisfaction with a product and how well they are able to use it. It's used to evaluate general functionality.

‍

Comparative research methods involve asking users to choose which of two solutions they prefer, and they may be used to compare a product with its competitors.

‍

Top tip: Depending on what research is being done, and how much qualitative or quantitative data is wanted.

‍

‍

Which method is right for you? 🧐

‍

Whether the testing is done in-person, remote, moderated or unmoderated will depend on your purpose, what you want out of the testing, and to some extent your budget. 

‍

Depending on what you are testing, each of the usability testing methods we explored here can offer an answer. If you are at the development stage of a product it can be useful to conduct a usability test on the entire product. Checking the intuitive usability of your website, to ensure users can make the best decisions, quickly. Or adding, changing or upgrading a product can also be the moment to check on a specific question around usability. Planning and understanding your objectives are key to selecting the right usability testing option for your project.

‍

Let's take a look at a couple of examples of usability testing.

‍

‍

1. Lab based, in-person moderated testing - mid-life website

‍

Imagine you have a website that sells sports equipment. Over time your site has become cluttered and disorganized, much like a bricks and mortar store may. You’ve noticed a drop in sales in certain areas. How do you find out what is going wrong or where users are getting lost? Having an in-person, lab (or other controlled environment), moderated usability test with users you can set tasks, watch (and record) what they do.

‍

The researcher can literally be standing or sitting next to the participant throughout, recording contextual information such as how they interacted with the mouse, laptop or even the seat. Watching for cues as to the comfort of the participant and asking questions about why they make decisions can provide richer insights. Maybe they wanted purple yoga pants, but couldn’t find the ‘yoga’ section which was listed under gym rather than a clothing section.

‍

Meaning you can look at how your stock is organised, or even investigate undertaking a card sort. This provides robust and fully rounded feedback on users behaviours, expectations and experiences. Providing data that can directly be turned into actionable directives when redeveloping the website. 

‍

‍

2. Remote, moderated assessment testing - app product development

‍

You are looking at launching an app for parents to access for information and updates for the school. It’s still in development stage and at this point you want to know how easy the app is to use. Setting some very specific set tasks for participants to complete the app can be sent to them and they can be left to complete (or not). Providing feedback and comments around the usability.

‍

The next step may be to use first click testing to see how and where the interface is clicked and where participants may be spending time, or becoming lost. Whilst the feedback and data gathered from this testing can be light, it will be very direct to the questions asked. And will provide data to back up (or possibly not) what assumptions were made.

‍

‍

3. Moderated, In-person, explorative testing - new product development

‍

You’re right at the start of the development process. The idea is new and fresh and the basics are being considered. What better way to get an understanding of what your users’ truly want than an explorative study.

‍

Open-ended questions with participants in a one-on-one environment (or possibly in groups) can provide rich data and insights for the development team. Imagine you have an exciting new promotional app that you are developing for a client. There are similar apps on the market but none as exciting as what your team has dreamt up. By putting it (and possibly the competitors) to participants they can give direct feedback on what they like, love and loathe.

‍

They can also help brainstorm ideas or better ways to make the app work, or improve the interface. All of this done, before there is money sunk in development.

‍

‍

Wrap up 🌯

‍

Key objectives will dictate which usability testing method will deliver the answers to your questions.

‍

Whether it’s in-person, remote, moderated or comparative with a bit of planning you can gather data around your users very real experience of your product. Identify issues, successes and failures. Addressing your user experience with real data, and knowledge can but lead to a more intuitive product.