Affinity mapping - an introduction

In-person moderated user testing is a valuable part of any research project. Meaning you can see first-hand how your users interact with your prototypes and products. But in-person isn’t always a viable option. What to do if your project needs user testing but it’s just not possible to get in front of your users personally? 

‍

Let’s talk unmoderated user testing. This approach sidesteps the need to meet your participants face-to-face as it’s done entirely remotely, over the internet. By it’s very nature there are also considerable benefits to unmoderated user testing.

‍

What is unmoderated user testing? 💻👀

‍

In the most basic sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants complete the testing activity by themselves and in their own time. For the most part, everything else stays the same.

‍

The key differences are:

‍

• You can’t ask follow-up questions
• You can’t use low-fidelity prototypes
• You can’t support participants (beyond the initial instructions you send them).

‍

Is unmoderated user testing right for your research project?

‍

By nature, unmoderated user research does not include any direct interaction between the researcher and the study participants. This is really the biggest benefit and also the biggest drawback. 

‍

Benefits of unmoderated usability testing 👩🏻💻

‍

• Speed and turnaround  - As there is no need to schedule meetings with each participant, unmoderated testing is usually much faster to initiate and complete. Depending on the study, it may be possible to launch a study and receive results in just a few hours.

• Size of study (participant numbers) - Unmoderated user testing also allows you to collect feedback from dozens or even hundreds of users at the same time.

• Location (local and/or international) -Testing online removes reliance on participants being physically present for the testing which broadens the ability to make contact with participants within your country or across the globe. 

‍

If you’d like to know more about the benefits of unmoderated usability testing, take a look at our article five reasons you should consider unmoderated user testing.

‍

Limitations of unmoderated usability testing 🚧

‍

• Early-prototype testing is difficult without a moderator to explain and help participants recover from errors or limitations of the prototype.

• Participant behavior - Without a moderator, participants tend to be less engaged and behave less realistically in tasks that depend on imagination, decision-making, or emotional responses.

• Inability to ask follow-up questions - by not being in the testing with the participant, the facilitator can’t ask further questions to get a deeper understanding of the participant’s reasoning. As you can’t rely on human judgment through a moderator being in the room with the participants and the ability to adjust the test in the moment, unmoderated usability testing needs thorough up front planning.

‍

Because of these limitations unmoderated usability testing usually works best for evaluating live websites and apps or highly functional prototypes.  It’s great for testing activities that don’t require a lot of imagination or emotion from participants. Such as testing functionality or answering direct queries to do with your product.

‍

What’s involved when setting up unmoderated usability testing? 🤔💭

‍

1. Define testing goals

With any usability testing, it pays to define your goals before getting underway with setting up the software. What do you want to know from the participants? Goals vary from test to test. Understanding your goals upfront will help you to make the correct tool choice.

‍

2. Define your demographic

With a clear understanding of your goal, now it’s time to consider which participants are right for your study. Think about who they are, their demographic, and where they live. Are they new users or existing? Are they experts or novices?

‍

3. Selecting testing software

As unmoderated studies, are done remotely, the software used to faciliate the study plays a key role in ensuring you get useful results. Without a facilitator, the software must guide the participants through the session and record what happens. Take the time to test software and select one that is right for your study.

‍

4. Write your own tasks and questions

Think through your goals and what you want to achieve from the testing. Many of the unmoderated testing services include study templates with generic example tasks. Remember they are templates, and your tasks and questions should be specific to your particular study. Any task instructions guiding the participants should be clear and directive.

‍

5. Trial session

You’ve done all of the upfront work, now it’s time to test that it works, the software does what you expect and the instructions you have written can be followed. Doing a test run is crucial, especially with unmoderated usability testing, as there won’t be a facilitator in the testing to fix any problems.

‍

6. Recruit participants

Having defined your target audience and demographic, now is the time to recruit participants. Ensuring you have some control over the recruitment process is important, either through screening questions or recruiting your own. There are services that  recruit from a pool of willing participants. Thiscan be a great way to get a wide range of users.

‍

7. Analyze results

You are likely to accumulate a lot of data from your unmoderated testing. You’ll need a way to organize and analyze the data to derive insights that are valuable. Depending on the type of usability testing you do will vary the type of results. Quantitative testing gives data-driven results and direct answers. Whereas qualitative testing through audio or video recordings of participants’ actions or comments will need time to analyze and look at behavioral observations. 

‍

Wrap Up 🌯

‍

Unmoderated usability testing can be a good option for your study. It may not be right for all of your studies all of the time. While it can be quick to implement and often cheaper than moderated usability testing, it still requires time and planning to ensure you get the data insights you are looking for. Following a checklist can be a great way to ensure you approach your research methodically.

‍

Are your users engaged in your website? The success of your website will largely depend on your answer. After all, engaged users are valuable users; they keep coming back and will recommend your site to colleagues, friends, and family. So, if you’re not sure if your users are engaged or not, consider looking into your user engagement metrics.

‍

User engagement can be measured using a number of key metrics provided by website analytics platforms. Metrics such as bounce rate, time on page, and click-through rate all provide clues to user engagement and therefore overall website user experience.

‍

This article will help you understand user engagement and why it’s important to measure. We’ll also discuss how to apply user engagement insights to improve website success. Combining a little bit of data with some user research is a powerful thing, so let’s get into it.

‍

‍

‍

Understanding User Engagement Metrics 📐

‍

User engagement metrics provide valuable insight for both new and existing websites. They should be checked regularly as a sort of ‘pulse check’ for website user experience and performance. So, what metrics should you be looking at? Website metrics can be overwhelming; there are hundreds if not thousands to analyze, so let’s focus on three:

‍

‍

Bounce rate

‍
Measures the percentage of users that visit just one page on your site before leaving. If your bounce rate is high it suggests that users aren’t finding the content relevant, engaging, or useful. It points to a poor initial reaction to your site and means that users are arriving, making a judgment about your design or content, and then leaving.

‍

‍

Time on page

‍
Calculated by the time difference between the point when a person lands on the page and when they move on to the next one. It indicates how engaging or relevant individual pages on your website are. Low time on page figures suggest that users aren’t getting what they need from a certain page, either in terms of the content, the aesthetics, or both.

‍

‍

Click-through rate

‍
Click-through rate compares the number of times someone clicks on your content, to the number of impressions you get (how many times an internal link or ad was viewed). The higher the rate, the better the engagement and performance of that element. User experience design can influence click-through rates through copywriting, button contrasts, heading structure, navigation, etc.

‍

‍

Conversion rate

‍
Conversion rates are perhaps the pinnacle of user engagement metrics. Conversion rate is the percentage of users that perform specific tasks you define. They are therefore dictated by your goals, which could include form submissions, transactions, etc. If your website has high conversion rates, you can be fairly confident that your website is matching your users’ needs, requirements, and expectations.

‍

But how do these metrics help? Well, they don’t give you an answer directly. The metrics point to potential issues with website user experience. They guide further research and subsequent updates that lead to website improvement. In the next section, we’ll discuss how these and others can support better website user experiences.

‍

‍

‍

Identifying Areas for Improvement 💡

‍

So, you’ve looked at your website’s user engagement metrics and discovered some good, and some bad. The good news is, there’s value in discovering both! The catch? You just need to find it. Remember, the metrics on their own don’t give you answers; they provide you direction.

‍

The ‘clues’ that user engagement metrics provide are the starting point for further research. Remember, we want to make data-driven decisions. We want to avoid making assumptions and jumping to conclusions about why our website is reporting certain metrics. Fortunately, there are a bunch of different ways to do this.

‍

User research data can be gathered by using both qualitative and quantitative research techniques. Insights into user behavior and needs can reveal why your website might be performing in certain ways. Research can include both qualitative and quantitative techniques.

‍

‍

Qualitative research techniques

‍

• Usability test – Test a product with people by observing them as they attempt to complete various tasks.
• User interview – Sit down with a user to learn more about their background, motivations and pain points.
• Contextual inquiry – Learn more about your users in their own environment by asking them questions before moving onto an observation activity.
• Focus group – Gather 6 to 10 people for a forum-like session to get feedback on a product.

‍

‍

Quantitate research techniques

‍

• Card sorts – Find out how people categorize and sort information on your website.
• First-click tests – See where people click first when tasked with completing an action.
• A/B tests – Compare 2 versions of a design in order to work out which is more effective.
• Clickstream analysis – Analyze aggregate data about website visits.
• Tree-testing - Test your site structure using text-only categorization and labels

‍

The type of research depends on what question you want to answer. Being specific about your question will help you identify what research technique(s) to deploy and ultimately the quality of your answer. If you’re serious about website improvement; identify problem areas with user engagement metrics, and investigate how to fix them with user research.

‍

‍

‍

Optimizing Content and Design

‍

If you have conducted user research and found weak areas on your website, there are many things to consider. Three good places to start are navigation, content, and website layout. Combined, these have a huge impact on user experience and can be leveraged to address disappointing engagement metrics.

‍

‍

Navigation

‍
Navigation is a crucial aspect of creating a good user experience since it fundamentally connects pages and content which allows users to find what they need. Navigation should be simple and easy to follow, with important information/actions at the top of menus. Observing the results of card sorting, tree testing, and user testing can be particularly useful in website optimization efforts. You may find that search bars, breadcrumb trails, and internal links can also help overcome navigation issues.

‍

‍

Content

‍
Are users seeing compelling or relevant content when they arrive on your site? Is your content organized in a way that encourages further exploration? Card sorting and content audits are useful in answering these questions and can help provide you with the insights required to optimize your content. You should identify what content might be redundant, out of date, or repetitive, as well as any gaps that may need filling.

‍

‍

Layout

‍
A well-designed layout can improve the overall usability of a website, making it easier for users to find what they're looking for, understand the content, and engage with it. Consider how consistent your heading structures are and be sure to use consistent styling throughout the site, such as similar font sizes and colors. Don’t be afraid to use white space; it’s great at breaking up sections and making content more readable.

‍

An additional factor related to layout is mobile optimization. Mobile-first design is necessary for apps, but it should also factor into your website design. How responsive is your website? How easy is it to navigate on mobile? Is your font size appropriate? You might find that poor mobile experience is negatively impacting user engagement metrics.

‍

‍

‍

Measuring Success 🔎

‍

User experience design is an iterative, ongoing process, so it’s important to keep a record of your website’s user experience metrics at various points of development. Fortunately, website analytics platforms will provide you with historic user data and key metrics; but be sure to keep a separate record of what improvements you make along the way. This will help you pinpoint what changes impacted different metrics.

‍

Define your goals and create a website optimization checklist that monitors key metrics on your site. For example, whenever you make an update, ensure bounce rates don’t exceed a certain number during the days following; check that your conversion rates are performing as they should be; check your time on sites hasn’t dropped. Be sure to compare metrics between desktop and mobile too.

‍

User’s needs and expectations change over time, so keep an eye on how new content is performing. For example, which new blog posts have attracted the most attention? What pages or topics have had the most page views compared to the previous period? Tracking such changes can help to inform what your users are currently engaged in, and will help guide your user experience improvements.

‍

‍

‍

Conclusion 🤗

‍

User engagement metrics allow you to put clear parameters around user experience. They allow you to measure where your website is performing well, and where your website might need improving. Their main strength is in how accessible they are; you can access key metrics on website analytics platforms in moments. However, user engagement metrics on their own may not reveal how and why certain website improvements should be made. In order to understand what’s going on, you often need to dig a little deeper.

‍

Time on page, bounce rate, click-through rate, and conversion rates are all great starting points to understand your next steps toward website improvement. Use them to define where further research may be needed. Not sure why your average pages per session is two? Try conducting first-click testing; where are they heading that seems to be a dead end? Is your bounce rate too high? Conduct a content audit to find out if your information is still relevant, or look into navigation roadblocks. Whatever the question; keep searching for the answer.

‍

User engagement metrics will keep you on your toes, but that’s a good thing. They empower you to make ongoing website improvements and ensure that users are at the heart of your website design. 

A great user experience (UX) is one of the largest drivers of growth and revenue through user satisfaction. However, when budgets get tight, or there is a squeeze on timelines, user research is one of the first things to go. Often at the cost of user satisfaction.  

‍

This short sighted view can mean project managers are preoccupied with achieving milestones and short term goals. And UX teams get stuck researching products they weren’t actually involved with developing. As a result no one has the space and understanding to really develop a product that speaks to users needs, desires and wants. There must  be a better way to produce a product that is user-driven.  Thankfully there is.

‍

What is user research and why should project managers care about it? 👨🏻💻

‍

User research is an important part of the product development process. Primarily, user research involves using different research methods to gather information about your end users. 

‍

Essentially it aims to create the best possible experience for your users by listening and learning directly from those that already or potentially will use your product. You might conduct interviews to help you understand a particular problem, carry out a tree test to identify bottlenecks or problems in your navigation, or do some usability testing to directly observe your users as they perform different tasks on your website or in your app. Or a combination of these to understand what users really want.

‍

To a project manager and team, this likely sounds fairly familiar, that any project can’t be managed in a silo. Regular check-ins and feedback are essential to making smart decisions. The same with UX research. It can make the whole process quicker and more efficient. By taking a step back, digging into your users’ minds, and gaining a fuller understanding of what they want upfront, it can curtail short-term views and decisions.

‍

Bringing more user research into your development process has major benefits for the team, and the ultimately the quality of that final product. There are three key benefits:

‍

1. Saves your development team time and effort. Ensuring the team is working on what users want, not wasting time on features that don’t measure up.
2. Gives your users a better experience by meeting their requirements.
3. Helps your team innovate quickly by understanding what users really want.

‍

As a project manager, making space and planning for user research can be one of the best ways to ensure the team is creating a product that truly is user-driven.

‍

How to bring research into your product development process 🤔

‍

There are a couple of ways you can bring UX research into your product development process. 

‍

1. Start with a dedicated research project.
2. Integrate UX research throughout the development project.

‍

It can be more difficult to integrate UX research throughout the process, as it means planning the project with various stages of research built in to check the development of features. But ultimately this approach is likely to turn out the best product. One that has been considered, checked and well thought out through the whole product development process. To help you on the way we have laid out 6 key steps to help you integrate UX research into your product development process.

‍

6 key steps to integrate UX research 👟

‍

Step 1: Define your research questions

‍

Take a step back, look at your product and define your research questions. 

‍

It may be tempting just to ask, ‘do users like our latest release?’ This however does not get to why or what your users like or don’t like. Try instead:

• What do our users really want from our product?
• Where are they currently struggling while using our website?
• How can we design a better product for our users?

‍

These questions help to form the basis of specific questions about your product and specific areas of research to explore which in turn help shape the type of research you undertake.

‍

Step 2: Create your research plan

‍

With a few key research questions to focus on, it’s time to create your research plan.

‍

A great research plan covers your project’s goals, scope, timing, and deliverables. It’s essential for keeping yourself organized but also for getting key stakeholder signoff.

‍

Step 3: Prepare any research logistics

‍

Every project plan requires attention to detail including a user research project. And with any good project there are a set of steps to help make sense of it.

‍

1. Method: Based on your questions, what is the best user research method to use? 
2. Schedule: When will the research take place? How long will it go on for? If this is ongoing research, plan how it will be implemented and how often.
3. Location: Where will the research take place? 
4. Resources: What resources do you need? This could be technical support or team members.
5. Participants: Define who you want to research. Who is eligible to take part in this research? How will you find the right people?
6. Data: How will you capture the research data? Where will it be stored? How will you analyze the data and create insights and reports that can be used?
7. Deliverables: What is the ultimate goal for your research project?

‍

Step 4: Decide which method will be used

‍

Many user research methods benefit from an observational style of testing. Particularly if you are looking into why users undertake a specific task or struggle.

‍

Typically, there are two approaches to testing:

‍

1. Moderated testing is when a moderator is present during the test to answer questions, guide the participant, or dig deeper with further questions.
2. Unmoderated testing is when a participant is left on their own to carry out the task. Often this is done remotely and with very specific instructions.Your key questions will determine which method will works best for your research.  Find our more about the differences.

‍

Step 5: Run your research session

‍

It’s time to gather insights and data. The questions you are asking will influence how you run your research sessions and the methods you’ve chosen. 

‍

If you are running surveys you will be asking users through a banner or invitation to fill out your survey. Unmoderated and very specific questions. Gathering qualitative data and analyzing patterns.

‍

If you’re using something qualitative like interviews or heat mapping, you’ll want to implement software and gather as much information as possible.

‍

Step 6: Prepare a research findings report and share with stakeholders

‍

Analyze your findings, interrogate your data and find those insights that dive into the way your users think. How do they love your product? But how do they also struggle?

‍

Pull together your findings and insights into an easy to understand report. And get socializing. Bring your key stakeholders together and share your findings. Bringing everyone across the findings together can bring everyone on the journey. And for the development process can mean decisions can be user-driven. 

‍

Wrap Up 🥙

‍

Part of any project, UX research should be essential to developing a product that is user-driven. Integrating user research into your development process can be challenging. But with planning and strategy it can be hugely beneficial to saving time and money in the long run.