5 ways to increase user research in your organization

Knowing and understanding why and how your users use your product is invaluable for getting to the nitty gritty of usability. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative. 

‍

Put super simply, usability testing literally is testing how usable your product is for your users. If your product isn’t usable users often won’t complete their task, let alone come back for more. No one wants to lose users before they even get started. Usability testing gets under their skin and really into the how, why and what they want (and equally what they don’t).

‍

As we have been getting used to video calling regularly and using the internet for interactions, usability testing has followed suit. Being able to access participants remotely has allowed us to diversify the participant pool by not being restricted to those that are close enough to be in-person. This has also allowed an increase in the number of participants per test, as it becomes more cost-effective to perform remote usability testing.

‍

But if we’re remote, does this mean it can’t be moderated? No - remote testing, along with modern technology, can mean that remote testing can be facilitated and moderated. But what is the best method - moderated or unmoderated?

‍

What is moderated remote research testing? 🙋🏻

‍

In traditional usability testing, moderated research is done in person. With the moderator and the participant in the same physical space. This, of course, allows for conversation and observational behavioral monitoring. Meaning the moderator can note not only what the participant answers but how and even make note of the body language, surroundings, and other influencing factors. 

‍

This has also meant that traditionally, the participant pool has been limited to those that can be available (and close enough) to make it into a facility for testing. And being in person has meant it takes time (and money) to perform these tests.

‍

As technology has moved along and the speed of internet connections and video calling has increased, this has opened up a world of opportunities for usability testing. Allowing usability testing to be done remotely. Moderators can now set up testing remotely and ‘dial in’ to observe participants anywhere they are. And potentially even running focus groups or other testing in a group format across the internet. 

‍

Pros:

‍

- In-depth gathering of insights through a back-and-forth conversation and observing of the participants.

‍

- Follow-up questions don’t underestimate the value of being available to ask questions throughout the testing. And following up in the moment.

‍

- Observational monitoring noticing and noting the environment and how the participants are behaving, can give more insight into how or why they choose to make a decision.

‍

- Quick remote testing can be quicker to start, find participants, and complete than in-person. This is because you only need to set up a time to connect via the internet, rather than coordinating travel times, etc.

‍

- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing. This broadens your ability to broaden the pool, and participants can be either within your country or global. 

‍

Cons:

‍

- Time-consuming having to be present at each test takes time. As does analyzing the data and insights generated. But remember, this is quality data.

‍

- Limited interactions with any remote testing there is only so much you can observe or understand across the window of a computer screen. It can be difficult to have a grasp on all the factors that might be influencing your participants.

‍

What is unmoderated remote research testing? 😵💫

‍

In its most simple sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants are left to complete the testing by themselves and in their own time. For the most part, everything else stays the same. 

‍

Removing the moderator, means that there isn’t anyone to respond to queries or issues in the moment. This can either delay, influence, or even potentially force participants to not complete or maybe not be as engaged as you may like. Unmoderated research testing suits a very simple and direct type of test. With clear instructions and no room for inference. 

‍

Pros:

‍

- Speed and turnaround,  as there is no need to schedule meetings with each and every participant. Unmoderated usability testing is usually much faster to initiate and complete.

‍

- Size of study (participant numbers) unmoderated usability testing allows you to collect feedback from dozens or even hundreds of users at the same time. 

- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing, which broadens your participant pool.  And unmoderated testing means that it literally can be anywhere while participants complete the test in their own time.

‍

Cons:

‍

- Follow-up questions as your participants are working on their own and in their own time, you can’t facilitate and ask questions in the moment. You may be able to ask limited follow-up questions.

‍

- Products need to be simple to use unmoderated testing does not allow for prototypes or any product or site that needs guidance. 

‍

- Low participant support without the moderator any issues with the test or the product can’t be picked up immediately and could influence the output of the test.

‍

When should you do which? 🤔

‍

Each moderated and unmoderated remote usability testing have its use and place in user research. It really depends on the question you are asking and what you are wanting to know.

‍

Moderated testing allows you to gather in-depth insights, follow up with questions, and engage the participants in the moment. The facilitator has the ability to guide participants to what they want to know, to dig deeper, or even ask why at certain points. This method doesn’t need as much careful setup as the participants aren’t on their own. While this is all done online, it does still allow connection and conversation. This method allows for more investigative research. Looking at why users might prefer one prototype to another. Or possibly tree testing a new website navigation to understand where they might get lost and querying why the participant made certain choices.

‍

Unmoderated testing, on the other hand, is literally leaving the participants to it. This method needs very careful planning and explaining upfront. The test needs to be able to be set and run without a moderator. This lends itself more to wanting to know a direct answer to a query. Such as a card sort on a website to understand how your users might sort information. Or a first click to see how/where users will click on a new website.

‍

Wrap Up 🌯

‍

With the ability to expand our pool of participants across the globe with all of the advances (and acceptance of) technology and video calling etc, the ability to expand our understanding of users’ experiences is growing. Remote usability testing is a great option when you want to gather information from users in the real world. Depending on your query, moderated or unmoderated usability testing will suit your study. As with all user testing, being prepared and planning ahead will allow you to make the most of your test.

Co-authored by Brandon Dorn, UX designer at Viget.As user experience designers, making sure that websites and tools are usable is a critical component of our work, and conducting user research enables us to assess whether we’re achieving that goal or not. Even if we want to incorporate research, however, certain constraints may stand in our way.

‍

A few years ago, we realized that we were facing this issue at Viget, a digital design agency, and we decided to make an effort to prioritize user research. Almost two years ago, we shared initial thoughts on our progress in this blog post. We’ve continued to learn and grow as researchers since then and hope that what we’ve learned along the way can help your clients and coworkers understand the value of research and become better practitioners. Below are some of those lessons.

‍

Make research a priority for your organization

‍

Before you can do more research, it needs to be prioritized across your entire organization — not just within your design team. To that end, you should:

‍

• Know what you’re trying to achieve. By defining specific goals, you can share a clear message with the broader organization about what you’re after, how you can achieve those goals, and how you will measure success. At Viget, we shared our research goals with everyone at the company. In addition, we talked to the business development and project management teams in more depth about specific ways that they could help us achieve our goals, since they have the greatest impact on our ability to do more research.

• Track your progress. Once you’ve made research a priority, make sure to review your goals on an ongoing basis to ensure that you’re making progress and share your findings with the organization. Six months after the research group at Viget started working on our goals, we held a retrospective to figure out what was working — and what wasn’t.
• Adjust your approach as needed. You won’t achieve your goals overnight. As you put different tactics into action, adjust your approach if something isn’t helping you achieve your goals. Be willing to experiment and don’t feel bad if a specific tactic isn’t successful.

‍

Educate your colleagues and clients

‍

If you want people within your organization to get excited about doing more research, they need to understand what research means. To educate your colleagues and clients, you should:

• Explain the fundamentals of research. If someone has not conducted research before, they may not be familiar or feel comfortable with the vernacular. Provide an overview of the fundamental terminology to establish a basic level of understanding. In a blog post, Speaking the Same Language About Research, we outline how we established a common vocabulary at Viget.
• Help others understand the landscape of research methods. As designers, we feel comfortable talking about different methodologies and forget that that information will be new to many people. Look for opportunities to increase understanding by sharing your knowledge. At Viget, we make this happen in several ways. Internally, we give presentations to the company, organize group viewing sessions for webinars about user research, and lead focused workshops to help people put new skills into practice. Externally, we talk about our services and share knowledge through our blog posts. We are even hosting a webinar about conducting user interviews in November and we'd love for you to join us.
• Incorporate others into the research process. Don't just tell people what research is and why it's important — show them. Look for opportunities to bring more people into the research process. Invite people to observe sessions so they can experience research firsthand or have them take on the role of the notetaker. Another simple way to make people feel involved is to share findings on an ongoing basis rather than providing a report at the end of the process.

‍

Broaden your perspective while refining your skill set

‍

Our commitment to testing assumptions led us to challenge ourselves to do research on every project. While we're dogmatic about this goal, we're decidedly un-dogmatic about the form our research takes from one project to another. To pursue this goal, we seek to:

• Expand our understanding. To instill a culture of research at Viget, we've found it necessary to question our assumptions about what research looks like. Books like Erika Hall’s Just Enough Research teach us the range of possible approaches for getting useful user input at any stage of a project, and at any scale. Reflect on any methodological biases that have become well-worn paths in your approach to research. Maybe your organization is meticulous about metrics and quantitative data, and could benefit from a series of qualitative studies. Maybe you have plenty of anecdotal and qualitative evidence about your product that could be better grounded in objective analysis. Aim to establish a balanced perspective on your product through a diverse set of research lenses, filling in gaps as you learn about new approaches.
• Adjust our approach to project constraints. We've found that the only way to consistently incorporate research in our work is to adjust our approach to the context and constraints of any given project. Client expectations, project type, business goals, timelines, budget, and access to participants all influence the type, frequency, and output of our research. Iterative prototype testing of an email editor, for example, looks very different than post-launch qualitative studies for an editorial website. While some projects are research-intensive, short studies can also be worthwhile.
• Reflect on successes and shortcomings. We have a longstanding practice of holding post-project team retrospectives to reflect on and document lessons for future work. Research has naturally come up in these conversations, and many of the things we've discussed you're reading right now. As an agency with a diverse set of clients, it's been important for us to understand what types of research work for what types of clients, and when. Make sure to take time to ask these questions after projects. Mid-project retrospectives can be beneficial, especially on long engagements, yet it's hard to see the forest when you're in the weeds.

‍

Streamline qualitative research processes 🚄

‍

Learning to be more efficient at planning, conducting, and analyzing research has helped us overturn the idea that some projects merit research while others don't. Remote moderated usability tests are one of our preferred methods, yet, in our experience, the biggest obstacle to incorporating these tests isn't the actual moderating or analyzing, but the overhead of acquiring and scheduling participants. While some agencies contract out the work of recruiting, we've found it less expensive and more reliable to collaborate with our clients to find the right people for our tests. That said, here are some recommendations for holding efficient qualitative tests:

• Know your tools ahead of time. We use a number of tools to plan, schedule, annotate, and analyze qualitative tests (we're inveterate spreadsheet users). Learn your tools beforehand, especially if you're trying something new. Tools should fade into the background during tests, which Reframer does nicely.
• Establish a recruiting process. When working with clients to find participants, we'll often provide an email template tailored to the project for them to send to existing or potential users of their product. This introductory email will contain a screener that asks a few project-related demographic or usage questions, and provides us with participant email addresses which we use to follow-up with a link to a scheduling tool. Once this process is established, the project manager will ensure that the UX designer on the team has a regular flow of participants. The recruiting process doesn't take care of itself – participants cancel, or reschedule, or sometimes don't respond at all – yet establishing an approach ahead of time allows you, the researcher, to focus on the research in the midst of the project.
• Start recruiting early. Don't wait until you've finished writing a testing script to begin recruiting participants. Once you determine the aim and focal points of your study, recruit accordingly. Scripts can be revised and approved in the meantime.

‍

Be proactive about making research happen 🤸

‍

As a generalist design agency, we work with clients whose industries and products vary significantly. While some clients come to us with clear research priorities in mind, others treat it as an afterthought. Rare, however, is the client who is actively opposed to researching their product. More often than not, budget and timelines are the limiting factors. So we try not to make research an ordeal, but instead treat it as part of our normal process even if a client hasn't explicitly asked for it. Common-sense perspectives like Jakob Nielsen’s classic “Discount Usability for the Web” remind us that some research is always better than none, and that some can still be meaningfully pursued. We aren’t pushy about research, of course, but instead try to find a way to make it happen when it isn't a definite priority.

‍

‍World Usability Day is coming up on November 9, so now is a great time to stop and reflect on how you approach research and to brainstorm ways to improve your process. The tips above reflect some of the lessons we’ve learned at Viget as we’ve tried to improve our own process. We’d love to hear about approaches you’ve used as well.

Having a library of user research books is invaluable. Whether you’re an old hand in the field of UX research or just dipping your toes in the water, being able to reference detailed information on methods, techniques and tools will make your life much easier.

‍

There’s really no shortage of user research/UX reading lists online, so we wanted to do something a little different. We’ve broken our list up into sections to make finding the right book for a particular topic as easy as possible.

‍

General user research guides

‍

These books cover everything you need to know about a number of UX/user research topics. They’re great to have on your desk to refer back to – we certainly have them on the bookshelf here at Optimal Workshop.

‍

Observing the User Experience: A Practitioner's Guide to User Research

Mike Kuniavsky

‍

‍

This book covers 13 UX research techniques in a reference format. There’s a lot of detail, making it a useful resource for people new to the field and those who just need more clarification around a certain topic. There’s also a lot of practical information that you’ll find applicable in the real world. For example, information about how to work around research budgets and tight time constraints.

‍

• Goodreads link

‍

Just Enough Research

Erika Hall

‍

In Just Enough Research, author Erika Hall explains that user research is something everyone can and should do. She covers several research methods, as well as things like how to identify your biases and make use of your findings. Designers are also likely to find this one quite useful, as she clearly covers the relationship between research and design.

‍

• Goodreads link

‍

Research Methods in Human-Computer Interaction

‍

Harry Hochheiser, Jonathan Lazar, Jinjuan Heidi Feng

‍

‍

Like Observing the User Experience, this is a dense guide – but it’s another essential one. Here, experts on human-computer interaction and usability explain different qualitative and quantitative research methods in an easily understandable format. There are also plenty of real examples to help frame your thinking around the usefulness of different research methods.

‍

• Goodreads link

‍

Information architecture

‍

If you’re new to information architecture (IA), understanding why it’s such an important concept is a great place to start. There’s plenty of information online, but there are also several well-regarded books that make great starting points.

‍

Information Architecture for the World Wide Web: Designing Large-Scale Web Sites

Peter Morville, Louis Rosenfeld

‍

‍

‍

You’ll probably hear this book referred to as “the polar bear book”, just because the cover features a polar bear. But beyond featuring a nice illustration of a bear, this book clearly covers the process of creating large websites that are both easy to navigate and appealing to use. It’s a useful book for designers, information architects and user researchers.

‍

• Goodreads link

‍

How to Make Sense of Any Mess

Abby Covert

‍

‍

This is a great introduction to information architecture and serves as a nice counter to the polar bear book, being much shorter and more easily digestible. Author Abby Covert explains complex concepts in a way anyone can understand and also includes a set of lessons and exercises with each chapter.

‍

• Goodreads link

‍

User interviews

‍

For those new to the task, the prospect of interviewing users is always daunting. That makes having a useful guide that much more of a necessity!

‍

Interviewing Users: How to Uncover Compelling Insights

Steve Portigal

‍

‍

While interviewing users may seem like something that doesn’t require a guide, an understanding of different interview techniques can go a long way. This book is essentially a practical guide to the art of interviewing users. Author Steve Portigal covers how to build rapport with your participants and the art of immersing yourself in how other people see the world – both key skills for interviewers!

‍

• Goodreads link

‍

Usability testing

‍

Web usability is basically the ease of use of a website. It’s a broad topic, but there are a number of useful books that explain why it’s important and outline some of the key principles.

‍

Don't Make Me Think: A Common Sense Approach to Web Usability

Steve Krug

‍

‍

Don’t Make Me Think is the first introduction to the world of UX and usability for many people, and for good reason – it’s a concise introduction to the topics and is easy to digest. Steve Krug explains some of the key principles of intuitive navigation and information architecture clearly and without overly technical language. In the latest edition, he’s updated the book to include mobile usability considerations.

‍

As a testament to just how popular this book is, it was released in 2000 and has since had 2 editions and sold 400,000 copies.

• Goodreads link

‍

Design

‍

The design–research relationship is an important one, even if it’s often misunderstood. Thankfully, authors like Don Norman and Vijay Kumar are here to explain everything.

‍

The Design of Everyday Things

Don Norman

‍

‍

This book, by cognitive scientist and usability engineer Don Norman, explains how design is the communication between an object and its user, and how to improve this communication as a way of improving the user experience. If nothing else, this book will force you to take another look at the design of everyday objects and assess whether or not they’re truly user-friendly.

• Goodreads link

‍

101 Design Methods: A Structured Approach for Driving Innovation in Your Organization

Vijay Kumar

‍

‍

A guidebook for innovation in the context of product development, this book approaches the subject in a slightly different way to many other books on the same subject. The focus here is that the practice of creating new products is actually a science – not an art. Vijay Kumar outlines practical methods and useful tools that researchers and designers can use to drive innovation, making this book useful for anyone involved in product development.

• Goodreads link

‍

See our list on Goodreads

‍

We've put together a list of all of the above books on Goodreads, which you can access here.

‍

Further reading

‍

For experienced practitioners and newcomers alike, user research can often seem like a minefield to navigate. It can be tricky to figure out which method to use when, whether you bring a stakeholder into your usability test (you should) and how much you should pay participants. Take a look at some of the other articles on our blog if you’d like to learn more.