The Age of the Customer is well and truly here. In every industry and vertical across the globe, UX professionals now dictate the terms, placing customers at the heart of every design decision. Or at least, this is the new reality that’s unfolding in the organizations and businesses that don’t want to be left behind.

Make no mistake; simply claiming to be the best is no longer enough. To survive and thrive, people need to be placed at the heart. The golden key that will allow organizations to pivot to this new reality lies with that of the user researcher.

But it’s not enough to simply “do user research”. Sure, some customer insight is obviously better than none at all, but to really be useful it needs to be effective research. That’s what this article is all about.

Get comfortable, because this is going to be a long one – for good reason.

Why (effective) user research is so important

You are not your user. As much as you may like to think that you are, you’re not. It can be a tricky proposition to get your head around, especially when we regularly assume that everyone thinks like us. There are 8 billion people out there who have a vastly different set of experiences and perspectives than you. With that in mind, when we start to generalize based on our own personal experiences, this is what’s known as availability bias.

Unfortunately, solving this is issue not as easy as getting into a room with customers and having a chat. People don’t always tell the truth! This isn’t to say that the participant in your last user interview was flat out lying to you, but the things that people say are different from the things that people do. It;’s your job (as a user researcher) to intuit the actual behaviors and actions, and identify their needs based on this data.

When you’re doing your job correctly, you’ve given your organization the best possible chance of success. Everything  – and I mean everything – starts with a solid understanding of your users. Doors will open, paths will reveal themselves – you get the idea.

The qualities of an effective user researcher

Let me preface this section by saying that you don’t have to have all of these qualities in spades, the list below is really just a way for you to better understand some of the traits of an effective user researcher, to get you thinking and on the right path.

• Curious: User research can be quite repetitive, especially when you get to the 6th user interview and need to ask the same questions. A genuine curiosity about people, the challenges they face and their behaviors will go a long way in helping you to push through.
• Pragmatic: Being an idealist has its uses, but it’s also important to be pragmatic. As a researcher, you need to operate on a fine line and balance your capacity to do research with business goals, finances and the desires of your stakeholders. Do the most with what you’ve got.
• Organized: It takes a lot to plan a research project, from scheduling testing sessions to assembling large slide decks for presentations. You’ve got to manage a large number of complex components, so it’s important that you can organize and prioritize.
• Collaborative: User research is most effective when it’s carried out collaboratively. This means working with your team, with the organization and with other disciplines. Think outside the box: Who stands to benefit from your research and how can you involve them?
• Empathetic: Real, natural empathy is a rare trait, but adopting an empathetic mindset is something everyone can (and should) learn. Beyond just uncovering insights from your participants, consider what these insights mean and how they all connect. This will truly enable you to understand your users.
• Sociable: You don’t have to suddenly adopt an extroverted persona, but being actively interested in other people will help you build relationships both inside your organization and with customers.
• Perceptive: User research means listening and observing. During a user interview or usability test, you need to be able to filter all of the data entering your mind and extract the most relevant insights.
• Analytical: In a similar vein to perceptiveness, being analytical is also key if you want to understand all of the data that your research will produce. Filter, examine, extract and move on.

How to run user research effectively (and at a low cost)

There are innumerable methods for user research, but many are resource- and time-intensive. What’s more, certain research methods come attached with significant costs.

But, research doesn’t have to be the time and money sink that it can often first appear to be. Certain actions before you ever step into the room with a participant can make a world of difference.

Conduct research at the start

User research is obviously valuable whenever you do it, but you’ll see the biggest impact when you carry it out right the start of a project. Conduct research to get the lay of the land; to learn how and why customers make certain decisions, and where the biggest opportunities lie.

Note: Don’t research in a silo, involve your team, stakeholders and other interested parties.

Have clear goals – and a plan

Every research project needs a clear objective, and that comes from a detailed UX research plan, which includes well-formulated research questions. Every project will have a different question, but they’re the best starting point to ensure research success.

Choose the right methods

There’s no shortage of research methods to choose from, but being an effective user researcher is all about being able to pick the right methods for each project, and use them correctly. Nearly every research project will benefit from using a combination of qualitative and quantitative methods in order to generate the most useful insights.

To understand which method to use, it’s a good idea to view them using the following framework:

Involve stakeholders

Bring stakeholders into your research project as early as possible. These are the people that will end up utilizing the results of your work, and chances are they’re the ones who’ll have the most questions at the end. Involve them through consultation, regular updates, the all-too-important presentation at the end of the project and by letting them take notes for you during research sessions.

Wrap up

It’s not enough to simply run a card sort now (although that’s still a very useful exercise). You need to think cohesively about the role of your research in your organization and make sure that you’re as aware of your bias as you are of the various methods and tools available to you. Happy researching!

In-person user testing is an important part of any research project, helping you to see first-hand how your users interact with your prototypes and products – but what are you supposed to do when it’s not a viable option?

The recent outbreak of coronavirus is forcing user researchers and designers to rethink traditional testing approaches, especially in-person testing. So what’s the solution? Enter unmoderated user testing. This approach circumvents the need to meet your participants face-to-face as it’s done entirely over the internet. As you can probably guess, this also means there are a few considerable benefits.

Here, we'll take a look at 5 reasons to consider this testing approach. But first, let's explore what unmoderated user testing is.

What is unmoderated user testing?

In the most basic sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants complete the testing activity by themselves and in their own time. For the most part, everything else stays the same.

The key differences are that:

• You’re unable to ask follow-up questions
• You can’t use low-fidelity prototypes
• You can’t support participants (beyond the initial instructions you send them).

However, there are a number of upsides to unmoderated user testing, which we’ll cover below.

1. You can access participants from all over the globe

There’s a good chance that your organization’s customers don’t exist solely in your city, or even just in your country, so why limit yourself to testing local individuals? Moderated user testing requires you to either bring in people who can visit your office or for you to actually travel to another physical location and host testing sessions there.

With unmoderated user testing, you can use a variety of participant recruitment services to access large groups of participants from all over the globe. Making these services even more useful is the fact many allow you to recruit the exact participants you need. For example, drivers of Toyota hybrid vehicles who live in Calgary.

2. Unmoderated user testing is cheaper

Have a think for a moment about all of the typical costs that go along with a hosted user testing session. There’s the cost of a location if you’re traveling to another city, the remuneration for the people you’re testing and the cost of equipment (that you may not typically have access to). Sure, moderated testing can be made much more affordable if you’re hosting a session in your own office and you have all of the required gear, but that’s not the case for everyone doing research.

Unmoderated user testing really only requires you to choose the tool with which you want to run your user test (variable cost), set up your study and then send out the link to your participants.

3. It’s easier to manage

Unmoderated user testing means you can set aside the difficult task of managing participants in person, from scheduling through to finding notetakers and people to help you with the recording equipment. As we noted in the above section about cost, all you have to do is select the tool and then set up and run your study.

4. Automatic analysis

Running in-person, qualitative usability testing sessions can deliver some truly useful insights. There’s really nothing like sitting down in front of a participant and seeing how they interact with the product you’re working on, hearing their frustrations and learning about how they work. But any insights you gain from these sessions you’ll have to derive yourself. There’s no magic button that can generate useful qualitative analysis for you.

With unmoderated user testing, and especially with the right set of tools, you can run your tests and then have analysis generated automatically from your data. Take our IA tool Treejack as just one example. The functionality built into the tool means you can send out text-based versions of your website structure and then see how people make their way through the website to find what they’re looking for. At the end of your test, Treejack will present you with an array of useful, detailed visualizations like this one:

5. There’s less chance of bias

Ever heard of the observer effect? It’s a theory that basically states that the observation of a phenomenon will inevitably change that phenomenon, commonly due to the instruments used in the measurement. The observer effect and other biases often come into play during moderated research sessions specifically as a result of having a moderator in the room – typically with their own biases. Removing the moderator from the equation means you’ll get more reliable data from your study.

And the best place to get started?

Unmoderated user research requires unmoderated testing tools. With health concerns like coronavirus and influenza leading to reduced travel and in turn making in-person testing more difficult, there’s never been a better time to start using unmoderated testing tools. If you haven’t already, take our full set of 5 tools for a spin for free (no credit card required).

"Dear Optimal WorkshopMy question is about biasing users with the wording of questions. It seems that my co-workers and I spend too much time debating the wording of task items in usability tests or questions on surveys. Do you have any 'best practices' for wordings that evoke unbiased feedback from users?" — Dominic

‍

Dear Dominic, Oh I feel your pain! I once sat through a two hour meeting that was dominated by a discussion on the merits of question marks!It's funny how wanting to do right by users and clients can tangle us up like fine chains in an old jewellery box. In my mind, we risk provoking bias when any aspect of our research (from question wording to test environment) influences participants away from an authentic response. So there are important things to consider outside of the wording of questions as well. I'll share my favorite tips, and then follow it up with a must-read resource or two.

‍

Balance your open and closed questions

‍

The right balance of open and closed questions is essential to obtaining unbiased feedback from your users. Ask closed questions only when you want a very specific answer like 'How old are you?' or 'Are you employed?' and ask open questions when you want to gain an understanding of what they think or feel. For example, don’t ask the participant'Would you be pleased with that?' (closed question). Instead, ask 'How do you feel about that?' or even better 'How do you think that might work?' Same advice goes for surveys, and be sure to give participants enough space to respond properly — fifty characters isn’t going to cut it.

‍

Avoid using words that are linked to an emotion

‍

The above questions lead me to my next point — don’t use words like ‘happy’. Don’t ask if they like or dislike something. Planting emotion based words in a survey or usability test is an invite for them to tell you what they think you want to hear . No one wants to be seen as being disagreeable. If you word a question like this, chances are they will end up agreeing with the question itself, not the content or meaning behind it...does that make sense? Emotion based questions only serve to distract from the purpose of the testing — leave them at home.

‍

Keep it simple and avoid jargon

‍

No one wants to look stupid by not understanding the terms used in the question. If it’s too complicated, your user might just agree or tell you what they think you want to hear to avoid embarrassment. Another issue with jargon is that some terms may have multiple meanings which can trigger a biased reaction depending on the user’s understanding of the term. A friend of mine once participated in user testing where they were asked if what they were seeing made them feel ‘aroused’. From a psychology perspective, that means you’re awake and reacting to stimuli.

‍

From the user's perspective? I’ll let you fill in the blanks on that one. Avoid using long, wordy sentences when asking questions or setting tasks in surveys and usability testing. I’ve seen plenty of instances of overly complicated questions that make the user tune out (trust me, you would too!). And because people don't tend to admit their attention has wandered during a task, you risk getting a response that lacks authenticity — maybe even one that aims to please (just a thought...).

‍

Encourage participants to share their experiences (instead of tying them up in hypotheticals)

‍

Instead of asking your user what they think they would do in a given scenario, ask them to share an example of a time when they actually did do it. Try asking questions along the lines of 'Can you tell me about a time when you….?' or 'How many times in the last 12 months have you...?' Asking them to recall an experience they had allows you to gain factual insights from your survey or usability test, not hypothetical maybes that are prone to bias.

‍

Focus the conversation by asking questions in a logical order

‍

If you ask usability testing or survey questions in an order that doesn’t quite follow a logical flow, the user may think that the order holds some sort of significance which in turn may change the way they respond. It’s a good idea to ensure that the questions tell a story and follow a logical progression for example the steps in a process — don’t ask me if I’d be interested in registering for a service if you haven’t introduced the concept yet (you’d be surprised how often this happens!). For further reading on this, be sure to check out this great article from usertesting.com.

‍

More than words — the usability testing experience as a whole

‍

Reducing bias by asking questions the right way is really just one part of the picture. You can also reduce bias by influencing the wider aspects of the user testing process, and ensuring the participant is comfortable and relaxed.

‍

Don’t let the designer facilitate the testing

‍

This isn’t always possible, but it’s a good idea to try to get someone else to facilitate the usability testing on your design (and choose to observe if you like). This will prevent you from bringing your own bias into the room, and participants will be more comfortable being honest when the designer isn't asking the questions. I've seen participants visibly relax when I've told them I'm not the designer of a particular website, when it's apparent they've arrived expecting that to be the case.

‍

Minimize discomfort and give observers a role

‍

The more comfortable your participants are, with both the tester and the observer, the more they can be themselves. There are labs out there with two-way mirrors to hide observers, but in all honesty the police interrogation room isn’t always the greatest look! I prefer to have the observer in the testing room, while being conscious that participants may instinctively be uncomfortable with being observed. I’ve seen observer guidelines that insist observers (in the room) stay completely silent the entire time, but I think that can be pretty creepy for participants! Here's what works best (in my humble opinion).

‍

The facilitator leads the testing session, of course, but the observer is able to pipe up occasionally, mostly for clarification purposes, and certainly join in the welcoming, 'How's the weather?' chit chat before the session begins. In fact, when I observe usability testing, I like to be the one who collects the participant from the foyer. I’m the first person they see and it’s my job to make them feel welcome and comfortable, so when they find out I'll be observing, they know me already. Anything you can do to make the participant feel at home will increase the authenticity of their responses.

‍

A note to finish

‍

At the end of the day the reality is we’re all susceptible to bias. Despite your best efforts you’re never going to eradicate it completely, but just being aware of and understanding it goes a long way to reducing its impacts. Usability testing is, after all, something we design. I’ll leave you with this quote from Jeff Sauro's must-read article on 9 biases to watch out for in usability testing:

‍

"We do the best we can to simulate a scenario that is as close to what users would actually do .... However, no amount of realism in the tasks, data, software or environment can change the fact that the whole thing is contrived. This doesn't mean it's not worth doing."

‍

‍