April 24, 2019
6 min

6 things to consider when setting up a research practice

With UX research so closely tied to product success, setting up a dedicated research practice is fast becoming important for many organizations. It’s not an easy process, especially for organizations that have had little to do with research, but the end goal is worth the effort.

But where exactly are you supposed to start? This article provides 6 key things to keep in mind when setting up a research practice, and should hopefully ensure you’ve considered all of the relevant factors.

1) Work out what your organization needs

The first and most simple step is to take stock of the current user research situation within the organization. How much research is currently being done? Which teams or individuals are talking to customers on an ongoing basis? Consider if there are any major pain points with the current way research is being carried out or bottlenecks in getting research insights to the people that need them. If research isn't being practiced, identify teams or individuals that don't currently have access to the resources they need, and consider ways to make insights available to the people that need them.

2) Consolidate your insights

UX research should be communicating with nearly every part of an organization, from design teams to customer support, engineering departments and C-level management. The insights that stem from user research are valuable everywhere. Of course, the opposite is also true: insights from support and sales are useful for understanding customers and how the current product is meeting people's needs.

When setting up a research practice, identify which teams you should align with, and then reach out. Sit down with these teams and explore how you can help each other. For your part, you’ll probably need to explain the what and why of user research within the context of your organization, and possibly even explain at a basic level some of the techniques you use and the data you can obtain.

Then, get in touch with other teams with the goal of learning from them. A good research practice needs a strong connection to other parts of the business with the express purpose of learning. For example, by working with your organization’s customer support team, you’ll have a direct line to some of the issues that customers deal with on a regular basis. A good working relationship here means they’ll likely feed these insights back to you, in order to help you frame your research projects.

By working with your sales team, they’ll be able to share issues prospective customers are dealing with. You can follow up on this information with research, the results of which can be fed into the development of your organization’s products.

It can also be fruitful to develop an insights repository, where researchers can store any useful insights and log research activities. This means that sales, customer support and other interested parties can access the results of your research whenever they need to.

When your research practice is tightly integrated other key areas of the business, the organization is likely to see innumerable benefits from the insights>product loop.

3) Figure out which tools you will use

By now you’ve hopefully got an idea of how your research practice will fit into the wider organization – now it’s time to look at the ways in which you’ll do your research. We’re talking, of course, about research methods and testing tools.

We won’t get into every different type of method here (there are plenty of other articles and guides for that), but we will touch on the importance of qualitative and quantitative methods. If you haven’t come across these terms before, here’s a quick breakdown:

  • Qualitative research – Focused on exploration. It’s about discovering things we cannot measure with numbers, and often involves speaking with users through observation or user interviews.
  • Quantitative research – Focused on measurement. It’s all about gathering data and then turning this data into usable statistics.

All user research methods are designed to deliver either qualitative or quantitative data, and as part of your research practice, you should ensure that you always try to gather both types. By using this approach, you’re able to generate a clearer overall picture of whatever it is you’re researching.

Next comes the software. A solid stack of user research testing tools will help you to put research methods into practice, whether for the purposes of card sorting, carrying out more effective user interviews or running a tree test.

There are myriad tools available now, and it can be difficult to separate the useful software from the chaff. Here’s a list of research and productivity tools that we recommend.

Tools for research

Here’s a collection of research tools that can help you gather qualitative and quantitative data, using a number of methods.

  • Treejack – Tree testing can show you where people get lost on your website, and help you take the guesswork out of information architecture decisions. Like OptimalSort, Treejack makes it easy to sort through information and pairs this with in-depth analysis features.
  • dScout – Imagine being able to get video snippets of your users as they answer questions about your product. That’s dScout. It’s a video research platform that collects in-context “moments” from a network of global participants, who answer your questions either by video or through photos.
  • Ethnio – Like dScout, this is another tool designed to capture information directly from your users. It works by showing an intercept pop-up to people who land on your website. Then, once they agree, it runs through some form of research.
  • OptimalSort – Card sorting allows you to get perspective on whatever it is you’re sorting and understand how people organize information. OptimalSort makes it easier and faster to sort through information, and you can access powerful analysis features.
  • Reframer – Taking notes during user interviews and usability tests can be quite time-consuming, especially when it comes to analyze the data. Reframer gives individuals and teams a single tool to store all of their notes, along with a set of powerful analysis features to make sense of their data.
  • Chalkmark – First-click testing can show you what people click on first in a user interface when they’re asked to complete a task. This is useful, as when people get their first click correct, they’re much more likely to complete their task. Chalkmark makes the process of setting up and running a first-click test easy. What’s more, you’re given comprehensive analysis tools, including a click heatmap.

Tools for productivity

These tools aren’t necessarily designed for user research, but can provide vital links in the process.

  • Whimsical – A fantastic tool for user journeys, flow charts and any other sort of diagram. It also solves one of the biggest problems with online whiteboards – finicky object placement.
  • Descript – Easily transcribe your interview and usability test audio recordings into text.
  • Google Slides – When it inevitably comes time to present your research findings to stakeholders, use Google Slides to create readable, clear presentations.

4) Figure out how you’ll track findings over time

With some idea of the research methods and testing tools you’ll be using to collect data, now it’s time to think about how you’ll manage all of this information. A carefully ordered spreadsheet and folder system can work – but only to an extent. Dedicated software is a much better choice, especially given that you can scale these systems much more easily.

A dedicated home for your research data serves a few distinct purposes. There’s the obvious benefit of being able to access all of your findings whenever you need them, which means it’s much easier to create personas if the need arises. A dedicated home also means your findings will remain accessible and useful well into the future.

When it comes to software, Reframer stands as one of the better options for creating a detailed customer insights repository as you’re able to capture your sessions directly in the tool and then apply tags afterwards. You can then easily review all of your observations and findings using the filtering options. Oh, and there’s obviously the analysis side of the tool as well.

If you’re looking for a way to store high-level findings – perhaps if you’re intending to share this data with other parts of your organization – then a tool like Confluence or Notion is a good option. These tools are basically wikis, and include capable search and navigation options too.

5) Where will you get participants from?

A pool of participants you can draw from for your user research is another important part of setting up a research practice. Whenever you need to run a study, you’ll have real people you can call on to test, ask questions and get feedback from.

This is where you’ll need to partner other teams, likely sales and customer support. They’ll have direct access to your customers, so make sure to build a strong relationship with these teams. If you haven’t made introductions, it can helpful to put together a one-page sheet of information explaining what UX research is and the benefits of working with your team.

You may also want to consider getting in some external help. Participant recruitment services are a great way to offload the heavy lifting of sourcing quality participants – often one of the hardest parts of the research process.

6) Work out how you'll communicate your research

Perhaps one of the most important parts of being a user researcher is taking the findings you uncover and communicating them back to the wider organization. By feeding insights back to product, sales and customer support teams, you’ll form an effective link between your organization’s customers and your organization. The benefits here are obvious. Product teams can build products that actually address customer pain points, and sales and support teams will better understand the needs and expectations of customers.

Of course, it isn’t easy to communicate findings. Here are a few tips:

  • Document your research activities: With a clear record of your research, you’ll find it easier to pull out relevant findings and communicate these to the right teams.
  • Decide who needs what: You’ll probably find that certain roles (like managers) will be best served by a high-level overview of your research activities (think a one-page summary), while engineers, developers and designers will want more detailed research findings.

Continue reading

Share this article
Author
Optimal
Workshop

Related articles

View all blog articles
Learn more
1 min read

Moderated vs unmoderated research: which approach is best?

Knowing and understanding why and how your users use your product is invaluable for getting to the nitty gritty of usability. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative. 

Put super simply, usability testing literally is testing how usable your product is for your users. If your product isn’t usable users often won’t complete their task, let alone come back for more. No one wants to lose users before they even get started. Usability testing gets under their skin and really into the how, why and what they want (and equally what they don’t).

As we have been getting used to video calling regularly and using the internet for interactions, usability testing has followed suit. Being able to access participants remotely has allowed us to diversify the participant pool by not being restricted to those that are close enough to be in-person. This has also allowed an increase in the number of participants per test, as it becomes more cost-effective to perform remote usability testing.

But if we’re remote, does this mean it can’t be moderated? No - remote testing, along with modern technology, can mean that remote testing can be facilitated and moderated. But what is the best method - moderated or unmoderated?

What is moderated remote research testing? 🙋🏻

In traditional usability testing, moderated research is done in person. With the moderator and the participant in the same physical space. This, of course, allows for conversation and observational behavioral monitoring. Meaning the moderator can note not only what the participant answers but how and even make note of the body language, surroundings, and other influencing factors. 

This has also meant that traditionally, the participant pool has been limited to those that can be available (and close enough) to make it into a facility for testing. And being in person has meant it takes time (and money) to perform these tests.

As technology has moved along and the speed of internet connections and video calling has increased, this has opened up a world of opportunities for usability testing. Allowing usability testing to be done remotely. Moderators can now set up testing remotely and ‘dial in’ to observe participants anywhere they are. And potentially even running focus groups or other testing in a group format across the internet. 

Pros:

- In-depth gathering of insights through a back-and-forth conversation and observing of the participants.

- Follow-up questions don’t underestimate the value of being available to ask questions throughout the testing. And following up in the moment.

- Observational monitoring noticing and noting the environment and how the participants are behaving, can give more insight into how or why they choose to make a decision.

- Quick remote testing can be quicker to start, find participants, and complete than in-person. This is because you only need to set up a time to connect via the internet, rather than coordinating travel times, etc.

- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing. This broadens your ability to broaden the pool, and participants can be either within your country or global. 

Cons:

- Time-consuming having to be present at each test takes time. As does analyzing the data and insights generated. But remember, this is quality data.

- Limited interactions with any remote testing there is only so much you can observe or understand across the window of a computer screen. It can be difficult to have a grasp on all the factors that might be influencing your participants.

What is unmoderated remote research testing? 😵💫

In its most simple sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants are left to complete the testing by themselves and in their own time. For the most part, everything else stays the same. 

Removing the moderator, means that there isn’t anyone to respond to queries or issues in the moment. This can either delay, influence, or even potentially force participants to not complete or maybe not be as engaged as you may like. Unmoderated research testing suits a very simple and direct type of test. With clear instructions and no room for inference. 

Pros:

- Speed and turnaround,  as there is no need to schedule meetings with each and every participant. Unmoderated usability testing is usually much faster to initiate and complete.

- Size of study (participant numbers) unmoderated usability testing allows you to collect feedback from dozens or even hundreds of users at the same time. 


- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing, which broadens your participant pool.  And unmoderated testing means that it literally can be anywhere while participants complete the test in their own time.

Cons:

- Follow-up questions as your participants are working on their own and in their own time, you can’t facilitate and ask questions in the moment. You may be able to ask limited follow-up questions.

- Products need to be simple to use unmoderated testing does not allow for prototypes or any product or site that needs guidance. 

- Low participant support without the moderator any issues with the test or the product can’t be picked up immediately and could influence the output of the test.

When should you do which? 🤔

Each moderated and unmoderated remote usability testing have its use and place in user research. It really depends on the question you are asking and what you are wanting to know.

Moderated testing allows you to gather in-depth insights, follow up with questions, and engage the participants in the moment. The facilitator has the ability to guide participants to what they want to know, to dig deeper, or even ask why at certain points. This method doesn’t need as much careful setup as the participants aren’t on their own. While this is all done online, it does still allow connection and conversation. This method allows for more investigative research. Looking at why users might prefer one prototype to another. Or possibly tree testing a new website navigation to understand where they might get lost and querying why the participant made certain choices.

Unmoderated testing, on the other hand, is literally leaving the participants to it. This method needs very careful planning and explaining upfront. The test needs to be able to be set and run without a moderator. This lends itself more to wanting to know a direct answer to a query. Such as a card sort on a website to understand how your users might sort information. Or a first click to see how/where users will click on a new website.

Wrap Up 🌯

With the ability to expand our pool of participants across the globe with all of the advances (and acceptance of) technology and video calling etc, the ability to expand our understanding of users’ experiences is growing. Remote usability testing is a great option when you want to gather information from users in the real world. Depending on your query, moderated or unmoderated usability testing will suit your study. As with all user testing, being prepared and planning ahead will allow you to make the most of your test.

Learn more
1 min read

User research and agile squadification at Trade Me

Hi, I’m Martin. I work as a UX researcher at Trade Me having left Optimal Experience (Optimal Workshop's sister company) last year. For those of you who don’t know, Trade Me is New Zealand’s largest online auction site that also lists real estate to buy and rent, cars to buy, jobs listings, travel accommodation and quite a few other things besides. Over three quarters of the population are members and about three quarters of the Internet traffic for New Zealand sites goes to the sites we run.

Leaving a medium-sized consultancy and joining Trade Me has been a big change in many ways, but in others not so much, as I hadn’t expected to find myself operating in a small team of in-house consultants. The approach the team is taking is proving to be pretty effective, so I thought I’d share some of the details of the way we work with the readers of Optimal Workshop’s blog. Let me explain what I mean…

What agile at Trade Me looks like

Over the last year or so, Trade Me has moved all of its development teams over to Agile following a model pioneered by Spotify. All of the software engineering parts of the business have been ‘squadified’. These people produce the websites & apps or provide and support the infrastructure that makes everything possible.Across Squads, there are common job roles in ‘Chapters’ (like designers or testers) and because people are not easy to force into boxes, and why should they be, there are interest groups called ‘Guilds’.The squads are self-organizing, running their own processes and procedures to get to where they need to. In practice, this means they use as many or as few of the Kanban, Scrum, and Rapid tools they find useful. Over time, we’ve seen that squads tend to follow similar practices as they learn from each other.

How our UX team fits in

Our UX team of three sits outside the squads, but we work with them and with the product owners across the business.How does this work? It might seem counter-intuitive to have UX outside of the tightly-integrated, highly-focused squads, sometimes working with product owners working on stuff that might have little to do with what’s being currently developed in the squads. This comes down to the way Trade Me divides down the UX responsibilities within the organization. Within each squad there is a designer. He or she is responsible for how that feature or app looks, and, more importantly, how it acts — interaction design as well as visual design.Then what do we do, if we are the UX team?

We represent the voice of Trade Me’s users

By conducting research with Trade Me’s users we can validate the squads’ day-to-day decisions, and help frame decisions on future plans. We do this by wearing two hats. Wearing the pointy hats of structured, detailed researchers, we look into long-term trends: the detailed behaviours and goals of our different audiences. We’ve conducted lots of one-on-one interviews with hundreds of people, including top sellers, motor parts buyers, and job seekers, as well as running surveys, focus groups and user testing sessions of future-looking prototypes. For example, we recently spent time with a number of buyers and sellers, seeking to understand their motivations and getting under their skin to find out how they perceive Trade Me.

This kind of research enables Trade Me to anticipate and respond to changes in user perception and satisfaction.Swapping hats to an agile beanie (and stretching the metaphor to breaking point), we react to the medium-term, short-term and very short-term needs of the squads testing their ideas, near-finished work and finished work with users, as well as sometimes simply answering questions and providing opinion, based upon our research. Sometimes this means that we can be testing something in the afternoon having only heard we are needed in the morning. This might sound impossible to accommodate, but the pace of change at Trade Me is such that stuff is getting deployed pretty much every day, many of which affects our users directly. It’s our job to ensure that we support our colleagues to do the very best we can for our users.

How our ‘drop everything’ approach works in practice

Screen Shot 2014-07-11 at 10.00.21 am

We recently conducted five or six rounds (no one can quite remember, we did it so quickly) of testing of our new iPhone application (pictured above) — sometimes testing more than one version at a time. The development team would receive our feedback face-to-face, make changes and we’d be testing the next version of the app the same or the next day. It’s only by doing this that we can ensure that Trade Me members will see positive changes happening daily rather than monthly.

How we prioritize what needs to get done

To help us try to decide what we should be doing at any one time we have some simple rules to prioritise:

  • Core product over other business elements
  • Finish something over start something new
  • Committed work over non-committed work
  • Strategic priorities over non-strategic priorities
  • Responsive support over less time-critical work
  • Where our input is crucial over where our input is a bonus

Applying these rules to any situation makes the decision whether to jump in and help pretty easy.At any one time, each of us in the UX team will have one or more long-term projects, some medium-term projects, and either some short-term projects or the capacity for some short-term projects (usually achieved by putting aside a long-term project for a moment).

We manage our time and projects on Trello, where we can see at a glance what’s happening this and next week, and what we’ve caught sniff of in the wind that might be coming up, or definitely is coming up.On the whole, both we and the squads favour fast response, bulleted list, email ‘reports’ for any short-term requests for user testing.  We get a report out within four hours of testing (usually well within that). After all, the squads are working in short sprints, and our involvement is often at the sharp end where delays are not welcome. Most people aren’t going to read past the management summary anyway, so why not just write that, unless you have to?

How we share our knowledge with the organization

Even though we mainly keep our reporting brief, we want the knowledge we’ve gained from working with each squad or on each product to be available to everyone. So we maintain a wiki that contains summaries of what we did for each piece of work, why we did it and what we found. Detailed reports, if there are any, are attached. We also send all reports out to staff who’ve subscribed to the UX interest email group.

Finally, we send out a monthly email, which looks across a bunch of research we’ve conducted, both short and long-term, and draws conclusions from which our colleagues can learn. All of these latter activities contribute to one of our key objectives: making Trade Me an even more user-centred organization than it is.I’ve been with Trade Me for about six months and we’re constantly refining our UX practices, but so far it seems to be working very well.Right, I’d better go – I’ve just been told I’m user testing something pretty big tomorrow and I need to write a test script!

Learn more
1 min read

How to convince others of the importance of UX research

There’s not much a parent won’t do to ensure their child has the best chance of succeeding in life. Unsurprisingly, things are much the same in product development. Whether it’s a designer, manager, developer or copywriter, everyone wants to see the product reach its full potential.

Key to a product’s success (even though it’s still not widely practiced) is UX research. Without research focused on learning user pain points and behaviors, development basically happens in the dark. Feeding direct insights from customers and users into the development of a product means teams can flick the light on and make more informed design decisions.

While the benefits of user research are obvious to anyone working in the field, it can be a real challenge to convince others of just how important and useful it is. We thought we’d help.

Define user research

If you want to sell the importance of UX research within your organization, you’ve got to ensure stakeholders have a clear understanding of what user research is and what they stand to gain from backing it.

In general, there are a few key things worth focusing on when you’re trying to explain the benefits of research:

  • More informed design decisions: Companies make major design decisions far too often without considering users. User research provides the data needed to make informed decisions.
  • Less uncertainty and risk: Similarly, research reduces risk and uncertainty simply by giving companies more clarity around how a particular product or service is used.
  • Retention and conversion benefits: Research means you’ll be more aligned with the needs of your customers and prospective customers.

Use the language of the people you’re trying to convince. A capable UX research practice will almost always improve key business metrics, namely sales and retention.

The early stages

When embarking on a project, book in some time early in the process to answer questions, explain your research approach and what you hope to gain from it. Here are some of the key things to go over:

  • Your objectives: What are you trying to achieve? This is a good time to cover your research questions.
  • Your research methods: Which methods will you be using to carry out your research? Cover the advantages of these methods and the information you’re likely to get from using them.
  • Constraints: Do you see any major obstacles? Any issues with resources?
  • Provide examples: Nothing shows the value of doing research quite like a case study. If you can’t find an example of research within your own organization, see what you can find online.

Involve others in your research

When trying to convince someone of the validity of what you’re doing, it’s often best to just show them. There are a couple of effective ways you can do this – at a team or individual level and at an organizational level.

We’ll explain the best way to approach this below, but there’s another important reason to bring others into your research. UX research can’t exist in a vacuum – it thrives on integration and collaboration with other teams. Importantly, this also means working with other teams to define the problems they’re trying to solve and the scope of their projects. Once you’ve got an understanding of what they’re trying to achieve, you’ll be in a better position to help them through research.

Educate others on what research is

Education sessions (lunch-and-learns) are one of the best ways to get a particular team or group together and run through the what and why of user research. You can work with them to work out what they’d like to see from you, and how you can help each other.

Tailor what you’re saying to different teams, especially if you’re talking to people with vastly different skill sets. For example, developers and designers are likely to see entirely different value in research.

Collect user insights across the organization

Putting together a comprehensive internal repository focused specifically on user research is another excellent way to grow awareness. It can also help to quantify things that may otherwise fall by the wayside. For example, you can measure the magnitude of certain pain points or observe patterns in feature requests. Using a platform like Notion or Confluence (or even Google Drive if you don’t want a dedicated platform), log all of your study notes, insights and research information that you find useful.

Whenever someone wants to learn more about research within the organization, they’ll be able to find everything easily.

Bring stakeholders along to research sessions

Getting a stakeholder along to a research session (usability tests and user interviews are great starting points) will help to show them the value that face-to-face sessions with users can provide.

To really involve an observer in your UX research, assign them a specific role. Note taker, for example. With a short briefing on best-practices for note taking, they can get a feel for what’s like to do some of the work you do.

You may also want to consider bringing anyone who’s interested along to a research session, even if they’re just there to observe.

Share your findings – consistently

Research is about more than just testing a hypothesis, it’s important to actually take your research back to the people who can action the data.

By sharing your research findings with teams and stakeholders regularly, your organization will start to build up an understanding of the value that ongoing research can provide, meaning getting approval to pursue research in future becomes easier. This is a bit of a chicken and egg situation, but it’s a practice that all researchers need to get into – especially those embedded in large teams or organizations.

Anything else you think is worth mentioning? Let us know in the comments.

Read more

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.