Mixed methods research in 2021

Return on investment (ROI) is often the term on everyone’s lips when starting a big project or even when reviewing a website. It’s especially popular with those that hold the purse strings.  As UX researchers it is important to consider the ROI of the work we do and understand how to measure this. 

‍

We’ve lined up 5 key ways to measure ROI for UX research to help you get the conversation underway with stakeholders so you can show real and tangible benefits to your organization. 

‍

‍

1. Meet and exceed user expectations

‍

Put simply, a product that meets and exceeds user expectations leads to increased revenue. When potential buyers are able to find and purchase what they’re looking for, easily, they’ll complete their purchase, and are far more likely to come back. The simple fact that users can finish their task will increase sales and improve overall customer satisfaction which has an influence on their loyalty. Repeat business means repeat sales. Means increased revenue.

‍

Creating, developing and maintaining a usable website is more important than you might think. And this is measurable! Tracking and analyzing website performance prior to the UX research and after can be insightful and directly influenced by changes made based on UX research.

‍

Measurable: review the website (product) performance prior to UX research and after changes have been made. The increase in clicks, completed tasks and/or baskets will tell the story.

‍

‍

2. Reduce development time

‍

UX research done at the initial stages of a project can lead to a reduction in development time of by 33% to 50%! And reduced time developing, means reduced costs (people and overheads) and a speedier to market date. What’s not to love? 

‍

Measurable: This one is a little more tricky as you have saved time (and cost) up front. Aiding in speed to market and performance prior to execution. Internal stakeholder research may be of value post the live date to understand how the project went.

‍

‍

3. Ongoing development costs

‍

And the double hitter? Creating a product that has the user in mind up front, reduces the need to rehash or revisit as quickly. Reducing ongoing costs. Early UX research can help with the detection of errors early on in the development process. Fixing errors after development costs a company up to 100 times more than dealing with the same error before development.

‍

Measureable: Again, as UX research has saved time and money up front this one can be difficult to track. Though depending on your organization and previous projects you could conduct internal research to understand how the project compares and the time and cost savings.

‍

‍

4. Meeting user requirements

‍

Did you know that 70% of projects fail due to the lack of user acceptance? This is often because project managers fail to understand the user requirements properly. Thanks to UX research early on, gaining insights into users and only spending time developing the functions users actually want, saving time and reducing development costs. Make sure you get confirmation on those requirements by iterative testing. As always, fail early, fail often. Robust testing up front means that in the end, you’ll have a product that will meet the needs of the user.

‍

Measurable: Where is the product currently? How does it perform? Set a benchmark up front and review post UX research. The deliverables should make the ROI obvious.

‍

‍

5. Investing in UX research leads to an essential competitive advantage.

‍

Thanks to UX research you can find out exactly what your customers want, need and expect from you. This gives you a competitive advantage over other companies in your market. But you should be aware that more and more companies are investing in UX while customers are ever more demanding, their expectations continue to grow and they don’t tolerate bad experiences. And going elsewhere is an easy decision to make.

‍

Measurable: Murky this one, but no less important. Knowing, understanding and responding to competitors can help keep you in the lead, and developing products that meet and exceed those user expectations.

‍

‍

Wrap up

‍

Showing the ROI on the work we do is an essential part of getting key stakeholders on board with our research. It can be challenging to talk the same language, ultimately we all want the same outcome…a product that works well for our users, and delivers additional revenue.

‍

For some continued reading (or watching in this case), Anna Bek, Product and Delivery Manager at Xplor explored the same concept of "How to measure experience" during her UX New Zealand 2020 – watch it here as she shares a perspective on UX ROI.

What is UX research?

‍

User experience (UX) research, or user research as it’s commonly referred to, is an important part of the product design process. Primarily, UX research involves using different research methods to gather information about how your users interact with your product. It is an essential part of developing, building and launching a product that truly meets the requirements of your users. 

‍

UX research is essential at all stages of a products' life cycle:

‍

1. Planning
2. Building
3. Introduction
4. Growth & Maturity

‍

While there is no one single time to conduct UX research it is best-practice to continuously gather information throughout the lifetime of your product. The good news is many of the UX research methods do not fit just one phase either, and can (and should) be used repeatedly. After all, there are always new pieces of functionality to test and new insights to discover. We introduce you to best-practice UX research methods for each lifecycle phase of your product.

‍

‍

‍

1. Product planning phase

‍

While the planning phase it is about creating a product that fits your organization, your organization’s needs and meeting a gap in the market it’s also about meeting the needs, desires and requirements of your users. Through UX research you’ll learn which features are necessary to be aligned with your users. And of course, user research lets you test your UX design before you build, saving you time and money.

‍

‍

Qualitative Research Methods

‍

Usability Testing - Observational

‍

One of the best ways to learn about your users and how they interact with your product is to observe them in their own environment. Watch how they accomplish tasks, the order they do things, what frustrates them, and what makes the task easier and/or more enjoyable for your subject. The data can be collated to inform the usability of your product, improving intuitive design, and what resonates with users.

‍

Competitive Analysis

‍

Reviewing products already in the market can be a great start to the planning process. Why are your competitors’ products successful and how well do they behave for users. Learn from their successes, and even better build on where they may not be performing the best and find your niche in the market.

‍

‍

Quantitative Research Methods

‍

Surveys and Questionnaires

‍

Surveys are useful for collecting feedback or understanding attitudes. You can use the learnings from your survey of a subset of users to draw conclusions about a larger population of users.

‍

There are two types of survey questions:
‍

Closed questions are designed to capture quantitative information. Instead of asking users to write out answers, these questions often use multi-choice answers.

‍

Open questions are designed to capture qualitative information such as motivations and context.  Typically, these questions require users to write out an answer in a text field.

‍

‍

‍

2. Product building phase

‍

Once you've completed your product planning research, you’re ready to begin the build phase for your product. User research studies undertaken during the build phase enable you to validate the UX team’s deliverables before investing in the technical development.

‍

‍

Qualitative Research Methods

‍

Focus groups

‍

Generally involve 5-10 participants and include demographically similar individuals. The study is set up so that members of the group can interact with one another and can be carried out in person or remotely.
‍

‍
Besides learning about the participants’ impressions and perceptions of your product, focus group findings also include what users believe to be a product’s most important features, problems they might encounter while using the product, as well as their experiences with other products, both good and bad.

‍

‍

Quantitative Research Methods

‍

Card sorting gives insight into how users think. Tools like card sorting reveal where your users expect to find certain information or complete specific tasks. This is especially useful for products with complex or multiple navigations and contributes to the creation of an intuitive information architecture and user experience.

‍

Tree testing gives insight into where users expect to find things and where they’re getting lost within your product. Tools like tree testing help you test your information architecture.
Card sorting and tree testing are often used together. Depending on the purpose of your research and where you are at with your product, they can provide a fully rounded view of your information architecture.

‍

‍

‍

3. Product introduction phase

‍

You’ve launched your product, wahoo! And you’re ready for your first real life, real time users. Now it’s time to optimize your product experience. To do this, you’ll need to understand how your new users actually use your product.

‍

‍

Qualitative Research Methods

‍

Usability testing involves testing a product with users. Typically it involves observing users as they try to follow and complete a series of tasks. As a result you can evaluate if the design is intuitive and if there are any usability problems.

‍

User Interviews - A user interview is designed to get a deeper understanding of a particular topic. Unlike a usability test, where you’re more likely to be focused on how people use your product, a user interview is a guided conversation aimed at better understanding your users. This means you’ll be capturing details like their background, pain points, goals and motivations.

‍

‍

Quantitative Research Methods

‍

A/B Testing is a way to compare two versions of a design in order to work out which is more effective. It’s typically used to test two versions of the same webpage, for example, using a different headline, image or call to action to see which one converts more effectively. This method offers a way to validate smaller design choices where you might not have the data to make an informed decision, like the color of a button or the layout of a particular image.

‍

Flick-click testing shows you where people click first when trying to complete a task on a website. In most cases, first-click testing is performed on a very simple wireframe of a website, but it can also be carried out on a live website using a tool like first-time clicking.

‍

‍

‍

4. Growth and maturity phase

‍

If you’ve reached the growth stage, fantastic news! You’ve built a great product that’s been embraced by your users. Next on your to-do list is growing your product by increasing your user base and then eventually reaching maturity and making a profit on your hard work.

‍

Growing your product involves building new or advanced features to satisfy specific customer segments. As you plan and build these enhancements, go through the same research and testing process you used to create the first release. The same holds true for enhancements as well as a new product build — user research ensures you’re building the right thing in the best way for your customers.

‍

‍

Qualitative research methods

‍

User interviews will focus on how your product is working or if it’s missing any features, enriching your knowledge about your product and users.

‍

It allows you to test your current features, discover new possibilities for additional features and think about discarding  existing ones. If your customers aren’t using certain features, it might be time to stop supporting them to reduce costs and help you grow your profits during the maturity stage.

‍

‍

Quantitative research methods

‍

Surveys and questionnaires can help gather information around which features will work best for your product, enhancing and improving the user experience. 

‍

A/B testing during growth and maturity occurs within your sales and onboarding processes. Making sure you have a smooth onboarding process increases your conversion rate and reduces wasted spend — improving your bottom line.

‍

‍

‍

Final Thoughts: Why Continuous UX Research Matters

‍

UX research testing throughout the lifecycle of your product helps you continuously evolve and develop a product that responds to what really matters - your users.

‍

Talking to, testing, and knowing your users will allow you to push your product in ways that make sense with the data to back up decisions. Go forth and create the product that meets your organizations needs by delivering the very best user experience for your users.

There’s not much a parent won’t do to ensure their child has the best chance of succeeding in life. Unsurprisingly, things are much the same in product development. Whether it’s a designer, manager, developer or copywriter, everyone wants to see the product reach its full potential.

‍

Key to a product’s success (even though it’s still not widely practiced) is UX research. Without research focused on learning user pain points and behaviors, development basically happens in the dark. Feeding direct insights from customers and users into the development of a product means teams can flick the light on and make more informed design decisions.

‍

While the benefits of user research are obvious to anyone working in the field, it can be a real challenge to convince others of just how important and useful it is. We thought we’d help.

‍

Define user research

‍

If you want to sell the importance of UX research within your organization, you’ve got to ensure stakeholders have a clear understanding of what user research is and what they stand to gain from backing it.

‍

In general, there are a few key things worth focusing on when you’re trying to explain the benefits of research:

‍

• More informed design decisions: Companies make major design decisions far too often without considering users. User research provides the data needed to make informed decisions.
• Less uncertainty and risk: Similarly, research reduces risk and uncertainty simply by giving companies more clarity around how a particular product or service is used.
• Retention and conversion benefits: Research means you’ll be more aligned with the needs of your customers and prospective customers.

‍

Use the language of the people you’re trying to convince. A capable UX research practice will almost always improve key business metrics, namely sales and retention.

‍

The early stages

‍

When embarking on a project, book in some time early in the process to answer questions, explain your research approach and what you hope to gain from it. Here are some of the key things to go over:

‍

• Your objectives: What are you trying to achieve? This is a good time to cover your research questions.
• Your research methods: Which methods will you be using to carry out your research? Cover the advantages of these methods and the information you’re likely to get from using them.
• Constraints: Do you see any major obstacles? Any issues with resources?
• Provide examples: Nothing shows the value of doing research quite like a case study. If you can’t find an example of research within your own organization, see what you can find online.

‍

Involve others in your research

‍

When trying to convince someone of the validity of what you’re doing, it’s often best to just show them. There are a couple of effective ways you can do this – at a team or individual level and at an organizational level.

‍

We’ll explain the best way to approach this below, but there’s another important reason to bring others into your research. UX research can’t exist in a vacuum – it thrives on integration and collaboration with other teams. Importantly, this also means working with other teams to define the problems they’re trying to solve and the scope of their projects. Once you’ve got an understanding of what they’re trying to achieve, you’ll be in a better position to help them through research.

‍

Educate others on what research is

‍

Education sessions (lunch-and-learns) are one of the best ways to get a particular team or group together and run through the what and why of user research. You can work with them to work out what they’d like to see from you, and how you can help each other.

‍

Tailor what you’re saying to different teams, especially if you’re talking to people with vastly different skill sets. For example, developers and designers are likely to see entirely different value in research.

‍

Collect user insights across the organization

‍

Putting together a comprehensive internal repository focused specifically on user research is another excellent way to grow awareness. It can also help to quantify things that may otherwise fall by the wayside. For example, you can measure the magnitude of certain pain points or observe patterns in feature requests. Using a platform like Notion or Confluence (or even Google Drive if you don’t want a dedicated platform), log all of your study notes, insights and research information that you find useful.

‍

Whenever someone wants to learn more about research within the organization, they’ll be able to find everything easily.

‍

Bring stakeholders along to research sessions

‍

Getting a stakeholder along to a research session (usability tests and user interviews are great starting points) will help to show them the value that face-to-face sessions with users can provide.

‍

To really involve an observer in your UX research, assign them a specific role. Note taker, for example. With a short briefing on best-practices for note taking, they can get a feel for what’s like to do some of the work you do.

‍

You may also want to consider bringing anyone who’s interested along to a research session, even if they’re just there to observe.

‍

Share your findings – consistently

‍

Research is about more than just testing a hypothesis, it’s important to actually take your research back to the people who can action the data.

‍

By sharing your research findings with teams and stakeholders regularly, your organization will start to build up an understanding of the value that ongoing research can provide, meaning getting approval to pursue research in future becomes easier. This is a bit of a chicken and egg situation, but it’s a practice that all researchers need to get into – especially those embedded in large teams or organizations.

‍

Anything else you think is worth mentioning? Let us know in the comments.

‍

Read more

‍

• Selling your design recommendations to clients and colleagues – Guest writer Jeff Tang outlines some of his techniques for presenting UX recommendations and answering objections.
• Quantifying the ROI of UX – Ashlea McKay delves into one of the toughest UX questions to answer: “What do we get for our money?”
• How to lead a UX team – With user-centered design growing in organizations around the world, we’ll need capable leaders to run UX teams.
• How do I explain what UX is? – Ashlea McKay covers how you can explain UX to people who may not necessarily have familiarity with the field.