Quantifying the value of User Research in 2024 

What happens when you mix New Zealand's finest chocolate with 870 of Europe's brightest UX minds? Pure magic, as we discovered at UXDX Dublin 2024!

‍

‍

A sweet start

‍

Our UXDX journey began with pre-event drinks (courtesy of yours truly, Optimal Workshop) and a special treat from down under - a truckload of Whittaker's chocolate that quickly became the talk of the conference. Our impromptu card sorting exercise with different Whittaker's flavors revealed some interesting preferences, with Coconut Slab emerging as the clear favorite among attendees!

‍

‍

Cross-Functional Collaboration: More Than Just a Buzzword

‍

The conference's core theme of breaking down silos between design, product, and engineering teams resonated deeply with our mission at Optimal Workshop. Andrew Birgiolas from Sephora delivered what I call a "magical performance" on collaboration as a product, complete with an unforgettable moment where he used his shoe to demonstrate communication scenarios (now that's what we call thinking on your feet!).

‍

‍

Purpose-driven design

‍

Frank Gaine's session on organizational purpose was a standout moment, emphasizing the importance of alignment at three crucial levels:

‍

- Company purpose

- Team purpose

- Individual purpose

‍

This multi-layered approach to purpose struck a chord with attendees, reminding us that effective UX research and design must be anchored in clear, meaningful objectives at every level.

‍

‍

The art of communication

‍

One of the most practical takeaways came from Kelle Link's session on navigating enterprise ecosystems. Her candid discussion about the necessity of becoming proficient in deck creation sparked knowing laughter from the audience. As our CEO noted, it's a crucial skill for communicating with senior leadership, board members, and investors - even if it means becoming a "deck ninja" (to use a more family-friendly term).

‍

‍

Standardization meets innovation

‍

Chris Grant's insights on standardization hit home: "You need to standardize everything so things are predictable for a team." This seemingly counterintuitive approach to fostering innovation resonated with our own experience at Optimal Workshop - when the basics are predictable, teams have more bandwidth for tackling the unpredictable challenges that drive real innovation.

‍

‍

Building impactful product teams

‍

Matt Fenby-Taylor's discussion of the "pirate vs. worker bee" persona balance was particularly illuminating. Finding team members who can maintain that delicate equilibrium between creative disruption and methodical execution is crucial for building truly impactful product teams.

‍

‍

Research evolution

‍

A key thread throughout the conference was the evolution of UX research methods. Nadine Piecha's "Beyond Interviews" session emphasized that research is truly a team sport, requiring involvement from designers, PMs, and other stakeholders. This aligns perfectly with our mission at Optimal Workshop to make research more accessible and actionable for everyone.

‍

‍

The AI conversation

‍

The debate on AI's role in design and research between John Cleere and Kevin Hawkins sparked intense discussions. The consensus? AI will augment rather than replace human researchers, allowing us to focus more on strategic thinking and deeper insights - a perspective that aligns with our own approach to integrating AI capabilities.

‍

‍

Looking ahead

‍

As we reflect on UXDX 2024, a few things are clear:

‍

1. The industry is evolving rapidly, but the fundamentals of human-centered design remain crucial

‍

2. Cross-functional collaboration isn't just nice to have - it's essential for delivering impactful products

‍

3. The future of UX research and design is bright, with teams becoming more integrated and methodologies more sophisticated

‍

‍

The power of community

‍

Perhaps the most valuable aspect of UXDX wasn't just the formal sessions, but the connections made over coffee (which we were happy to provide!) and, yes, New Zealand chocolate. The mix of workshops, forums, and networking opportunities created an environment where ideas could flow freely and partnerships could form naturally.

‍

‍

What's next?

‍

As we look forward to UXDX 2025, we're excited to see how these conversations evolve. Will AI transform how we approach UX research? How will cross-functional collaboration continue to develop? And most importantly, which Whittaker's chocolate flavor will reign supreme next year?

‍

One thing's for certain - the UX community is more vibrant and collaborative than ever, and we're proud to be part of its evolution. I’ve said it before and I’ll say it again, the industry has a very bright future. 

‍

See you next year! We’ll remember to bring more Coconut Slab chocolate next time - it seems we've created quite a demand!

‍

Knowing and understanding why and how your users use your product is invaluable for getting to the nitty gritty of usability. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative. 

‍

Put super simply, usability testing literally is testing how usable your product is for your users. If your product isn’t usable users often won’t complete their task, let alone come back for more. No one wants to lose users before they even get started. Usability testing gets under their skin and really into the how, why and what they want (and equally what they don’t).

‍

As we have been getting used to video calling regularly and using the internet for interactions, usability testing has followed suit. Being able to access participants remotely has allowed us to diversify the participant pool by not being restricted to those that are close enough to be in-person. This has also allowed an increase in the number of participants per test, as it becomes more cost-effective to perform remote usability testing.

‍

But if we’re remote, does this mean it can’t be moderated? No - remote testing, along with modern technology, can mean that remote testing can be facilitated and moderated. But what is the best method - moderated or unmoderated?

‍

What is moderated remote research testing?

‍

In traditional usability testing, moderated research is done in person. With the moderator and the participant in the same physical space. This, of course, allows for conversation and observational behavioral monitoring. Meaning the moderator can note not only what the participant answers but how and even make note of the body language, surroundings, and other influencing factors. 

‍

This has also meant that traditionally, the participant pool has been limited to those that can be available (and close enough) to make it into a facility for testing. And being in person has meant it takes time (and money) to perform these tests.

‍

As technology has moved along and the speed of internet connections and video calling has increased, this has opened up a world of opportunities for usability testing. Allowing usability testing to be done remotely. Moderators can now set up testing remotely and ‘dial in’ to observe participants anywhere they are. And potentially even running focus groups or other testing in a group format across the internet. 

‍

Pros of moderated remote research testing:

‍

- In-depth gathering of insights through a back-and-forth conversation and observing of the participants.

‍

- Follow-up questions don’t underestimate the value of being available to ask questions throughout the testing. And following up in the moment.

‍

- Observational monitoring noticing and noting the environment and how the participants are behaving, can give more insight into how or why they choose to make a decision.

‍

- Quick remote testing can be quicker to start, find participants, and complete than in-person. This is because you only need to set up a time to connect via the internet, rather than coordinating travel times, etc.

‍

- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing. This broadens your ability to broaden the pool, and participants can be either within your country or global. 

‍

Cons of moderated remote research testing:

‍

- Time-consuming having to be present at each test takes time. As does analyzing the data and insights generated. But remember, this is quality data.

‍

- Limited interactions with any remote testing there is only so much you can observe or understand across the window of a computer screen. It can be difficult to have a grasp on all the factors that might be influencing your participants.

‍

What is unmoderated remote research testing?

‍

In its most simple sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants are left to complete the testing by themselves and in their own time. For the most part, everything else stays the same. 

‍

Removing the moderator, means that there isn’t anyone to respond to queries or issues in the moment. This can either delay, influence, or even potentially force participants to not complete or maybe not be as engaged as you may like. Unmoderated research testing suits a very simple and direct type of test. With clear instructions and no room for inference. 

‍

Pros of unmoderated remote research testing:

‍

- Speed and turnaround,  as there is no need to schedule meetings with each and every participant. Unmoderated usability testing is usually much faster to initiate and complete.

‍

- Size of study (participant numbers) unmoderated usability testing allows you to collect feedback from dozens or even hundreds of users at the same time. 

- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing, which broadens your participant pool.  And unmoderated testing means that it literally can be anywhere while participants complete the test in their own time.

‍

Cons of unmoderated remote research testing:

‍

- Follow-up questions as your participants are working on their own and in their own time, you can’t facilitate and ask questions in the moment. You may be able to ask limited follow-up questions.

‍

- Products need to be simple to use unmoderated testing does not allow for prototypes or any product or site that needs guidance. 

‍

- Low participant support without the moderator any issues with the test or the product can’t be picked up immediately and could influence the output of the test.

‍

When should you do moderated vs unmoderated remote usability testing?

‍

Each moderated and unmoderated remote usability testing have its use and place in user research. It really depends on the question you are asking and what you are wanting to know.

‍

Moderated testing allows you to gather in-depth insights, follow up with questions, and engage the participants in the moment. The facilitator has the ability to guide participants to what they want to know, to dig deeper, or even ask why at certain points. This method doesn’t need as much careful setup as the participants aren’t on their own. While this is all done online, it does still allow connection and conversation. This method allows for more investigative research. Looking at why users might prefer one prototype to another. Or possibly tree testing a new website navigation to understand where they might get lost and querying why the participant made certain choices.

‍

Unmoderated testing, on the other hand, is literally leaving the participants to it. This method needs very careful planning and explaining upfront. The test needs to be able to be set and run without a moderator. This lends itself more to wanting to know a direct answer to a query. Such as a card sort on a website to understand how your users might sort information. Or a first click to see how/where users will click on a new website.

‍

Planning your next user test? Here’s how to choose the right method

‍

With the ability to expand our pool of participants across the globe with all of the advances (and acceptance of) technology and video calling etc, the ability to expand our understanding of users’ experiences is growing. Remote usability testing is a great option when you want to gather information from users in the real world. Depending on your query, moderated or unmoderated usability testing will suit your study. As with all user testing, being prepared and planning ahead will allow you to make the most of your test.

Many organizations are aware that staying relevant essential for their success. This can mean a lot of things to different organizations. What it often means is coming up with plenty of new, innovative ideas and products to keep pace with the demands and needs of the marketplace. It also means keeping up with the expectations and needs of your users, which often means  shorter and shorter product development life cycle times.  While maintaining this pace can be daunting, it can also be seen as a strength, tightening up your processes and cutting out unnecessary steps.

‍

A vital part of developing new (or tweaking existing) products is considering the end user first. There really is no point in creating anything new if it isn’t meeting a need or filling a gap in the market. How can you make sure you are hitting the right mark? Ask your users.  We look into some of the key user research methods available to help you in your product development process.

‍

If you want to know more about how to fit research into your product development process, take a read here.

‍

What is user research? 👨🏻💻

‍

User experience (UX) research, or user research as it’s commonly referred to, is an important part of the product development process. Primarily, UX research involves using different research methods to gather qualitative and quantitative data and insights about how your users interact with your product. It is an essential part of developing, building, and launching a product that truly meets the needs, desires, and requirements of your users. 

‍

At its simplest, user research is talking to your users and understanding what they want and why. And using this to deliver what they need.

‍

How does user research fit into the product development process? 🧩🧩

‍

User research is an essential part of the product development process. By asking questions of your users about how your product works and what place it fills in the market, you can create a product that delivers what the market needs to those who need it. 

‍

Without user research, you could literally be firing arrows in the dark, or at the very best, working from a very internal organizational view based on assuming that what you believe users need is what they want. With user research, you can collect qualitative and quantitative data that clearly tells you where and what users would like to see and how they would use it.

‍

Investing in user research right at the start of the product development process can save the team and the organization heavy investment in time and money. With detailed data responses, your brand-new product can leapfrog many development hurdles, delivering a final product that users love and want to keep using. Firing arrows to hit a bullseye.

‍

What user research methods should we use? 🥺

‍

Qualitative ResearchMethods

Qualitative research is about exploration. It focuses on discovering things we cannot measure with numbers and typically involves getting to know users directly through interviews or observation.

‍

Usability Testing – Observational

One of the best ways to learn about your users and how they interact with your new product is to observe them in their own environment. Watch how they accomplish tasks, the order they do things, what frustrates them, and what makes the task easier and/or more enjoyable for your subject. The data can be collated to inform the usability of your product, improving intuitive design and what resonates with your users.

‍

Competitive Analysis

Reviewing products already on the market can be a great start to the product development process. Why are your competitors’ products successful? And how well do they behave for users? Learn from their successes, and even better, build on where they may not be performing as well and find where your product fills the gap in the market.

‍

Quantitative Research Methods

Quantitative research is about measurement. It focuses on gathering data and then turning this data into usable statistics.

‍

Surveys

Surveys are a popular user research method for gathering information from a wide range of people. In most cases, a survey will feature a set of questions designed to assess someone’s thoughts on a particular aspect of your new product. They’re useful for getting feedback or understanding attitudes, and you can use the learnings from your survey of a subset of users to draw conclusions about a larger population of users.

‍

Wrap Up 🌯

‍

Gathering information on your users during the product development process and before you invest time and money can be hugely beneficial to the entire process. Collating robust data and insights to guide the new product development and respond directly to user needs, and filling that all-important niche. Undertaking user experience research shouldn’t stop at product development but throughout each and every step of your product life cycle. If you want to find out more about UX research throughout the life cycle of your product, take a read of our article UX research for each product phase.