Quantifying the value of User Research in 2024 

Making a difference through UX was a shared passion among an impressive line-up of 7 researchers, strategists, and designers from the global UX community at this year’s 100% virtual 3-day UX New Zealand conference.

‍

‍

1. From bombs to bots: the evolving landscape of frontline research

‍

These days Darya Pilram, Senior Researcher at Twitter, spends her days trying to understand the motivation and techniques of groups who ‘hire’ technology to spread harmful narratives.  The desert of Mogadishu and the urban conflicts of South Africa are just some of the unlikely places she’s leveraged the power of frontline research to create change.

‍

"I realized the only way to influence change was by bringing folks along with me - and so I did.  I bought them right into the field with me."

‍

‍

‍

‍

2. Beautifully accessible: why embracing inclusive design shouldn’t hold back your creativity

‍

Experience Designer Beth McPhail refuses to buy into the mindset that ‘accessibility is a creativity killer’. She challenges her peers to view accessibility as an opportunity to grow creatively while making technology more inclusive.

‍

“Accessibility is making it possible for someone to attend the party…and lose themselves in the music.”

‍

‍

‍

‍

3. Innovating within the Justice sector | Part 2: For a fairer start - design’s role in shaping mana enhancing social & systemic change.

‍

Kelsey Gee is back challenging designers across all levels to think differently about how design can be used across different mediums and constraints to generate meaningful experiences and meaningful change.  In this session, she explores design’s role in creating empowering experiences that break both cycles of crime and institutional racism. (If you missed Part 1 from Mini Con head over here)

‍

"I truly believe that our superpower lies in our ability to redesign society, especially for our whanau and our most vulnerable communities…and once again explore design’s role in creating equal opportunities across safe, seamless, and healing public services."

‍

‍

‍

‍

4. First do no harm: make your designs more trauma-informed and survivor sensitive

‍

In 1985, a researcher botched an interview question which led to a new understanding of trauma and its long-term effects. It grew awareness of the need to be trauma-informed in your work but what’s it actually mean?  UX Researcher Melissa Eggleston explores what it means to be trauma-informed and shares practical advice on how to achieve it.

‍

"Trauma is everywhere and something for us to think about…regardless of whether we’re working with people we know are dealing with traumatic events…it’s really all over the place."

‍

‍

‍

5. Changing the way we design high-risk products to make meaningful impact

‍

One in five people experiences “mental illness or significant mental distress” in New Zealand.  It’s a problem the Government knows needs to be addressed but how? In her powerful presentation, Rachael Reeves reveals what’s involved in balancing the complexities of Government with the need to rethink the way we design health products.

‍

"Be warned you can’t please everyone and it can be tough to keep product vision aligned when you’re talking about serious consequences for people."

‍

‍

‍

6. Remote research with new internet users (yes you can!)

‍

One billion new internet users (NIU) will come online for the first time over the next 5 years. These NIU's are using their first smartphones, with most of their online activities focused on communication, maintaining social connections, and entertainment. Tiane Lee, UX Research Lead at Google outlines the challenges and considerations behind adapting research for varying levels of digital literacy, including practical ideas for planning and conducting remote research with NIU.

‍

"NIU’s are typically less digitally literate, they may show lower confidence in digital capability, and they may perceive lower value of the internet for things like chatting and entertainment.”

‍

‍

‍

‍

7. Conditions Design: weaving the invisible threads of service design, value orchestration, and culture building  

‍

Michael Tam introduces us to the niche field of conditions design and cites a purpose built high diving board on Wellington city’s busy waterfront in New Zealand as a good example of conditions design.  Find out why in this fascinating talk.

‍

"What really impressed me here…hats off to the council because they didn’t design an experience that would discourage people from doing it. It’s designed for people to have fun (vs Hong Kong where public spaces are designed for Tai Chi not fun like this). The design allows it to happen by influencing human behavior to stay safe but encouraging fun and exploration.”

‍

‍

‍

‍

For a taste of what even more speakers from UX New Zealand 2022 had to share, head over to our highlights reel

Knowing and understanding why and how your users use your product can be invaluable for getting to the nitty gritty of usability. Where they get stuck and where they fly through. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative.

‍

Usability testing can be done in several ways, each way has its benefits. Put super simply, usability testing literally is testing how useable your product is for your users. If your product isn't useable users will not stick around or very often complete their task, let alone come back for more.

‍

‍

What is usability testing?

‍

Usability testing is a research method used to evaluate how easy something is to use by testing it with representative users.

‍

These tests typically involve observing a participant as they work through a series of tasks involving the product being tested. Having conducted several usability tests, you can analyze your observations to identify the most common issues.

‍

We go into the three main methods of usability testing:

1. Moderated and unmoderated
2. Remote or in person
3. Explorative, assessment or comparative

‍

‍

1. Moderated or unmoderated usability testing

‍

Moderated usability testing

Moderated usability testing is done in-person or remotely by a researcher who introduces the test to participants, answers their queries, and asks follow-up questions. Often these tests are done in real time with participants and can involve other research stakeholders. Moderated testing usually produces more in-depth results thanks to the direct interaction between researchers and test participants. However, this can be expensive to organize and run.

‍

Top tip: Use moderated testing to investigate the reasoning behind user behavior.

‍

Unmoderated usability testing

‍
Unmoderated usability testing is done without direct supervision; likely participants are in their own homes and/or using their own devices to browse the website that is being tested. And often at their own pace.  The cost of unmoderated testing is lower, though participant answers can remain superficial and making follow-up questions can be difficult.

‍

Top tip: Use unmoderated testing to test a very specific question or observe and measure behavior patterns.

‍

‍

2. Research or in-person usability testing

‍

Remote usability testing

‍
‍Remote usability testing is done over the internet or by phone. Allowing the participants to have the time and space to work in their own environment and at their own pace. This however doesn’t give the researcher much in the way of contextual data because you’re unable to ask questions around intention or probe deeper if the participant makes a particular decision. Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources.

‍

Top tip: Use remote testing when a large group of participants are needed and the questions asked can be direct and unambiguous.

‍

In-person usability testing

‍
‍In-person usability testing, as the name suggests, is done in the presence of a researcher. In-person testing does provide contextual data as researchers can observe and analyze body language and facial expressions. You’re also often able to converse with participants and find out more about why they do something. However, in-person testing can be expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and often pay) participants.

‍

Top tip: In-person testing gives researchers more time and insight into motivation for decisions.

‍

‍

3. Explorative, Assessment or comparative testing

‍

These three usability testing methods generate different types of information:

‍

Explorative testing

‍
‍Explorative testing is open-ended. Participants are asked to brainstorm, give opinions, and express emotional impressions about ideas and concepts. The information is typically collected in the early stages of product development and helps researchers pinpoint gaps in the market, identify potential new features, and workshop new ideas.

‍

Assessment research

‍
‍Assessment research is used to test a user's satisfaction with a product and how well they are able to use it. It's used to evaluate general functionality.

‍

Comparative research

‍
‍Comparative research methods involve asking users to choose which of two solutions they prefer, and they may be used to compare a product with its competitors.

‍

Top tip: Depending on what research is being done, and how much qualitative or quantitative data is wanted.

‍

‍

Which method is right for you?

‍

Whether the testing is done in-person, remote, moderated or unmoderated will depend on your purpose, what you want out of the testing, and to some extent your budget. 

‍

Depending on what you are testing, each of the usability testing methods we explored here can offer an answer. If you are at the development stage of a product it can be useful to conduct a usability test on the entire product. Checking the intuitive usability of your website, to ensure users can make the best decisions, quickly. Or adding, changing or upgrading a product can also be the moment to check on a specific question around usability. Planning and understanding your objectives are key to selecting the right usability testing option for your project.

‍

Let's take a look at a couple of examples of usability testing.

‍

‍

1. Lab based, in-person moderated testing - mid-life website

‍

Imagine you have a website that sells sports equipment. Over time your site has become cluttered and disorganized, much like a bricks and mortar store may. You’ve noticed a drop in sales in certain areas. How do you find out what is going wrong or where users are getting lost? Having an in-person, lab (or other controlled environment), moderated usability test with users you can set tasks, watch (and record) what they do.

‍

The researcher can literally be standing or sitting next to the participant throughout, recording contextual information such as how they interacted with the mouse, laptop or even the seat. Watching for cues as to the comfort of the participant and asking questions about why they make decisions can provide richer insights. Maybe they wanted purple yoga pants, but couldn’t find the ‘yoga’ section which was listed under gym rather than a clothing section.

‍

Meaning you can look at how your stock is organised, or even investigate undertaking a card sort. This provides robust and fully rounded feedback on users behaviours, expectations and experiences. Providing data that can directly be turned into actionable directives when redeveloping the website. 

‍

‍

2. Remote, moderated assessment testing - app product development

‍

You are looking at launching an app for parents to access for information and updates for the school. It’s still in development stage and at this point you want to know how easy the app is to use. Setting some very specific set tasks for participants to complete the app can be sent to them and they can be left to complete (or not). Providing feedback and comments around the usability.

‍

The next step may be to use first click testing to see how and where the interface is clicked and where participants may be spending time, or becoming lost. Whilst the feedback and data gathered from this testing can be light, it will be very direct to the questions asked. And will provide data to back up (or possibly not) what assumptions were made.

‍

‍

3. Moderated, In-person, explorative testing - new product development

‍

You’re right at the start of the development process. The idea is new and fresh and the basics are being considered. What better way to get an understanding of what your users’ truly want than an explorative study.

‍

Open-ended questions with participants in a one-on-one environment (or possibly in groups) can provide rich data and insights for the development team. Imagine you have an exciting new promotional app that you are developing for a client. There are similar apps on the market but none as exciting as what your team has dreamt up. By putting it (and possibly the competitors) to participants they can give direct feedback on what they like, love and loathe.

‍

They can also help brainstorm ideas or better ways to make the app work, or improve the interface. All of this done, before there is money sunk in development.

‍

‍

Usability testing summary: When to use each method (and why)

‍

Key objectives will dictate which usability testing method will deliver the answers to your questions.

‍

Whether it’s in-person, remote, moderated or comparative with a bit of planning you can gather data around your users very real experience of your product. Identify issues, successes and failures. Addressing your user experience with real data, and knowledge can but lead to a more intuitive product.

Introduction

‍

User research is invaluable, but in fast-paced environments, researchers often struggle with tight deadlines, limited resources, and the need to prove their impact. In our recent UX Insider webinar, Weidan Li, Senior UX Researcher at Seek, shared insights on Efficient Research—an approach that optimizes Speed, Quality, and Impact to maximize the return on investment (ROI) of understanding customers.

‍

At the heart of this approach is the Efficient Research Framework, which balances these three critical factors:

• Speed – Conducting research quickly without sacrificing key insights.
• Quality – Ensuring rigor and reliability in findings.
• Impact – Making sure research leads to meaningful business and product changes.

‍

Within this framework, Weidan outlined nine tactics that help UX researchers work more effectively. Let’s dive in.

‍

‍

1. Time Allocation: Invest in What Matters Most

‍

Not all research requires the same level of depth. Efficient researchers prioritize their time by categorizing projects based on urgency and impact:

• High-stakes decisions (e.g., launching a new product) require deep research.
• Routine optimizations (e.g., tweaking UI elements) can rely on quick testing methods.
• Low-impact changes may not need research at all.

‍

By allocating time wisely, researchers can avoid spending weeks on minor issues while ensuring critical decisions are well-informed.

‍

‍

2. Assistance of AI: Let Technology Handle the Heavy Lifting

‍

AI is transforming UX research, enabling faster and more scalable insights. Weidan suggests using AI to:

• Automate data analysis – AI can quickly analyze survey responses, transcripts, and usability test results.
• Generate research summaries – Tools like ChatGPT can help synthesize findings into digestible insights.
• Speed up recruitment – AI-powered platforms can help find and screen participants efficiently.

‍

While AI can’t replace human judgment, it can free up researchers to focus on higher-value tasks like interpreting results and influencing strategy.

‍

‍

3. Collaboration: Make Research a Team Sport

‍

Research has a greater impact when it’s embedded into the product development process. Weidan emphasizes:

• Co-creating research plans with designers, PMs, and engineers to align on priorities.
• Involving stakeholders in synthesis sessions so insights don’t sit in a report.
• Encouraging non-researchers to run lightweight studies, such as A/B tests or quick usability checks.

‍

When research is shared and collaborative, it leads to faster adoption of insights and stronger decision-making.

‍

‍

4. Prioritization: Focus on the Right Questions

‍

With limited resources, researchers must choose their battles wisely. Weidan recommends using a prioritization framework to assess:

• Business impact – Will this research influence a high-stakes decision?
• User impact – Does it address a major pain point?
• Feasibility – Can we conduct this research quickly and effectively?

‍

By filtering out low-priority projects, researchers can avoid research for research’s sake and focus on what truly drives change.

‍

‍

5. Depth of Understanding: Go Beyond Surface-Level Insights

‍

Speed is important, but efficient research isn’t about cutting corners. Weidan stresses that even quick studies should provide a deep understanding of users by:

• Asking why, not just what – Observing behavior is useful, but uncovering motivations is key.
• Using triangulation – Combining methods (e.g., usability tests + surveys) to validate findings.
• Revisiting past research – Leveraging existing insights instead of starting from scratch.

‍

Balancing speed with depth ensures research is not just fast, but meaningful.

‍

‍

6. Anticipation: Stay Ahead of Research Needs

‍

Proactive researchers don’t wait for stakeholders to request studies—they anticipate needs and set up research ahead of time. This means:

• Building a research roadmap that aligns with upcoming product decisions.
• Running continuous discovery research so teams have a backlog of insights to pull from.
• Creating self-serve research repositories where teams can find relevant past studies.

‍

By anticipating research needs, UX teams can reduce last-minute requests and deliver insights exactly when they’re needed.

‍

‍

7. Justification of Methodology: Explain Why Your Approach Works

‍

Stakeholders may question research methods, especially when they seem time-consuming or expensive. Weidan highlights the importance of educating teams on why specific methods are used:

• Clearly explain why qualitative research is needed when stakeholders push for just numbers.
• Show real-world examples of how past research has led to business success.
• Provide a trade-off analysis (e.g., “This method is faster but provides less depth”) to help teams make informed choices.

‍

A well-justified approach ensures research is respected and acted upon.

‍

‍

8. Individual Engagement: Tailor Research Communication to Your Audience

‍

Not all stakeholders consume research the same way. Weidan recommends adapting insights to fit different audiences:

• Executives – Focus on high-level impact and key takeaways.
• Product teams – Provide actionable recommendations tied to specific features.
• Designers & Engineers – Share usability findings with video clips or screenshots.

‍

By delivering insights in the right format, researchers increase the likelihood of stakeholder buy-in and action.

‍

‍

9. Business Actions: Ensure Research Leads to Real Change

‍

The ultimate goal of research is not just understanding users—but driving business decisions. To ensure research leads to action:

• Follow up on implementation – Track whether teams apply the insights.
• Tie findings to key metrics – Show how research affects conversion rates, retention, or engagement.
• Advocate for iterative research – Encourage teams to re-test and refine based on new data.

‍

Research is most valuable when it translates into real business outcomes.

‍

‍

Final Thoughts: Research That Moves the Needle

‍

Efficient research is not just about doing more, faster—it’s about balancing speed, quality, and impact to maximize its influence. Weidan’s nine tactics help UX researchers work smarter by:

‍
✔️  Prioritizing high-impact work
✔️  Leveraging AI and collaboration
✔️  Communicating research in a way that drives action

‍

By adopting these strategies, UX teams can ensure their research is not just insightful, but transformational.

‍

‍

Watch the full webinar here

‍