Quantifying the value of User Research in 2024 

Making a difference through UX was a shared passion among an impressive line-up of 7 researchers, strategists, and designers from the global UX community at this year’s 100% virtual 3-day UX New Zealand conference.

‍

‍

1. From bombs to bots: the evolving landscape of frontline research

‍

These days Darya Pilram, Senior Researcher at Twitter, spends her days trying to understand the motivation and techniques of groups who ‘hire’ technology to spread harmful narratives.  The desert of Mogadishu and the urban conflicts of South Africa are just some of the unlikely places she’s leveraged the power of frontline research to create change.

‍

"I realized the only way to influence change was by bringing folks along with me - and so I did.  I bought them right into the field with me."

‍

‍

‍

‍

2. Beautifully accessible: why embracing inclusive design shouldn’t hold back your creativity

‍

Experience Designer Beth McPhail refuses to buy into the mindset that ‘accessibility is a creativity killer’. She challenges her peers to view accessibility as an opportunity to grow creatively while making technology more inclusive.

‍

“Accessibility is making it possible for someone to attend the party…and lose themselves in the music.”

‍

‍

‍

‍

3. Innovating within the Justice sector | Part 2: For a fairer start - design’s role in shaping mana enhancing social & systemic change.

‍

Kelsey Gee is back challenging designers across all levels to think differently about how design can be used across different mediums and constraints to generate meaningful experiences and meaningful change.  In this session, she explores design’s role in creating empowering experiences that break both cycles of crime and institutional racism. (If you missed Part 1 from Mini Con head over here)

‍

"I truly believe that our superpower lies in our ability to redesign society, especially for our whanau and our most vulnerable communities…and once again explore design’s role in creating equal opportunities across safe, seamless, and healing public services."

‍

‍

‍

‍

4. First do no harm: make your designs more trauma-informed and survivor sensitive

‍

In 1985, a researcher botched an interview question which led to a new understanding of trauma and its long-term effects. It grew awareness of the need to be trauma-informed in your work but what’s it actually mean?  UX Researcher Melissa Eggleston explores what it means to be trauma-informed and shares practical advice on how to achieve it.

‍

"Trauma is everywhere and something for us to think about…regardless of whether we’re working with people we know are dealing with traumatic events…it’s really all over the place."

‍

‍

‍

5. Changing the way we design high-risk products to make meaningful impact

‍

One in five people experiences “mental illness or significant mental distress” in New Zealand.  It’s a problem the Government knows needs to be addressed but how? In her powerful presentation, Rachael Reeves reveals what’s involved in balancing the complexities of Government with the need to rethink the way we design health products.

‍

"Be warned you can’t please everyone and it can be tough to keep product vision aligned when you’re talking about serious consequences for people."

‍

‍

‍

6. Remote research with new internet users (yes you can!)

‍

One billion new internet users (NIU) will come online for the first time over the next 5 years. These NIU's are using their first smartphones, with most of their online activities focused on communication, maintaining social connections, and entertainment. Tiane Lee, UX Research Lead at Google outlines the challenges and considerations behind adapting research for varying levels of digital literacy, including practical ideas for planning and conducting remote research with NIU.

‍

"NIU’s are typically less digitally literate, they may show lower confidence in digital capability, and they may perceive lower value of the internet for things like chatting and entertainment.”

‍

‍

‍

‍

7. Conditions Design: weaving the invisible threads of service design, value orchestration, and culture building  

‍

Michael Tam introduces us to the niche field of conditions design and cites a purpose built high diving board on Wellington city’s busy waterfront in New Zealand as a good example of conditions design.  Find out why in this fascinating talk.

‍

"What really impressed me here…hats off to the council because they didn’t design an experience that would discourage people from doing it. It’s designed for people to have fun (vs Hong Kong where public spaces are designed for Tai Chi not fun like this). The design allows it to happen by influencing human behavior to stay safe but encouraging fun and exploration.”

‍

‍

‍

‍

For a taste of what even more speakers from UX New Zealand 2022 had to share, head over to our highlights reel

Knowing and understanding why and how your users use your product can be invaluable for getting to the nitty gritty of usability. Where they get stuck and where they fly through. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative.

‍

Usability testing can be done in several ways, each way has its benefits. Put super simply, usability testing literally is testing how useable your product is for your users. If your product isn't useable users will not stick around or very often complete their task, let alone come back for more.

‍

‍

What is usability testing?

‍

Usability testing is a research method used to evaluate how easy something is to use by testing it with representative users.

‍

These tests typically involve observing a participant as they work through a series of tasks involving the product being tested. Having conducted several usability tests, you can analyze your observations to identify the most common issues.

‍

We go into the three main methods of usability testing:

1. Moderated and unmoderated
2. Remote or in person
3. Explorative, assessment or comparative

‍

‍

1. Moderated or unmoderated usability testing

‍

Moderated usability testing

Moderated usability testing is done in-person or remotely by a researcher who introduces the test to participants, answers their queries, and asks follow-up questions. Often these tests are done in real time with participants and can involve other research stakeholders. Moderated testing usually produces more in-depth results thanks to the direct interaction between researchers and test participants. However, this can be expensive to organize and run.

‍

Top tip: Use moderated testing to investigate the reasoning behind user behavior.

‍

Unmoderated usability testing

‍
Unmoderated usability testing is done without direct supervision; likely participants are in their own homes and/or using their own devices to browse the website that is being tested. And often at their own pace.  The cost of unmoderated testing is lower, though participant answers can remain superficial and making follow-up questions can be difficult.

‍

Top tip: Use unmoderated testing to test a very specific question or observe and measure behavior patterns.

‍

‍

2. Research or in-person usability testing

‍

Remote usability testing

‍
‍Remote usability testing is done over the internet or by phone. Allowing the participants to have the time and space to work in their own environment and at their own pace. This however doesn’t give the researcher much in the way of contextual data because you’re unable to ask questions around intention or probe deeper if the participant makes a particular decision. Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources.

‍

Top tip: Use remote testing when a large group of participants are needed and the questions asked can be direct and unambiguous.

‍

In-person usability testing

‍
‍In-person usability testing, as the name suggests, is done in the presence of a researcher. In-person testing does provide contextual data as researchers can observe and analyze body language and facial expressions. You’re also often able to converse with participants and find out more about why they do something. However, in-person testing can be expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and often pay) participants.

‍

Top tip: In-person testing gives researchers more time and insight into motivation for decisions.

‍

‍

3. Explorative, Assessment or comparative testing

‍

These three usability testing methods generate different types of information:

‍

Explorative testing

‍
‍Explorative testing is open-ended. Participants are asked to brainstorm, give opinions, and express emotional impressions about ideas and concepts. The information is typically collected in the early stages of product development and helps researchers pinpoint gaps in the market, identify potential new features, and workshop new ideas.

‍

Assessment research

‍
‍Assessment research is used to test a user's satisfaction with a product and how well they are able to use it. It's used to evaluate general functionality.

‍

Comparative research

‍
‍Comparative research methods involve asking users to choose which of two solutions they prefer, and they may be used to compare a product with its competitors.

‍

Top tip: Depending on what research is being done, and how much qualitative or quantitative data is wanted.

‍

‍

Which method is right for you?

‍

Whether the testing is done in-person, remote, moderated or unmoderated will depend on your purpose, what you want out of the testing, and to some extent your budget. 

‍

Depending on what you are testing, each of the usability testing methods we explored here can offer an answer. If you are at the development stage of a product it can be useful to conduct a usability test on the entire product. Checking the intuitive usability of your website, to ensure users can make the best decisions, quickly. Or adding, changing or upgrading a product can also be the moment to check on a specific question around usability. Planning and understanding your objectives are key to selecting the right usability testing option for your project.

‍

Let's take a look at a couple of examples of usability testing.

‍

‍

1. Lab based, in-person moderated testing - mid-life website

‍

Imagine you have a website that sells sports equipment. Over time your site has become cluttered and disorganized, much like a bricks and mortar store may. You’ve noticed a drop in sales in certain areas. How do you find out what is going wrong or where users are getting lost? Having an in-person, lab (or other controlled environment), moderated usability test with users you can set tasks, watch (and record) what they do.

‍

The researcher can literally be standing or sitting next to the participant throughout, recording contextual information such as how they interacted with the mouse, laptop or even the seat. Watching for cues as to the comfort of the participant and asking questions about why they make decisions can provide richer insights. Maybe they wanted purple yoga pants, but couldn’t find the ‘yoga’ section which was listed under gym rather than a clothing section.

‍

Meaning you can look at how your stock is organised, or even investigate undertaking a card sort. This provides robust and fully rounded feedback on users behaviours, expectations and experiences. Providing data that can directly be turned into actionable directives when redeveloping the website. 

‍

‍

2. Remote, moderated assessment testing - app product development

‍

You are looking at launching an app for parents to access for information and updates for the school. It’s still in development stage and at this point you want to know how easy the app is to use. Setting some very specific set tasks for participants to complete the app can be sent to them and they can be left to complete (or not). Providing feedback and comments around the usability.

‍

The next step may be to use first click testing to see how and where the interface is clicked and where participants may be spending time, or becoming lost. Whilst the feedback and data gathered from this testing can be light, it will be very direct to the questions asked. And will provide data to back up (or possibly not) what assumptions were made.

‍

‍

3. Moderated, In-person, explorative testing - new product development

‍

You’re right at the start of the development process. The idea is new and fresh and the basics are being considered. What better way to get an understanding of what your users’ truly want than an explorative study.

‍

Open-ended questions with participants in a one-on-one environment (or possibly in groups) can provide rich data and insights for the development team. Imagine you have an exciting new promotional app that you are developing for a client. There are similar apps on the market but none as exciting as what your team has dreamt up. By putting it (and possibly the competitors) to participants they can give direct feedback on what they like, love and loathe.

‍

They can also help brainstorm ideas or better ways to make the app work, or improve the interface. All of this done, before there is money sunk in development.

‍

‍

Usability testing summary: When to use each method (and why)

‍

Key objectives will dictate which usability testing method will deliver the answers to your questions.

‍

Whether it’s in-person, remote, moderated or comparative with a bit of planning you can gather data around your users very real experience of your product. Identify issues, successes and failures. Addressing your user experience with real data, and knowledge can but lead to a more intuitive product.

If you missed our recent live training on Prototype Testing, don’t worry—we’ve got everything you need right here! You can catch up at your convenience, so grab a cup of tea, put your feet up, and enjoy the show.
‍

In the session, we explored the powerful new features of our Prototype Testing tool, offering a step-by-step guide to setting up, running, and analyzing your tests like a seasoned pro. This tool is a game-changer for your design workflow, helping you identify usability issues and gather real user feedback before committing significant resources to development.

Here’s a quick recap of the highlights:

‍

1. Creating a prototype test from scratch using images

‍

We walked through how to create a prototype test from scratch using static images. This method is perfect for early-stage design concepts, where you want to quickly test user flows without a fully interactive prototype.

‍

2. Preparing your Figma prototype for testing

‍

Figma users, we’ve got you covered! We discussed how to prepare your Figma prototype for the smoothest possible testing experience. From setting up interactions to ensuring proper navigation, these tips ensure participants have an intuitive experience during the test. For more detailed instructions, check out our help article 

‍

3. Seamless Figma prototype imports

‍

One of the standout features of the tool is its seamless integration with Figma. We showed how easy it is to import your designs directly from Figma into Optimal, streamlining the setup process. You can bring your working files straight in, and resync when you need to with one click of a button.

‍

4. Understanding usability metrics and analyzing results

‍

We explored how to analyze the usability metrics, and walked through what the results can indicate on click maps and paths. These visual tools allow you to see exactly how participants navigate your design, making it easier to spot pain points, dead ends, or areas of friction. By understanding user behavior, you can rapidly iterate and refine your prototypes for optimal user experience.

‍