How many participants do I need for qualitative research?

Usability experts play an essential role in the user interface design process by evaluating the usability of digital products from a very important perspective - the users! Usability experts utilize various techniques such as heuristic evaluation, usability testing, and user research to gather data on how users interact with digital products and services. This data helps to identify design flaws and areas for improvement, leading to the development of user-friendly and efficient products.

‍

Heuristic evaluation is a usability research technique used to evaluate the user interface design of a digital product based on a set of ‘heuristics’ or ‘usability principles’. These heuristics are derived from a set of established principles of user experience design - attributed to the landmark article “Improving a Human-Computer Dialogue” published by web usability pioneers Jakob Nielsen and Rolf Molich in 1990. The principles focus on the experiential aspects of a user interface. 

‍

In this article, we’ll discuss what heuristic evaluation is and how usability experts use the principles to create exceptional design. We’ll also discuss how usability testing works hand-in-hand with heuristic evaluation, and how minimalist design and user control impact user experience. So, let’s dive in!

‍

‍

Understanding Heuristic Evaluation

Heuristic evaluation helps usability experts to examine interface design against tried and tested rules of thumb. To conduct a heuristic evaluation, usability experts typically work through the interface of the digital product and identify any issues or areas for improvement based on these broad rules of thumb, of which there are ten. They broadly cover the key areas of design that impact user experience - not bad for an article published over 30 years ago!

‍

The ten principles are:

‍

1. Prevention error: Well-functioning error messages are good, but instead of messages, can these problems be removed in the first place? Remove the opportunity for slips and mistakes to occur.
2. Consistency and standards: Language, terms, and actions used should be consistent to not cause any confusion.
3. Control and freedom for users: Give your users the freedom and control to undo/redo actions and exit out of situations if needed.
4. System status visibility: Let your users know what’s going on with the site. Is the page they’re on currently loading, or has it finished loading?
5. Design and aesthetics: Cut out unnecessary information and clutter to enhance visibility. Keep things in a minimalist style.
6. Help and documentation: Ensure that information is easy to find for users, isn’t too large and is focused on your users’ tasks.
7. Recognition, not recall: Make sure that your users don’t have to rely on their memories. Instead, make options, actions and objects visible. Provide instructions for use too.
8. Provide a match between the system and the real world: Does the system speak the same language and use the same terms as your users? If you use a lot of jargon, make sure that all users can understand by providing an explanation or using other terms that are familiar to them. Also ensure that all your information appears in a logical and natural order.
9. Flexibility: Is your interface easy to use and it is flexible for users? Ensure your system can cater to users to all types, from experts to novices.
10. Help users to recognize, diagnose and recover from errors: Your users should not feel frustrated by any error messages they see. Instead, express errors in plain, jargon-free language they can understand. Make sure the problem is clearly stated and offer a solution for how to fix it.

‍

Heuristic evaluation is a cost-effective way to identify usability issues early in the design process (although they can be performed at any stage) leading to faster and more efficient design iterations. It also provides a structured approach to evaluating user interfaces, making it easier to identify usability issues. By providing valuable feedback on overall usability, heuristic evaluation helps to improve user satisfaction and retention.

‍

‍

‍

The Role of Usability Experts in Heuristic Evaluation

‍

‍

Usability experts play a central role in the heuristic evaluation process by providing feedback on the usability of a digital product, identifying any issues or areas for improvement, and suggesting changes to optimize user experience.

‍

One of the primary goals of usability experts during the heuristic evaluation process is to identify and prevent errors in user interface design. They achieve this by applying the principles of error prevention, such as providing clear instructions and warnings, minimizing the cognitive load on users, and reducing the chances of making errors in the first place. For example, they may suggest adding confirmation dialogs for critical actions, ensuring that error messages are clear and concise, and making the navigation intuitive and straightforward.

‍

Usability experts also use user testing to inform their heuristic evaluation. User testing involves gathering data from users interacting with the product or service and observing their behavior and feedback. This data helps to validate the design decisions made during the heuristic evaluation and identify additional usability issues that may have been missed. For example, usability experts may conduct A/B testing to compare the effectiveness of different design variations, gather feedback from user surveys, and conduct user interviews to gain insights into users' needs and preferences.

‍

Conducting user testing with users that represent, as closely as possible, actual end users, ensures that the product is optimized for its target audience. Check out our tool Reframer, which helps usability experts collaborate and record research observations in one central database.

‍

‍

‍

Minimalist Design and User Control in Heuristic Evaluation

‍

‍

Minimalist design and user control are two key principles that usability experts focus on during the heuristic evaluation process. A minimalist design is one that is clean, simple, and focuses on the essentials, while user control refers to the extent to which users can control their interactions with the product or service.

‍

Minimalist design is important because it allows users to focus on the content and tasks at hand without being distracted by unnecessary elements or clutter. Usability experts evaluate the level of minimalist design in a user interface by assessing the visual hierarchy, the use of white space, the clarity of the content, and the consistency of the design elements. Information architecture (the system and structure you use to organize and label content) has a massive impact here, along with the content itself being concise and meaningful.

‍

Incorporating minimalist design principles into heuristic evaluation can improve the overall user experience by simplifying the design, reducing cognitive load, and making it easier for users to find what they need. Usability experts may incorporate minimalist design by simplifying the navigation and site structure, reducing the number of design elements, and removing any unnecessary content (check out our tool Treejack to conduct site structure, navigation, and categorization research). Consistent color schemes and typography can also help to create a cohesive and unified design.

‍

User control is also critical in a user interface design because it gives users the power to decide how they interact with the product or service. Usability experts evaluate the level of user control by looking at the design of the navigation, the placement of buttons and prompts, the feedback given to users, and the ability to undo actions. Again, usability testing plays an important role in heuristic evaluation by allowing researchers to see how users respond to the level of control provided, and gather feedback on any potential hiccups or roadblocks.

‍

‍

‍

Usability Testing and Heuristic Evaluation

‍

‍

Usability testing and heuristic evaluation are both important components of the user-centered design process, and they complement each other in different ways.

‍

Usability testing involves gathering feedback from users as they interact with a digital product. This feedback can provide valuable insights into how users perceive and use the user interface design, identify any usability issues, and help validate design decisions. Usability testing can be conducted in different forms, such as moderated or unmoderated, remote or in-person, and task-based or exploratory. Check out our usability testing 101 article to learn more.

‍

On the other hand, heuristic evaluation is a method in which usability experts evaluate a product against a set of usability principles. While heuristic evaluation is a useful method to quickly identify usability issues and areas for improvement, it does not involve direct feedback from users.

‍

Usability testing can be used to validate heuristic evaluation findings by providing evidence of how users interact with the product or service. For example, if a usability expert identifies a potential usability issue related to the navigation of a website during heuristic evaluation, usability testing can be used to see if users actually have difficulty finding what they need on the website. In this way, usability testing provides a reality check to the heuristic evaluation and helps ensure that the findings are grounded in actual user behavior.

‍

Usability testing and heuristic evaluation work together in the design process by informing and validating each other. For example, a designer may conduct heuristic evaluation to identify potential usability issues and then use the insights gained to design a new iteration of the product or service. The designer can then use usability testing to validate that the new design has successfully addressed the identified usability issues and improved the user experience. This iterative process of designing, testing, and refining based on feedback from both heuristic evaluation and usability testing leads to a user-centered design that is more likely to meet user needs and expectations.

‍

‍

‍

Conclusion

‍

‍

Heuristic evaluation is a powerful usability research technique that usability experts use to evaluate digital product interfaces based on a set of established principles of user experience design. After all these years, the ten principles of heuristic evaluation still cover the key areas of design that impact user experience, making it easier to identify usability issues early in the design process, leading to faster and more efficient design iterations. Usability experts play a critical role in the heuristic evaluation process by identifying design flaws and areas for improvement, using user testing to validate design decisions, and ensuring that the product is optimized for its intended users.

‍

Minimalist design and user control are two key principles that usability experts focus on during the heuristic evaluation process. A minimalist design is clean, simple, and focuses on the essentials, while user control gives users the freedom and control to undo/redo actions and exit out of situations if needed. By following these principles, usability experts can create an exceptional design that enhances visibility, reduces cognitive load, and provides a positive user experience. 

‍

Ultimately, heuristic evaluation is a cost-effective way to identify usability issues at any point in the design process, leading to faster and more efficient design iterations, and improving user satisfaction and retention. How many of the ten heuristic design principles does your digital product satisfy? 

Return on investment (ROI) is often the term on everyone’s lips when starting a big project or even when reviewing a website. It’s especially popular with those that hold the purse strings.  As UX researchers it is important to consider the ROI of the work we do and understand how to measure this. 

‍

We’ve lined up 5 key ways to measure ROI for UX research to help you get the conversation underway with stakeholders so you can show real and tangible benefits to your organization. 

‍

‍

1. Meet and exceed user expectations

‍

Put simply, a product that meets and exceeds user expectations leads to increased revenue. When potential buyers are able to find and purchase what they’re looking for, easily, they’ll complete their purchase, and are far more likely to come back. The simple fact that users can finish their task will increase sales and improve overall customer satisfaction which has an influence on their loyalty. Repeat business means repeat sales. Means increased revenue.

‍

Creating, developing and maintaining a usable website is more important than you might think. And this is measurable! Tracking and analyzing website performance prior to the UX research and after can be insightful and directly influenced by changes made based on UX research.

‍

Measurable: review the website (product) performance prior to UX research and after changes have been made. The increase in clicks, completed tasks and/or baskets will tell the story.

‍

‍

2. Reduce development time

‍

UX research done at the initial stages of a project can lead to a reduction in development time of by 33% to 50%! And reduced time developing, means reduced costs (people and overheads) and a speedier to market date. What’s not to love? 

‍

Measurable: This one is a little more tricky as you have saved time (and cost) up front. Aiding in speed to market and performance prior to execution. Internal stakeholder research may be of value post the live date to understand how the project went.

‍

‍

3. Ongoing development costs

‍

And the double hitter? Creating a product that has the user in mind up front, reduces the need to rehash or revisit as quickly. Reducing ongoing costs. Early UX research can help with the detection of errors early on in the development process. Fixing errors after development costs a company up to 100 times more than dealing with the same error before development.

‍

Measureable: Again, as UX research has saved time and money up front this one can be difficult to track. Though depending on your organization and previous projects you could conduct internal research to understand how the project compares and the time and cost savings.

‍

‍

4. Meeting user requirements

‍

Did you know that 70% of projects fail due to the lack of user acceptance? This is often because project managers fail to understand the user requirements properly. Thanks to UX research early on, gaining insights into users and only spending time developing the functions users actually want, saving time and reducing development costs. Make sure you get confirmation on those requirements by iterative testing. As always, fail early, fail often. Robust testing up front means that in the end, you’ll have a product that will meet the needs of the user.

‍

Measurable: Where is the product currently? How does it perform? Set a benchmark up front and review post UX research. The deliverables should make the ROI obvious.

‍

‍

5. Investing in UX research leads to an essential competitive advantage.

‍

Thanks to UX research you can find out exactly what your customers want, need and expect from you. This gives you a competitive advantage over other companies in your market. But you should be aware that more and more companies are investing in UX while customers are ever more demanding, their expectations continue to grow and they don’t tolerate bad experiences. And going elsewhere is an easy decision to make.

‍

Measurable: Murky this one, but no less important. Knowing, understanding and responding to competitors can help keep you in the lead, and developing products that meet and exceed those user expectations.

‍

‍

Wrap up

‍

Showing the ROI on the work we do is an essential part of getting key stakeholders on board with our research. It can be challenging to talk the same language, ultimately we all want the same outcome…a product that works well for our users, and delivers additional revenue.

‍

For some continued reading (or watching in this case), Anna Bek, Product and Delivery Manager at Xplor explored the same concept of "How to measure experience" during her UX New Zealand 2020 – watch it here as she shares a perspective on UX ROI.

Lately, I’ve found myself sitting in the participant’s chair at the UX research table more and more and it’s been an eye-opening collection of experiences. As UX researchers, we’re definitely not our users, but we are someone else’s. We use products, services and tools too! I’ve recently discovered that participating in user research not only helps out a fellow UXer with their qualitative research, but has also helped me grow my skills.

‍

Here are 3 reasons why you should try being a participant for yourself.

‍

1. Build empathy for the participant experience 🫶

‍

Having facilitated hundreds of UX research sessions over the years, I didn’t think I would be as nervous as I was going in as a participant! When we design our research sessions we don’t often give our participants a heads up on what we’re going to talk about because we don’t want to influence them for one, but truly not knowing what to expect made me feel slightly jittery. Before and at the beginning of these experiences, I felt quite unprepared. I wasn’t expecting to feel that way.

‍

During these sessions I often found myself wondering things like: Were my answers detailed enough? Will my responses be kept confidential? (I could never seem to remember if that had been covered in the introduction!). I worried that I wasn’t giving helpful responses or if I was talking too much. When misunderstandings arose in the conversation, I felt responsible and I was kicking myself for not communicating clearly enough!

‍

As a participant, I completely lost track of time during the session and developed an entirely new appreciation for timekeeping as a facilitator! I could have talked well beyond the scope of the interview timeframe and needed to be kept on track.

‍

I also very quickly discovered that thinking aloud is a lot harder to do than it sounds. It’s not a simple matter of verbalizing your thoughts as they happen. You have to think them, process them, decide how to verbalize them and then talk – all while someone (often a complete stranger) is looking at you expectantly and eagerly. It’s awkward and it feels weird. And look, it is widely acknowledged that it’s challenging and unnatural, but I didn’t fully appreciate or understand that until I was required to do it myself in a research environment.

‍

Did I experience some of these thoughts and feelings because I’m a UX researcher and I know what the other side looks and feels like? Maybe. It’s certainly possible and I’d be remiss if I didn’t call out this potential bias, but I do feel that having done this I now have a greater capacity to empathize with the participant experience. I’ve now been on the receiving end of the introductory spiel (and tuned out during it!), I’ve now got first-hand experience with the on-the-spot answering of those questions we carefully craft and I’ve scrambled for words when I’ve been asked to explain why I said what I said.

‍

These experiences have taught me that there’s a good chance the participant is just as nervous as I am and that a little reassurance or confirmation of usefulness goes a very long way. I’ve learned that regardless of the skill or experience of the researcher, interview questions can be super confusing and hard to answer. Having questions rephrased and being told that what I shared was helpful made me feel significantly more comfortable and able to think clearly and open up more - therefore providing more value to the researcher.

‍

I’ve also been itching to find out what happened next after the sessions ended. Was I helpful and in what way? Where did the insight I provided lead? I’ve learned that while we walk away with more questions, so might our participants and what can we do about that? We’re so busy, but stopping for a moment further down the track if/where possible to reach out and say ‘Hey, I just thought you might like to see what we did with all that great information you shared with us’ might be nice. It’s not always practical or possible and sometimes we have to wait until the thing goes live, but it’s food for thought.

‍

2. Learn from other researcher’s styles 📚

‍

Every UX researcher is different and we all have our own approaches and styles. As a notetaker for other researchers, I’ve always enjoyed having the opportunity to observe them in action, but somehow experiencing it as a participant felt different in the best possible way. It felt more immersive. Perhaps it was because I didn’t have to think about anything else or observe the participant’s reactions and was able to absorb every second of that experience as it was happening to me rather than in front of me.

‍

I participated in a usability testing session with two researchers - one facilitating and one taking notes - and the notetaker was so unobtrusive and quiet that I completely forgot they were there! They said a quick hello at the beginning of the session and then sat back behind me and blended in with the furniture and didn’t make a sound until they piped up with a question at the very end. Note taking is such a grossly underrated UX research skill. There’s a lot to think about, a lot to avoid thinking about (e.g., jumping into solution right there in the session) and of course we have to be mindful of the potential impact of our behavior on the participant, but this went beyond stifling disappointment or resisting the urge to speak. This was a dignified and seamlessly elegant note taker existence unlike anything I have ever seen.

‍

In other sessions as a user interview participant I was delighted when researchers injected multiple moments of humor into those previously mentioned introductory spiel snooze fests. It did more than just make me feel comfortable - humor helped me focus better on what was being said and remember important details during and well after the session had ended.

‍

I also learned a thing or two about comfortable silences when I participated in a contextual session with two researchers. One researcher kept prodding and repeatedly asking questions while the other exuded a quiet and calm demeanor and simply patiently waited for me to complete my task and talk about it in my own time (and way). It won’t work in all situations or with all participants, but it made me feel relaxed and comfortable enough to talk through what I was thinking and seeing. This approach also made thinking aloud easier for me.

‍

It’s important to remember that it doesn’t matter if you’re an experienced UX researcher or if you’re just starting out, everyone can learn something new from another researcher and stepping up into a session as an actual participant is a great way to do that.

‍

3. Give back and grow our industry 🌱

‍

One of my favourite things about the UX industry has always been its sense of community. We’re a group of people who care. We care about our users, we care about each other and we care about our capability as an industry and where we’re headed. Agreeing to be a participant in another researcher’s study is a great way to give back.

‍

As I mentioned earlier, we use products and services too and in addition to this, there’s a whole heap of them out there designed just for us! It makes sense that in the same way that we as researchers ask our own customers and users to help us design better products, we should be open to doing the same for the people who design for us.

‍

The cool thing about being a participant who is also a researcher is that we pick up issues other people might not. We might be a little tougher and less likely to let a usability issue slide. We might be the person that provides the external and fresh-eyed validation a researcher needs to convince a stakeholder that a design needs to be changed or worked on some more. A researcher in the participant’s chair is a powerful hybrid - it’s a participant and expert reviewer in one.

‍

As a general rule of thumb, if you’re invited to participate in another researcher’s study, I do think it’s best to always be upfront with them about who you are and what you do so that they can determine if you are potentially too biased to be included. It’s their study and informed consent matters. Think about what you would want as a researcher. And if they’re specifically asking you to participate because you’re a researcher - that’s awesome!

‍

So those are just some of the reasons why you should take a spin in the participant’s chair. Professional development is a lifelong learning process for us all. I’m looking forward to implementing what I’ve learned from these experiences and continuing to plonk myself in that participant chair to keep growing my perspective and helping out other researchers along the way.

‍

Happy researching (and participating)!