July 12, 2023
3 min

5 key areas for effective ResearchOPs

Simply put, ResearchOps is about making sure your research operations are robust, thought through and managed. 

Having systems and processes around your UX research and your team keep everyone (and everything) organized. Making user research projects quicker to get started and more streamlined to run. And robust sharing, socializing, and knowledge storage means that everyone can understand the research insights and findings and put these to use - across the organization. And even better, find these when they need them. 

Using the same tools across the team allows the research team to learn from each other, and previous research projects and be able to compare apples with apples, with everyone included. Bringing the team together across tools, research and results.

We go into more detail in our ebook ResearchOps Checklist about exactly what you can do to make sure your research team is running at its best. Let’s take a quick look at 5 way to ensure you have the grounding for a successful ResearchOps team.

1. Knowledge management 📚

What do you do with all of the insights and findings of a user research project? How do you store them, how do you manage the insights, and how do you share and socialize?

Having processes in place that manage this knowledge is important to the longevity of your research. From filing to sharing across platforms, it all needs to be standardized so everyone can search, find and share.

2. Guidelines and process templates 📝

Providing a framework for how to run research projects is are important. Building on the knowledge base from previous research can improve research efficiencies and cut down on groundwork and administration. Making research projects quicker and more streamlined to get underway.

3. Governance 🏛

User research is all about people, real people. It is incredibly important that any research be legal, safe, and ethical. Having effective governance covered is vital.

4. Tool stack 🛠

Every research team needs a ‘toolbox’ that they can use whenever they need to run card sorts, tree tests, usability tests, user interviews, and more. But which software and tools to use?

Making sure that the team is using the same tools also helps with future research projects, learning from previous projects, and ensuring that the information is owned and run by the organization (rather than whichever individuals prefer). Reduce logins and password shares, and improve security with organization-wide tools and platforms. 

5. Recruitment 👱🏻👩👩🏻👧🏽👧🏾

Key to great UX research is the ability to recruit quality participants - fast! Having strong processes in place for screening, scheduling, sampling, incentivizing, and managing participants needs to be top of the list when organizing the team.

Wrap Up 💥

Each of these ResearchOps processes are not independent of the other. And neither do they flow from one to the other. They are part of a total wrap around for the research team, creating processes, systems and tools that are built to serve the team. Allowing them to focus on the job of doing great research and generating insights and findings that develop the very best user experience. 

Afterall, we are creating user experiences that keep our users engaged and coming back. Why not look at the teams user experience and make the most of that. Freeing time and space to socialize and share the findings with the organization. 

Share this article
Author
Optimal
Workshop

Related articles

View all blog articles
Learn more
1 min read

Radical Collaboration: how teamwork really can make the dream work

Natalie and Lulu have forged a unique team culture that focuses on positive outputs (and outcomes) for their app’s growing user base. In doing so, they turned the traditional design approach on its head and created a dynamic and supportive team. 

Natalie, Director of Design at Hatch, and Lulu, UX Design Specialist, recently spoke at UX New Zealand, the leading UX and IA conference in New Zealand hosted by Optimal Workshop, on their concept of “radical collaboration”.

In their talk, Nat and Lulu share their experience of growing a small app into a big player in the finance sector, and their unique approach to teamwork and culture which helped achieve it.

Background on Natalie Ferguson and Lulu Pachuau

Over the last two decades, Lulu and Nat have delivered exceptional customer experiences for too many organizations to count. After Nat co-founded Hatch, she begged Lulu to join her on their audacious mission: To supercharge wealth building in NZ. Together, they created a design and product culture that inspired 180,000 Kiwi investors to join in just 4 years.

Contact Details:

Email: natalie@sixfold.co.nz

LinkedIn: https://www.linkedin.com/in/natalieferguson/ and https://www.linkedin.com/in/lulupach/

Radical Collaboration - How teamwork makes the dream work 💪💪💪

Nat and Lulu discuss how they nurtured a team culture of “radical collaboration” when growing the hugely popular app Hatch, based in New Zealand. Hatch allows everyday New Zealanders to quickly and easily trade in the U.S. share market. 

The beginning of the COVID pandemic spelled huge growth for Hatch and caused significant design challenges for the product. This growth meant that the app had to grow from a baby startup to one that could operate at scale - virtually overnight. 

In navigating this challenge, Nat and Lulu coined the term radical collaboration, which aims to “dismantle organizational walls and supercharge what teams achieve”. Radical collaboration has six key pillars, which they discuss alongside their experience at Hatch.

Pillar #1: When you live and breathe your North star

Listening to hundreds of their customers’ stories, combined with their own personal experiences with money, compelled Lulu and Nat to change how their users view money. And so, “Grow the wealth of New Zealanders” became a powerful mission statement, or North Star, for Hatch. The mission was to give people the confidence and the ability to live their own lives with financial freedom and control. Nat and Lulu express the importance of truly believing in the mission of your product, and how this can become a guiding light for any team. 

Pillar #2: When you trust each other so much, you’re happy to give up control

As Hatch grew rapidly, trusting each other became more and more important. Nat and Lulu state that sometimes you need to take a step back and stop fueling growth for growth’s sake. It was at this point that Nat asked Lulu to join the team, and Nat’s first request was for Lulu to be super critical about the product design to date - no feedback was out of bounds. Letting go, feeling uncomfortable, and trusting your team can be difficult, but sometimes it’s what you need in order to drag yourself out of status quo design. This resulted in a brief hiatus from frantic delivery to take stock and reprioritize what was important - something that can be difficult without heavy doses of trust!

Pillar #3: When everyone wears all the hats

During their journey, the team at Hatch heard lots of stories from their users. Many of these stories were heard during “Hatcheversery Calls”, where team members would call users on their sign-up anniversary to chat about their experience with the app. Some of these calls were inspiring, insightful, and heartwarming.

Everyone at Hatch made these calls – designers, writers, customer support, engineers, and even the CEO. Speaking to strangers in this way was a challenge for some, especially since it was common to field technical questions about the business. Nevertheless, asking staff to wear many hats like this turned the entire team into researchers and analysts. By forcing ourselves and our team outside of our comfort zone, we forced each other to see the whole picture of the business, not just our own little piece.

Pillar #4: When you do what’s right, not what’s glam

In an increasingly competitive industry, designers and developers are often tempted to consistently deliver new and exciting features. In response to rapid growth, rather than adding more features to the app, Lulu and Nat made a conscious effort to really listen to their customers to understand what problems they needed solving. 

As it turned out, filing overseas tax returns was a significant and common problem for their customers - it was difficult and expensive. So, the team at Hatch devised a tax solution. This solution was developed by the entire team, with almost no tax specialists involved until the very end! This process was far from glamorous and it often fell outside of standard job descriptions. However, the team eventually succeeded in simplifying a notoriously difficult process and saved their customers a massive headache.

Pillar #5: When you own the outcome, not your output.

Over time Hatch’s user base changed from being primarily confident, seasoned investors, to being first-time investors. This new user group was typically scared of investing and often felt that it was only a thing wealthy people did.

At this point, Hatch felt it was necessary to take a step back from delivering updates to take stock of their new position. This meant deeply understanding their customers’ journey from signing up, to making their first trade. Once this was intimately understood, the team delivered a comprehensive onboarding process which increased the sign-up conversion rate by 10%!

Pillar #6: When you’re relentlessly committed to making it work

Nat and Lulu describe a moment when Allbirds wanted to work with Hatch to allow ordinary New Zealanders to be involved in their IPO launch on the New York stock exchange. Again, this task faced numerous tax and trade law challenges, and offering the service seemed like yet another insurmountable task. The team at Hatch nearly gave up several times during this project, but everyone was determined to get this feature across the line – and they did. As a result, New Zealanders were some of the few regular investors from outside the U.S that were able to take part in Albirds IPO. 

Why it matters 💥

Over four years, Hatch grew to 180,000 users who collectively invested over $1bn. Nat and Lulu’s success underscores the critical role of teamwork and collaboration in achieving exceptional user experiences. Product teams should remember that in the rapidly evolving tech industry, it's not just about delivering the latest features; it's about fostering a positive and supportive team culture that buys into the bigger picture.

The Hatch team grew to be more than team members and technical experts. They grew in confidence and appreciated every moving part of the business. Product teams can draw inspiration from Hatch's journey, where designers, writers, engineers, and even the CEO actively engaged with users, challenged traditional design decisions, and prioritized solving actual user problems. This approach led to better, more user-centric outcomes and a deep understanding of the end-to-end user experience.

Most importantly, through the good times and tough, the team grew to trust each other. The mission weaved its way through each member of the team, which ultimately manifested in positive outcomes for the user and the business.

Nat and Lulu’s concept of radical collaboration led to several positive outcomes for Hatch:

  • It changed the way they did business. Information was no longer held in the minds of a few individuals – instead, it was shared. People were able to step into other people's roles seamlessly. 
  • Hatch achieved better results faster by focusing on the end-to-end experience of the app, rather than by adding successive features. 
  • The team became more nimble – potential design/development issues were anticipated earlier because everyone knew what the downstream impacts of a decision would be.

Over the next week, Lulu and Nat encourage designers and researchers to get outside of their comfort zone and:

  • Visit customer support team
  • Pick up the phone and call a customer
  • Challenge status quo design decisions. Ask, does this thing solve an end-user problem?

Learn more
1 min read

Best UX Research Methods for Every Phase of Product Development

What is UX research?

User experience (UX) research, or user research as it’s commonly referred to, is an important part of the product design process. Primarily, UX research involves using different research methods to gather information about how your users interact with your product. It is an essential part of developing, building and launching a product that truly meets the requirements of your users. 

UX research is essential at all stages of a products' life cycle:

  1. Planning
  2. Building
  3. Introduction
  4. Growth & Maturity

While there is no one single time to conduct UX research it is best-practice to continuously gather information throughout the lifetime of your product. The good news is many of the UX research methods do not fit just one phase either, and can (and should) be used repeatedly. After all, there are always new pieces of functionality to test and new insights to discover. We introduce you to best-practice UX research methods for each lifecycle phase of your product.

1. Product planning phase

While the planning phase it is about creating a product that fits your organization, your organization’s needs and meeting a gap in the market it’s also about meeting the needs, desires and requirements of your users. Through UX research you’ll learn which features are necessary to be aligned with your users. And of course, user research lets you test your UX design before you build, saving you time and money.

Qualitative Research Methods

Usability Testing - Observational

One of the best ways to learn about your users and how they interact with your product is to observe them in their own environment. Watch how they accomplish tasks, the order they do things, what frustrates them, and what makes the task easier and/or more enjoyable for your subject. The data can be collated to inform the usability of your product, improving intuitive design, and what resonates with users.

Competitive Analysis

Reviewing products already in the market can be a great start to the planning process. Why are your competitors’ products successful and how well do they behave for users. Learn from their successes, and even better build on where they may not be performing the best and find your niche in the market.

Quantitative Research Methods

Surveys and Questionnaires

Surveys are useful for collecting feedback or understanding attitudes. You can use the learnings from your survey of a subset of users to draw conclusions about a larger population of users.

There are two types of survey questions:

Closed questions are designed to capture quantitative information. Instead of asking users to write out answers, these questions often use multi-choice answers.

Open questions are designed to capture qualitative information such as motivations and context.  Typically, these questions require users to write out an answer in a text field.

2. Product building phase

Once you've completed your product planning research, you’re ready to begin the build phase for your product. User research studies undertaken during the build phase enable you to validate the UX team’s deliverables before investing in the technical development.

Qualitative Research Methods

Focus groups

Generally involve 5-10 participants and include demographically similar individuals. The study is set up so that members of the group can interact with one another and can be carried out in person or remotely.


Besides learning about the participants’ impressions and perceptions of your product, focus group findings also include what users believe to be a product’s most important features, problems they might encounter while using the product, as well as their experiences with other products, both good and bad.

Quantitative Research Methods

Card sorting gives insight into how users think. Tools like card sorting reveal where your users expect to find certain information or complete specific tasks. This is especially useful for products with complex or multiple navigations and contributes to the creation of an intuitive information architecture and user experience.

Tree testing gives insight into where users expect to find things and where they’re getting lost within your product. Tools like tree testing help you test your information architecture.
Card sorting and tree testing are often used together. Depending on the purpose of your research and where you are at with your product, they can provide a fully rounded view of your information architecture.

3. Product introduction phase

You’ve launched your product, wahoo! And you’re ready for your first real life, real time users. Now it’s time to optimize your product experience. To do this, you’ll need to understand how your new users actually use your product.

Qualitative Research Methods

Usability testing involves testing a product with users. Typically it involves observing users as they try to follow and complete a series of tasks. As a result you can evaluate if the design is intuitive and if there are any usability problems.

User Interviews - A user interview is designed to get a deeper understanding of a particular topic. Unlike a usability test, where you’re more likely to be focused on how people use your product, a user interview is a guided conversation aimed at better understanding your users. This means you’ll be capturing details like their background, pain points, goals and motivations.

Quantitative Research Methods

A/B Testing is a way to compare two versions of a design in order to work out which is more effective. It’s typically used to test two versions of the same webpage, for example, using a different headline, image or call to action to see which one converts more effectively. This method offers a way to validate smaller design choices where you might not have the data to make an informed decision, like the color of a button or the layout of a particular image.

Flick-click testing shows you where people click first when trying to complete a task on a website. In most cases, first-click testing is performed on a very simple wireframe of a website, but it can also be carried out on a live website using a tool like first-time clicking.

4. Growth and maturity phase

If you’ve reached the growth stage, fantastic news! You’ve built a great product that’s been embraced by your users. Next on your to-do list is growing your product by increasing your user base and then eventually reaching maturity and making a profit on your hard work.

Growing your product involves building new or advanced features to satisfy specific customer segments. As you plan and build these enhancements, go through the same research and testing process you used to create the first release. The same holds true for enhancements as well as a new product build — user research ensures you’re building the right thing in the best way for your customers.

Qualitative research methods

User interviews will focus on how your product is working or if it’s missing any features, enriching your knowledge about your product and users.

It allows you to test your current features, discover new possibilities for additional features and think about discarding  existing ones. If your customers aren’t using certain features, it might be time to stop supporting them to reduce costs and help you grow your profits during the maturity stage.

Quantitative research methods

Surveys and questionnaires can help gather information around which features will work best for your product, enhancing and improving the user experience. 

A/B testing during growth and maturity occurs within your sales and onboarding processes. Making sure you have a smooth onboarding process increases your conversion rate and reduces wasted spend — improving your bottom line.

Final Thoughts: Why Continuous UX Research Matters

UX research testing throughout the lifecycle of your product helps you continuously evolve and develop a product that responds to what really matters - your users.

Talking to, testing, and knowing your users will allow you to push your product in ways that make sense with the data to back up decisions. Go forth and create the product that meets your organizations needs by delivering the very best user experience for your users.

Learn more
1 min read

Collating your user testing notes

It’s been a long day. Scratch that - it’s been a long week! Admit it. You loved every second of it.

Twelve hour days, the mad scramble to get the prototype ready in time, the stakeholders poking their heads in occasionally, dealing with no-show participants and the excitement around the opportunity to speak to real life human beings about product or service XYZ. Your mind is exhausted but you are buzzing with ideas and processing what you just saw. You find yourself sitting in your war room with several pages of handwritten notes and with your fellow observers you start popping open individually wrapped lollies leftover from the day’s sessions. Someone starts a conversation around what their favourite flavour is and then the real fun begins. Sound familiar? Welcome to the post user testing debrief meeting.

How do you turn those scribbled notes and everything rushing through your mind into a meaningful picture of the user experience you just witnessed? And then when you have that picture, what do you do next? Pull up a bean bag, grab another handful of those lollies we feed our participants and get comfy because I’m going to share my idiot-proof, step by step guide for turning your user testing notes into something useful.

Let’s talk

Get the ball rolling by holding a post session debrief meeting while it’s all still fresh your collective minds. This can be done as one meeting at the end of the day’s testing or you could have multiple quick debriefs in between testing sessions. Choose whichever options works best for you but keep in mind this needs to be done at least once and before everyone goes home and forgets everything. Get all observers and facilitators together in any meeting space that has a wall like surface that you can stick post its to - you can even use a window! And make sure you use real post its - the fake ones fall off!

Mark your findings (Tagging)

Before you put sharpie to post it, it’s essential to agree as a group on how you will tag your observations. Tagging the observations now will make the analysis work much easier and help you to spot patterns and themes. Colour coding the post its is by far the simplest and most effective option and how you assign the colours is entirely up to you. You could have a different colour for each participant or testing session, you could have different colours to denote participant attributes that are relevant to your study eg senior staff and junior staff, or you could use different colours to denote specific testing scenarios that were used. There’s many ways you could carve this up and there’s no right or wrong way. Just choose the option that suits you and your team best because you’re the ones who have to look at it and understand it. If you only have one colour post it eg yellow, you could colour code the pen colours you use to write on the notes or include some kind of symbol to help you track them.

Processing the paper (Collating)

That pile of paper is not going to process itself! Your next job as a group is to work through the task of transposing your observations to post it notes. For now, just stick them to the wall in any old way that suits you. If you’re the organising type, you could group them by screen or testing scenario. The positioning will all change further down the process, so at this stage it’s important to just keep it simple. For issues that occur repeatedly across sessions, just write them down on their own post its- doubles will be useful to see further down the track.In addition to  holding a debrief meetings, you also need to round up everything that was used to capture the testing session/s. And I mean EVERYTHING.

Handwritten notes, typed notes, video footage and any audio recordings need to be reviewed just in case something was missed. Any handwritten notes should be typed to assist you with the completion of the report. Don’t feel that you have to wait until the testing is completed before you start typing up your notes because you will find they pile up very quickly and if your handwriting is anything like mine…. Well let’s just say my short term memory is often required to pick up the slack and even that has it’s limits. Type them up in between sessions where possible and save each session as it’s own document. I’ll often use the testing questions or scenario based tasks to structure my typed notes and I find that makes it really easy to refer back to.Now that you’ve processed all the observations, it’s time to start sorting your observations to surface behavioural patterns and make sense of it all.

Spotting patterns and themes through affinity diagramming

Affinity diagramming is a fantastic tool for making sense of user testing observations. In fact it’s just about my favourite way to make sense of any large mass of information. It’s an engaging and visual process that grows and evolves like a living creature taking on a life of its own. It also builds on the work you’ve just done which is a real plus!By now, testing is over and all of your observations should all be stuck to a wall somewhere. Get everyone together again as a group and step back and take it all in. Just let it sit with you for a moment before you dive in. Just let it breathe. Have you done that? Ok now as individuals working at the same time, start by grouping things that you think belong together. It’s important to just focus on the content of the labels and try to ignore the colour coded tagging at this stage, so if session one was blue post its don’t group all the blue ones together just because they’re all blue! If you get stuck, try grouping by topic or create two groups eg issues and wins and then chunk the information up from there.

You will find that the groups will change several times over the course of the process  and that’s ok because that’s what it needs to do.While you do this, everyone else will be doing the same thing - grouping things that make sense to them.  Trust me, it’s nowhere near as chaotic as it sounds! You may start working as individuals but it won’t be long before curiosity kicks in and the room is buzzing with naturally occurring conversation.Make sure you take a step back regularly and observe what everyone else is doing and don’t be afraid to ask questions and move other people’s post its around- no one owns it! No matter how silly something may seem just put it there because it can be moved again. Have a look at where your tagged observations have ended up. Are there clusters of colour? Or is it more spread out? What that means will depend largely on how you decided to tag your findings. For example if you assigned each testing session its own colour and you have groups with lot’s of different colours in them you’ll find that the same issue was experienced by multiple people.Next, start looking at each group and see if you can break them down into smaller groups and at the same time consider the overall picture for bigger groups eg can the wall be split into say three high level groups.Remember, you can still change your groups at anytime.

Thinning the herd (Merging)

Once you and your team are happy with the groups, it’s time to start condensing the size of this beast. Look for doubled up findings and stack those post its on top of each other to cut the groups down- just make sure you can still see how many there were. The point of merging is to condense without losing anything so don’t remove something just because it only happened once. That one issue could be incredibly serious. Continue to evaluate and discuss as a group until you are happy. By now clear and distinct groups of your observations should have emerged and at a glance you should be able to identify the key findings from your study.

A catastrophe or a cosmetic flaw? (Scoring)

Scoring relates to how serious the issues are and how bad the consequences of not fixing them are. There are arguments for and against the use of scoring and it’s important to recognise that it is just one way to communicate your findings.I personally rarely use scoring systems. It’s not really something I think about when I’m analysing the observations. I rarely rank one problem or finding over another. Why? Because all data is good data and it all adds to the overall picture.I’ve always been a huge advocate for presenting the whole story and I will never diminish the significance of a finding by boosting another. That said, I do understand the perspective of those who place metrics around their findings. Other designers have told me they feel that it allows them to quantify the seriousness of each issue and help their client/designer/boss make decisions about what to do next.We’ve all got our own way of doing things, so I’ll leave it up to you to choose whether or not you score the issues. If you decide to score your findings there are a number of scoring systems you can use and if I had to choose one, I quite like Jakob Nielsen’s methodology for the simple way it takes into consideration multiple factors. Ultimately you should choose the one that suits your working style best.

Let’s say you did decide to score the issues. Start by writing down each key finding on it’s own post it and move to a clean wall/ window. Leave your affinity diagram where it is. Divide the new wall in half: one side for wins eg findings that indicate things that tested well and the other for issues. You don’t need to score the wins but you do need to acknowledge what went well because knowing what you’re doing well is just as important as knowing where you need to improve. As a group (wow you must be getting sick of each other! Make sure you go out for air from time to time!) score the issues based on your chosen methodology.Once you have completed this entire process you will have everything you need to write a kick ass report.

What could possibly go wrong? (and how to deal with it)

No process is perfect and there are a few potential dramas to be aware of:

People jumping into solution mode too early

In the middle of the debrief meeting, someone has an epiphany. Shouts of We should move the help button! or We should make the yellow button smaller! ring out and the meeting goes off the rails.I’m not going to point fingers and blame any particular role because we’ve all done it, but it’s important to recognise that’s not why we’re sitting here. The debrief meeting is about digesting and sharing what you and the other observers just saw. Observing and facilitating user testing is a privilege. It’s a precious thing that deserves respect and if you jump into solution mode too soon, you may miss something. Keep the conversation on track by appointing a team member to facilitate the debrief meeting.

Storage problems

Handwritten notes taken by multiple observers over several days of testing adds up to an enormous pile of paper. Not only is it a ridiculous waste of paper but they have to be securely stored for three months following the release of the report. It’s not pretty. Typing them up can solve that issue but it comes with it’s own set of storage related hurdles. Just like the handwritten notes, they need to be stored securely. They don’t belong on SharePoint or in the share drive or any other shared storage environment that can be accessed by people outside your observer group. User testing notes are confidential and are not light reading for anyone and everyone no matter how much they complain. Store any typed notes in a limited access storage solution that only the observers have access to and if anyone who shouldn’t be reading them asks, tell them that they are confidential and the integrity of the research must be preserved and respected.

Time issues

Before the storage dramas begin, you have to actually pick through the mountain of paper. Not to mention the video footage, and the audio and you have to chase up that sneaky observer who disappeared when the clock struck 5. All of this takes up a lot of time. Another time related issue comes in the form of too much time passing in between testing sessions and debrief meetings. The best way to deal with both of these issues  is to be super organised and hold multiple smaller debriefs in between sessions where possible. As a group, work out your time commitments before testing begins and have a clear plan in place for when you will meet.  This will prevent everything piling up and overwhelming you at the end.

Disagreements over scoring

At the end of that long day/week we’re all tired and discussions around scoring the issues can get a little heated. One person’s showstopper may be another person’s mild issue. Many of the ranking systems use words as well as numbers to measure the level of severity and it’s easy to get caught up in the meaning of the words and ultimately get sidetracked from the task at hand. Be proactive and as a group set ground rules upfront for all discussions. Determine how long you’ll spend discussing an issue and what you will do in the event that agreement cannot be reached. People want to feel heard and they want to feel like their contributions are valued. Given that we are talking about an iterative process, sometimes it’s best just to write everything down to keep people happy and merge and cull the list in the next iteration. By then they’ve likely had time to reevaluate their own thinking.

And finally...

We all have our own ways of making sense of our user testing observations and there really is no right or wrong way to go about it. The one thing I would like to reiterate is the importance of collaboration and teamwork. You cannot do this alone, so please don’t try. If you’re a UX team of one, you probably already have a trusted person that you bounce ideas off. They would be a fantastic person to do this with. How do you approach this process? What sort of challenges have you faced? Let me know in the comments below.

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.