October 31, 2024
10

Ready for take-off: Best practices for creating and launching remote user research studies

Optimal Workshop
"Hi Optimal Work,I was wondering if there are some best practices you stick to when creating or sending out different UX research studies (i.e. Card sorts, Prototyye Test studies, etc)? Thank you! Mary"

Indeed I do! Over the years I’ve learned a lot about creating remote research studies and engaging participants. That experience has taught me a lot about what works, what doesn’t and what leaves me refreshing my results screen eagerly anticipating participant responses and getting absolute zip. Here are my top tips for remote research study creation and launch success!

Creating remote research studies

Use screener questions and post-study questions wisely

Screener questions are really useful for eliminating participants who may not fit the criteria you’re looking for but you can’t exactly stop them from being less than truthful in their responses. Now, I’m not saying all participants lie on the screener so they can get to the activity (and potentially claim an incentive) but I am saying it’s something you can’t control. To help manage this, I like to use the post-study questions to provide additional context and structure to the research.

Depending on the study, I might ask questions to which the answers might confirm or exclude specific participants from a specific group. For example, if I’m doing research on people who live in a specific town or area, I’ll include a location based question after the study. Any participant who says they live somewhere else is getting excluded via that handy toggle option in the results section. Post-study questions are also great for capturing additional ideas and feedback after participants complete the activity as remote research limits your capacity to get those — you’re not there with them so you can’t just ask. Post-study questions can really help bridge this gap. Use no more than five post-study questions at a time and consider not making them compulsory.

Do a practice run

No matter how careful I am, I always miss something! A typo, a card with a label in the wrong case, forgetting to update a new version of an information architecture after a change was made — stupid mistakes that we all make. By launching a practice version of your study and sharing it with your team or client, you can stop those errors dead in their tracks. It’s also a great way to get feedback from the team on your work before the real deal goes live. If you find an error, all you have to do is duplicate the study, fix the error and then launch. Just keep an eye on the naming conventions used for your studies to prevent the practice version and the final version from getting mixed up!

Sending out remote research studies

Manage expectations about how long the study will be open for

Something that has come back to bite me more than once is failing to clearly explain when the study will close. Understandably, participants can be left feeling pretty annoyed when they mentally commit to complete a study only to find it’s no longer available. There does come a point when you need to shut the study down to accurately report on quantitative data and you’re not going to be able to prevent every instance of this, but providing that information upfront will go a long way.

Provide contact details and be open to questions

You may think you’re setting yourself up to be bombarded with emails, but I’ve found that isn’t necessarily the case. I’ve noticed I get around 1-3 participants contacting me per study. Sometimes they just want to tell me they completed it and potentially provide additional information and sometimes they have a question about the project itself. I’ve also found that sometimes they have something even more interesting to share such as the contact details of someone I may benefit from connecting with — or something else entirely! You never know what surprises they have up their sleeves and it’s important to be open to it. Providing an email address or social media contact details could open up a world of possibilities.

Don’t forget to include the link!

It might seem really obvious, but I can’t tell you how many emails I received (and have been guilty of sending out) that are missing the damn link to the study. It happens! You’re so focused on getting that delivery right and it becomes really easy to miss that final yet crucial piece of information.

To avoid this irritating mishap, I always complete a checklist before hitting send:

  • Have I checked my spelling and grammar?
  • Have I replaced all the template placeholder content with the correct information?
  • Have I mentioned when the study will close?
  • Have I included contact details?
  • Have I launched my study and received confirmation that it is live?
  • Have I included the link to the study in my communications to participants?
  • Does the link work? (yep, I’ve broken it before)

General tips for both creating and sending out remote research studies

Know your audience

First and foremost, before you create or disseminate a remote research study, you need to understand who it’s going to and how they best receive this type of content. Posting it out when none of your followers are in your user group may not be the best approach. Do a quick brainstorm about the best way to reach them. For example if your users are internal staff, there might be an internal communications channel such as an all-staff newsletter, intranet or social media site that you can share the link and approach content to.

Keep it brief

And by that I’m talking about both the engagement mechanism and the study itself. I learned this one the hard way. Time is everything and no matter your intentions, no one wants to spend more time than they have to. Even more so in situations where you’re unable to provide incentives (yep, I’ve been there). As a rule, I always stick to no more than 10 questions in a remote research study and for card sorts, I’ll never include more than 60 cards. Anything more than that will see a spike in abandonment rates and of course only serve to annoy and frustrate your participants. You need to ensure that you’re balancing your need to gain insights with their time constraints.

As for the accompanying approach content, short and snappy equals happy! In the case of an email, website, other social media post, newsletter, carrier pigeon etc, keep your approach spiel to no more than a paragraph. Use an audience appropriate tone and stick to the basics such as: a high level sentence on what you’re doing, roughly how long the study will take participants to complete, details of any incentives on offer and of course don’t forget to thank them.

Set clear instructions

The default instructions in Optimal Workshop’s suite of tools are really well designed and I’ve learned to borrow from them for my approach content when sending the link out. There’s no need for wheel reinvention and it usually just needs a slight tweak to suit the specific study. This also helps provide participants with a consistent experience and minimizes confusion allowing them to focus on sharing those valuable insights!

Create a template

When you’re on to something that works — turn it into a template! Every time I create a study or send one out, I save it for future use. It still needs minor tweaks each time, but I use them to iterate my template.What are your top tips for creating and sending out remote user research studies? Comment below!

Publishing date
October 31, 2024
Share this article

Related articles

min read
Product Roadmap Update 2024 🛣️ 🎉

We’re excited to announce the latest updates to our platform to help teams test, validate and gain insights early in the design process. Looking ahead to the rest of 2024, stay tuned as we expand Optimal’s research capabilities with new tools and features, including Prototype Testing, Qualitative Insights, and upgraded Surveys along with enhanced admin management and user roles!

What’s new at Optimal? 🤩

Managed Recruitment 🙋🙋🏾


We’re thrilled to share the launch of our new Managed Recruitment offering with high quality and expanded profiling capabilities for hard-to-reach audiences. Managed Recruitment leverages our in-house recruitment team and a new powerful, award-winning panel — PureSpectrum — to give you access to millions of participants across 100+ countries. Optimal handles recruitment logistics from start to finish, guaranteeing quality responses and replacing low-quality participants at no extra cost.

Early access is now available. To get started, submit your criteria via the request form. Starting in early August, you’ll be able to access Managed Recruitment via the Optimal Recruit tab. Recruitment is available for all Optimal tools except for Qualitative Insights (Reframer).

What’s coming next? 🔮🧙

Test your designs early and often with our latest tool:  Prototype Testing ⚙️

The newest edition to the Optimal toolkit is just around the corner.  Prototype testing will allow you to quickly  test designs with users throughout the design process, to help inform decisions so you can build on with confidence.


You’ll have the option to easily import low or high-fidelity prototypes directly from Figma or build your own prototypes from scratch  by uploading images or screenshots and creating clickable areas. Prototype testing allows you to Identify usability issues and areas for improvement throughout the design process to iterate and improve your designs quickly with your users.

Video Recording 🎬📽️🎞️

After we launch Prototype testing, we’ll be adding Video Recording to allow you to record a participant completing a prototype test. You can capture nonverbal cues like eye rolls and frustration, screens and voices to better understand user experiences, and gain deeper insights for Prototype Testing. Video recording is browser-based, with no plug in required eliminating setup complexities. Consent to record is captured upfront in the testing process and can be tailored to meet your organization’s policies.

Qualitative Insights & AI 🤖✨

You’ll be able to capture meaningful insights faster and more efficiently with the new Insights feature under our Qualitative Insights (Reframer) tool. Create Insights — takeaways or key findings from a study — and keep them organized under the new Insights tab. Create a new Insight from observations or use AI to generate Insights. Each Insight will contain a title, description, and associated observations.

Note that you and your organization will always maintain complete control over any AI engagement. All AI-generated results are editable. Leverage AI as needed for your studies or decide to turn off access completely for your organization.

Surveys Upgrade 📋

We’ll be transforming surveys with enhanced usability, advanced survey logic, and AI integration, empowering you to create, launch, and analyze surveys with ease. Our focus on improving usability will make creating, editing, and launching surveys easier than ever, including new question types, question cloning, question grouping, and AI assistance to streamline survey creation.

Help us build a better Optimal Workshop 🧰

We’re looking for customers to join our research and beta panel to help influence product development. From time to time, you’ll be invited to join us for interviews or surveys, and you’ll be rewarded for your time with a thank-you gift.  


If you’d like to join the team, email product@optimalworkshop.com.

min read
Why you should be using card sorting

On the fence about card sorting and why you should be using it to improve your user experience? Let’s take a look at why you should take advantage of this powerful user research method.

Simply put, card sorting can help you discover how your users think your content should be organized and categorized. Card sorting gives you insight into how people conceptualize, group and label ideas, enabling you to make confident, informed information architecture (IA) decisions.

What is card sorting?

In a card sort, participants sort cards containing different items into groups. You can use the results to figure out how to group and label the information on your website in a way that makes the most sense to your audience. 

Using card sorting at the start of your website build means you’re able to make decisions based on data, not assumptions. Being informed at the start of your website build can mean saving a lot of time later with revisions or rebuilds. Better to build something intuitive now, than be left wondering later why parts of your website aren’t working as you expect.

When should you use card sorting?

It’s best to do card sort research when you want to answer a specific, information-related question. For example, maybe you’re adding a new range of “natural products” to your Health and Beauty site. On the other hand, you may want to redesign how information is grouped together across your entire website.

Card sorting is at its most effective when you’ve got the information and detail you need but you just need guidance on how it's best (most intuitive) to organize it.

While card sorting is typically used in the early stages of the design process, when there’s no live IA, it’s also common to use the technique to make changes to a live IA down the line. 

Card sorting: A powerful way to understand users’ mental models

Card sorting is a powerful tool to understand your users and how they make sense of information when they arrive on your website. A good rule to keep in mind is that what makes sense to you and your colleagues may not make sense to your users.

Using an online card sorting tool like OptimalSort it can be useful to check in with users to understand where they think information should sit on your website. As product ranges increase or change over time, it can also be useful to undertake card sort research when updating your website. 

Let’s take a look at how card sorting might work for an e-commerce website. 

Imagine you run a health and beauty e-commerce business with an active and successful website with a vast range of products that can be grouped in many different ways. At worst the website is clunky and hard to search, making it difficult for shoppers to find the right product, quickly. At best everything is there but it doesn’t quite answer what the user is looking for. It can be incredibly powerful to have a fuller understanding of how our website is viewed from our shoppers (rather than just internally). The goal of our website should be to showcase our products in a way that makes shopping easy, quick and even intuitive.

We are introducing a full range of natural based products that include products intended for babies, children, women and men. These products have previously  been categorized by who they are intended for. But we want to know if there is a better way that these could be made available, especially with a market shift to an increased demand in  natural based products.

By doing some card sorting with OptimalSort, we gather data from users and the pattern that our audience use to group these products. Through the data analysis we have discovered that a large majority of our users would group by natural products first and then by who the product is intended for (baby, children, women, or men). Armed with this insight (amongst others) we can use it to influence our IA. Ultimately, we end up with a far more intuitive and streamlined user experience (UX).

Three ways to use card sorting

Did you know that there are multiple ways to use OptimalSort card sorting? Let’s take a look– you may be surprised.

1. Building a new website

This is by far the biggest use-case for card sorting. When looking at building a new website or making better use of an existing one, utilizing card sorting at the research stage can be insightful and informative. Seeing your website, products and/or navigation from your end user’s perspective can enlighten, inform and assist in creating an enhanced user experience.

2. Combine card sorting and tree testing

When combined with card sorting, tree testing, with Treejack, can help you to improve your navigation and give you a fuller understanding of how your website is used. Tree testing is a technique for evaluating the findability of topics on a website. It’s also commonly known as reverse card sorting and is the perfect technique to complement card sorting. After you’ve analyzed your card sorting results and transformed them into a draft IA, you can test these insights using a tree test. Using this technique, you task users with seeking as opposed to sorting. This technique aims to replicate the experience of using a website – without visual distractions.

Unlike usability testing, tree testing only focuses on the IA of your website. It makes the process of developing an IA much faster, as you can easily make refinements and tweaks without needing to get bogged down in costly redesigns. 

3. Make collaborative design decisions

You can use OptimalSort to get your team involved and let their feedback feed your designs — logos, icons, banners, images, the list goes on. By creating a closed image sort with categories where your team can group designs based on their preferences, you can get some quick feedback to help you figure out where you should focus your efforts.

Use OptimalSort to run your first card sort

Card sorting can take place in person, or online with a tool like OptimalSort. OptimalSort gives you the flexibility to conduct moderated and unmoderated card sorts online. Now, you can collect the data you need, how and when you need it. Plus it only takes a few minutes to design and launch your study. 

Not only is OptimalSort simple to use,  but it’s backed up with the strength of powerful analysis functionality. Taking the pain of trawling screeds of information, OptimalSort pulls out useful, usable insights from your card sorting data. This allows you to quickly identify common groups at a glance with comprehensive and vibrant visualizations and use this data to support design changes and recommendations. 

What is 3D Cluster View?

Part of our OptimalSort analysis is the 3D Cluster View (3DCV). While the addition of ‘3D’ may throw off red flags of being a gimmick, it’s actually entirely appropriate. 

The 3DCV basically allows you to visualize the similarity between cards as three-dimensional spatial relationships. Each point in the 3D visualization represents one of the cards from your original sort. Cards that are closer together were more frequently sorted into the same category. Likewise, when you see 2 cards that are quite far apart, they weren’t sorted together as frequently. If you’d like to find out more take a look at OptimalSort 3DCV, we think it’s pretty clever.

Wrap Up

If you’re now interested in a card sort of your own, we obviously recommend OptimalSort (which you can get started with for free). Or you want to find out more, take a look at our Card Sorting 101.

Happy sorting!

min read
How to get started with tree testing 🌱

Are your visitors really getting the most out of your website? Tree testing (or sometimes referred to as reverse card sorting) takes away the guesswork by telling you how easily, or not, people can find information on your website. Discover why Treejack is the tool of choice for website architects.

What’s tree testing and why does it matter? 🌲 👀

Whether you’re building a website from scratch or improving an existing website, tree testing helps you design your website architecture with confidence. How? Tools like Treejack use analysis to help assess how findable your content is for people visiting your website. 

It helps answer burning questions  like:

  • Do my labels make sense?
  • Is my content grouped logically?
  • Can people find what they want easily and quickly?  If not, why not?

Treejack provides invaluable intel for any Information Architect. Why? Knowing where and why people get lost trying to find your content, gives you a much better chance of fixing the actual problem. And the more easily people can find what they’re looking for, the better their experience which is ultimately better for everyone.

How’s tree testing work? 🌲🌳🌿

Tree testing can be broken down into two main parts: 

  • The Tree - Your tree is essentially your site map – a text-only version of your website structure.
  • The Task - Your task is the activity you ask participants to complete by clicking through your tree and choosing the information they think is right. Tools like Treejack analyse the data generated from doing the task to build a picture of how people actually navigated your content in order to try and achieve your task.  It tells you if they got it right or wrong, the path they took and the time it took them.

Whether you’re new to tree testing or already a convert, effective tree testing using Treejack has some key steps.

Step 1.  The ‘ Why’:  Purpose and goals of tree testing

Ask yourself what part of your information architecture needs improvement – is it your whole website or just parts of it? Also think about your audience, they’re the ones you’re trying to improve the website for so the more you know about their needs the better. 

Tip:  Make the most of what tree testing offers to improve your website by building it into your overall design project plan

Step 2.  The ‘How’:   Build your tree

You can build your tree using two main approaches: 

  • Create your tree in spreadsheet and import it into Treejack or
  • Build your tree in Treejack itself, using the labels and structure of your website.

Tip:  Your category labels are known as ‘parent nodes’. Your information labels are known as ‘child nodes’.

Step 3. The ‘What’: Write your tasks

The quality of your tasks will be reflected in the usefulness of your data so it’s worth making sure you create tasks that really test what you want to improve.

Tip:  Use plain language that feels natural and try to write your tasks in a way that reflects the way people who visit your website might actually think when they are trying to find information on your site.

Step 4.  The ‘Who’:  Recruit participants

The quality of your data will largely depend on the quality of your participants. You want people who are as close to your target audience as possible and with the right attitude - willing and committed to being involved.

Tip:  Consider offering some kind of incentive to participants – it shows you value their involvement.

Step 5.  The ‘insights’: Interpret your results

Now for the fun part – making sense of the results. Treejack presents the data from your tree testing as a series of tables and visualizations. You can download them in a spreadsheet in their raw format or customized to your needs.

Tip:  Use the results to gain quick, practical insights you can act on right away or as a starter to dive deeper into the data.

When should I use tree testing? ⌛

Tree testing is useful whenever you want to find out if your website content is labelled and organised in a way that’s easy to understand.  What’s more it can be applied for any website, big (10+ levels with 10000s of labels) or small (3 levels and 22 labels) and any size in between.  Our advice for using Treejack is simply this: test big, test small, test often.

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.