Moderated vs unmoderated research: which approach is best?

Return on investment (ROI) is often the term on everyone’s lips when starting a big project or even when reviewing a website. It’s especially popular with those that hold the purse strings.  As UX researchers it is important to consider the ROI of the work we do and understand how to measure this. 

‍

We’ve lined up 5 key ways to measure ROI for UX research to help you get the conversation underway with stakeholders so you can show real and tangible benefits to your organization. 

‍

‍

1. Meet and exceed user expectations

‍

Put simply, a product that meets and exceeds user expectations leads to increased revenue. When potential buyers are able to find and purchase what they’re looking for, easily, they’ll complete their purchase, and are far more likely to come back. The simple fact that users can finish their task will increase sales and improve overall customer satisfaction which has an influence on their loyalty. Repeat business means repeat sales. Means increased revenue.

‍

Creating, developing and maintaining a usable website is more important than you might think. And this is measurable! Tracking and analyzing website performance prior to the UX research and after can be insightful and directly influenced by changes made based on UX research.

‍

Measurable: review the website (product) performance prior to UX research and after changes have been made. The increase in clicks, completed tasks and/or baskets will tell the story.

‍

‍

2. Reduce development time

‍

UX research done at the initial stages of a project can lead to a reduction in development time of by 33% to 50%! And reduced time developing, means reduced costs (people and overheads) and a speedier to market date. What’s not to love? 

‍

Measurable: This one is a little more tricky as you have saved time (and cost) up front. Aiding in speed to market and performance prior to execution. Internal stakeholder research may be of value post the live date to understand how the project went.

‍

‍

3. Ongoing development costs

‍

And the double hitter? Creating a product that has the user in mind up front, reduces the need to rehash or revisit as quickly. Reducing ongoing costs. Early UX research can help with the detection of errors early on in the development process. Fixing errors after development costs a company up to 100 times more than dealing with the same error before development.

‍

Measureable: Again, as UX research has saved time and money up front this one can be difficult to track. Though depending on your organization and previous projects you could conduct internal research to understand how the project compares and the time and cost savings.

‍

‍

4. Meeting user requirements

‍

Did you know that 70% of projects fail due to the lack of user acceptance? This is often because project managers fail to understand the user requirements properly. Thanks to UX research early on, gaining insights into users and only spending time developing the functions users actually want, saving time and reducing development costs. Make sure you get confirmation on those requirements by iterative testing. As always, fail early, fail often. Robust testing up front means that in the end, you’ll have a product that will meet the needs of the user.

‍

Measurable: Where is the product currently? How does it perform? Set a benchmark up front and review post UX research. The deliverables should make the ROI obvious.

‍

‍

5. Investing in UX research leads to an essential competitive advantage.

‍

Thanks to UX research you can find out exactly what your customers want, need and expect from you. This gives you a competitive advantage over other companies in your market. But you should be aware that more and more companies are investing in UX while customers are ever more demanding, their expectations continue to grow and they don’t tolerate bad experiences. And going elsewhere is an easy decision to make.

‍

Measurable: Murky this one, but no less important. Knowing, understanding and responding to competitors can help keep you in the lead, and developing products that meet and exceed those user expectations.

‍

‍

Wrap up

‍

Showing the ROI on the work we do is an essential part of getting key stakeholders on board with our research. It can be challenging to talk the same language, ultimately we all want the same outcome…a product that works well for our users, and delivers additional revenue.

‍

For some continued reading (or watching in this case), Anna Bek, Product and Delivery Manager at Xplor explored the same concept of "How to measure experience" during her UX New Zealand 2020 – watch it here as she shares a perspective on UX ROI.

Introduction

‍

User research is invaluable, but in fast-paced environments, researchers often struggle with tight deadlines, limited resources, and the need to prove their impact. In our recent UX Insider webinar, Weidan Li, Senior UX Researcher at Seek, shared insights on Efficient Research—an approach that optimizes Speed, Quality, and Impact to maximize the return on investment (ROI) of understanding customers.

‍

At the heart of this approach is the Efficient Research Framework, which balances these three critical factors:

• Speed – Conducting research quickly without sacrificing key insights.
• Quality – Ensuring rigor and reliability in findings.
• Impact – Making sure research leads to meaningful business and product changes.

‍

Within this framework, Weidan outlined nine tactics that help UX researchers work more effectively. Let’s dive in.

‍

‍

1. Time Allocation: Invest in What Matters Most

‍

Not all research requires the same level of depth. Efficient researchers prioritize their time by categorizing projects based on urgency and impact:

• High-stakes decisions (e.g., launching a new product) require deep research.
• Routine optimizations (e.g., tweaking UI elements) can rely on quick testing methods.
• Low-impact changes may not need research at all.

‍

By allocating time wisely, researchers can avoid spending weeks on minor issues while ensuring critical decisions are well-informed.

‍

‍

2. Assistance of AI: Let Technology Handle the Heavy Lifting

‍

AI is transforming UX research, enabling faster and more scalable insights. Weidan suggests using AI to:

• Automate data analysis – AI can quickly analyze survey responses, transcripts, and usability test results.
• Generate research summaries – Tools like ChatGPT can help synthesize findings into digestible insights.
• Speed up recruitment – AI-powered platforms can help find and screen participants efficiently.

‍

While AI can’t replace human judgment, it can free up researchers to focus on higher-value tasks like interpreting results and influencing strategy.

‍

‍

3. Collaboration: Make Research a Team Sport

‍

Research has a greater impact when it’s embedded into the product development process. Weidan emphasizes:

• Co-creating research plans with designers, PMs, and engineers to align on priorities.
• Involving stakeholders in synthesis sessions so insights don’t sit in a report.
• Encouraging non-researchers to run lightweight studies, such as A/B tests or quick usability checks.

‍

When research is shared and collaborative, it leads to faster adoption of insights and stronger decision-making.

‍

‍

4. Prioritization: Focus on the Right Questions

‍

With limited resources, researchers must choose their battles wisely. Weidan recommends using a prioritization framework to assess:

• Business impact – Will this research influence a high-stakes decision?
• User impact – Does it address a major pain point?
• Feasibility – Can we conduct this research quickly and effectively?

‍

By filtering out low-priority projects, researchers can avoid research for research’s sake and focus on what truly drives change.

‍

‍

5. Depth of Understanding: Go Beyond Surface-Level Insights

‍

Speed is important, but efficient research isn’t about cutting corners. Weidan stresses that even quick studies should provide a deep understanding of users by:

• Asking why, not just what – Observing behavior is useful, but uncovering motivations is key.
• Using triangulation – Combining methods (e.g., usability tests + surveys) to validate findings.
• Revisiting past research – Leveraging existing insights instead of starting from scratch.

‍

Balancing speed with depth ensures research is not just fast, but meaningful.

‍

‍

6. Anticipation: Stay Ahead of Research Needs

‍

Proactive researchers don’t wait for stakeholders to request studies—they anticipate needs and set up research ahead of time. This means:

• Building a research roadmap that aligns with upcoming product decisions.
• Running continuous discovery research so teams have a backlog of insights to pull from.
• Creating self-serve research repositories where teams can find relevant past studies.

‍

By anticipating research needs, UX teams can reduce last-minute requests and deliver insights exactly when they’re needed.

‍

‍

7. Justification of Methodology: Explain Why Your Approach Works

‍

Stakeholders may question research methods, especially when they seem time-consuming or expensive. Weidan highlights the importance of educating teams on why specific methods are used:

• Clearly explain why qualitative research is needed when stakeholders push for just numbers.
• Show real-world examples of how past research has led to business success.
• Provide a trade-off analysis (e.g., “This method is faster but provides less depth”) to help teams make informed choices.

‍

A well-justified approach ensures research is respected and acted upon.

‍

‍

8. Individual Engagement: Tailor Research Communication to Your Audience

‍

Not all stakeholders consume research the same way. Weidan recommends adapting insights to fit different audiences:

• Executives – Focus on high-level impact and key takeaways.
• Product teams – Provide actionable recommendations tied to specific features.
• Designers & Engineers – Share usability findings with video clips or screenshots.

‍

By delivering insights in the right format, researchers increase the likelihood of stakeholder buy-in and action.

‍

‍

9. Business Actions: Ensure Research Leads to Real Change

‍

The ultimate goal of research is not just understanding users—but driving business decisions. To ensure research leads to action:

• Follow up on implementation – Track whether teams apply the insights.
• Tie findings to key metrics – Show how research affects conversion rates, retention, or engagement.
• Advocate for iterative research – Encourage teams to re-test and refine based on new data.

‍

Research is most valuable when it translates into real business outcomes.

‍

‍

Final Thoughts: Research That Moves the Needle

‍

Efficient research is not just about doing more, faster—it’s about balancing speed, quality, and impact to maximize its influence. Weidan’s nine tactics help UX researchers work smarter by:

‍
✔️  Prioritizing high-impact work
✔️  Leveraging AI and collaboration
✔️  Communicating research in a way that drives action

‍

By adopting these strategies, UX teams can ensure their research is not just insightful, but transformational.

‍

‍

Watch the full webinar here

‍

Introduction

‍

User Experience (UX) research has always been about people. How they think, how they behave, what they need, and—just as importantly—what they don’t yet realise they need. Traditional UX methodologies have long relied on direct human input: interviews, usability testing, surveys, and behavioral observation. The assumption was clear—if you want to understand people, you have to engage with real humans.

‍

But in 2025, that assumption is being challenged.

‍

The emergence of digital twins and synthetic users—AI-powered simulations of human behavior—is changing how researchers approach user insights. These technologies claim to solve persistent UX research problems: slow participant recruitment, small sample sizes, high costs, and research timelines that struggle to keep pace with product development. The promise is enticing: instantly accessible, infinitely scalable users who can test, interact, and generate feedback without the logistical headaches of working with real participants.

‍

Yet, as with any new technology, there are trade-offs. While digital twins may unlock efficiencies, they also raise important questions: Can they truly replicate human complexity? Where do they fit within existing research practices? What risks do they introduce?

‍

This article explores the evolving role of digital twins in UX research—where they excel, where they fall short, and what their rise means for the future of human-centered design.

‍

‍

The Traditional UX Research Model: Why Change?

‍

For decades, UX research has been grounded in methodologies that involve direct human participation. The core methods—usability testing, user interviews, ethnographic research, and behavioral analytics—have been refined to account for the unpredictability of human nature.

‍

This approach works well, but it has challenges:

‍

1. Participant recruitment is time-consuming. Finding the right users—especially niche audiences—can be a logistical hurdle, often requiring specialised panels, incentives, and scheduling gymnastics.
   ‍
2. Research is expensive. Incentives, moderation, analysis, and recruitment all add to the cost. A single usability study can run into tens of thousands of dollars.
   ‍
3. Small sample sizes create risk. Budget and timeline constraints often mean testing with small groups, leaving room for blind spots and bias.
   ‍
4. Long feedback loops slow decision-making. By the time research is completed, product teams may have already moved on, limiting its impact.

‍

In short: traditional UX research provides depth and authenticity, but it’s not always fast or scalable.

‍

Digital twins and synthetic users aim to change that.

‍

‍

What Are Digital Twins and Synthetic Users?

‍

While the terms digital twins and synthetic users are sometimes used interchangeably, they are distinct concepts.

‍

Digital Twins: Simulating Real-World Behavior

‍

A digital twin is a data-driven virtual representation of a real-world entity. Originally developed for industrial applications, digital twins replicate machines, environments, and human behavior in a digital space. They can be updated in real time using live data, allowing organisations to analyse scenarios, predict outcomes, and optimise performance.

‍

In UX research, human digital twins attempt to replicate real users' behavioral patterns, decision-making processes, and interactions. They draw on existing datasets to mirror real-world users dynamically, adapting based on real-time inputs.

‍

‍

Synthetic Users: AI-Generated Research Participants

‍

While a digital twin is a mirror of a real entity, a synthetic user is a fabricated research participant—a simulation that mimics human decision-making, behaviors, and responses. These AI-generated personas can be used in research scenarios to interact with products, answer questions, and simulate user journeys.

‍

Unlike traditional user personas (which are static profiles based on aggregated research), synthetic users are interactive and capable of generating dynamic feedback. They aren’t modeled after a specific real-world person, but rather a combination of user behaviors drawn from large datasets.

‍

Think of it this way:

‍

• A digital twin is a highly detailed, data-driven clone of a specific person, customer segment, or process.
• A synthetic user is a fictional but realistic simulation of a potential user, generated based on behavioral patterns and demographic characteristics.

‍

Both approaches are still evolving, but their potential applications in UX research are already taking shape.

‍

‍

Where Digital Twins and Synthetic Users Fit into UX Research

‍

‍

The appeal of AI-generated users is undeniable. They can:

‍

• Scale instantly – Test designs with thousands of simulated users, rather than just a handful of real participants.
• Eliminate recruitment bottlenecks – No need to chase down participants or schedule interviews.
• Reduce costs – No incentives, no travel, no last-minute no-shows.
• Enable rapid iteration – Get user insights in real time and adjust designs on the fly.
• Generate insights on sensitive topics – Synthetic users can explore scenarios that real participants might find too personal or intrusive.

‍

‍

These capabilities make digital twins particularly useful for:

‍

• Early-stage concept validation – Rapidly test ideas before committing to development.
• Edge case identification – Run simulations to explore rare but critical user scenarios.
• Pre-testing before live usability sessions – Identify glaring issues before investing in human research.

‍

However, digital twins and synthetic users are not a replacement for human research. Their effectiveness is limited in areas where emotional, cultural, and contextual factors play a major role.

‍

‍

The Risks and Limitations of AI-Driven UX Research

‍

For all their promise, digital twins and synthetic users introduce new challenges.

‍

1. They lack genuine emotional responses.
   AI can analyse sentiment, but it doesn’t feel frustration, delight, or confusion the way a human does. UX is often about unexpected moments—the frustrations, workarounds, and “aha” realisations that define real-world use.
   ‍
2. Bias is a real problem.
   AI models are trained on existing datasets, meaning they inherit and amplify biases in those datasets. If synthetic users are based on an incomplete or non-diverse dataset, the research insights they generate will be skewed.
   ‍
3. They struggle with novelty.
   Humans are unpredictable. They find unexpected uses for products, misunderstand instructions, and behave irrationally. AI models, no matter how advanced, can only predict behavior based on past patterns—not the unexpected ways real users might engage with a product.
   ‍
4. They require careful validation.
   How do we know that insights from digital twins align with real-world user behavior? Without rigorous validation against human data, there’s a risk of over-reliance on synthetic feedback that doesn’t reflect reality.

‍

‍

A Hybrid Future: AI + Human UX Research

‍

Rather than viewing digital twins as a replacement for human research, the best UX teams will integrate them as a complementary tool.

‍

Where AI Can Lead:

‍

• Large-scale pattern identification
• Early-stage usability evaluations
• Speeding up research cycles
• Automating repetitive testing

‍

Where Humans Remain Essential:

‍

• Understanding emotion, frustration, and delight
• Detecting unexpected behaviors
• Validating insights with real-world context
• Ethical considerations and cultural nuance

‍

The future of UX research is not about choosing between AI and human research—it’s about blending the strengths of both.

‍

‍

Final Thoughts: Proceeding With Caution and Curiosity

‍

Digital twins and synthetic users are exciting, but they are not a magic bullet. They cannot fully replace human users, and relying on them exclusively could lead to false confidence in flawed insights.

‍

Instead, UX researchers should view these technologies as powerful, but imperfect tools—best used in combination with traditional research methods.

‍

As with any new technology, thoughtful implementation is key. The real opportunity lies in designing research methodologies that harness the speed and scale of AI without losing the depth, nuance, and humanity that make UX research truly valuable.

‍

The challenge ahead isn’t about choosing between human or synthetic research. It’s about finding the right balance—one that keeps user experience truly human-centered, even in an AI-driven world.

‍

This article was researched with the help of Perplexity.ai. 

‍