My journey running a design sprint

AI creates beautiful designs, but only humans can validate if they work

‍

Let's talk about something that's fundamentally reshaping product development: AI-generated designs. It's not just a trendy tool; it's a complete transformation of the design workflow as we know it.

‍

Today's AI design tools aren't just creating mockups, they're generating entire design systems, producing variations at scale, and predicting user preferences before you've even finished your prompt. Instead of spending hours on iterations, designers are exploring dozens of directions in minutes.

‍

This is where platforms like Lovable shine with their vibe coding approach, generating design directions based on emotional and aesthetic inputs rather than just functional requirements, and while this AI-powered innovation is impressive, it raises a critical question for everyone creating digital products: How do we ensure these AI-generated designs actually resonate with real people?

‍

The Gap Between AI Efficiency and Human Connection

‍

The design process has fundamentally shifted. Instead of building from scratch, designers are prompting and curating. Rather than crafting each pixel, they're directing AI to explore design spaces.

‍

The whole interaction feels more experimental. Designers are using natural language to describe desired outcomes, and the AI responses feel like collaborative explorations rather than final deliverables.

‍

This shift has major implications for product teams:

‍

• If you're a product manager, you need to balance AI efficiency with proven user validation methods to ensure designs solve actual user problems.
• UX designers, you're now curating and refining AI outputs. When AI generates interfaces, will real users understand how to use them?
• Visual designers, your expertise is evolving. You need to develop prompting skills while maintaining your critical eye for what actually works.
• And UX researchers, there's an urgent need to validate AI-generated designs with real human feedback before implementation.

‍

The Future of Design: AI Innovation + Human Validation

‍

As AI design tools become more powerful, the teams that thrive will be those who balance technological innovation with human understanding. The winning approach isn't AI alone or human-only design, it's the thoughtful integration of both.

‍

Why Human Validation Is Essential for AI-Generated Designs

‍

AI is revolutionizing design creation, but it has inherent limitations that only human validation can address:

‍

• ‍AI Lacks Contextual Understanding While AI can generate visually impressive designs, it doesn't truly understand cultural nuances, emotional responses, or lived experiences of your users. Only human feedback can verify whether an AI-generated interface feels intuitive rather than just looking good.‍
• The "Uncanny Valley" of AI Design AI-generated designs sometimes create an "almost right but slightly off" feeling, technically correct but missing the human touch. Real user testing helps identify these subtle disconnects that might otherwise go unnoticed by design teams.‍
• AI Reinforces Patterns, Not Breakthroughs AI models are trained on existing design patterns, meaning they excel at iteration but struggle with true innovation. Human validation helps identify when AI-generated designs feel derivative versus when they create genuine emotional connections with users.‍
• Diverse User Needs Require Human Insight AI may not account for accessibility considerations, cultural sensitivities, or edge cases without explicit prompting. Human validation ensures designs work for your entire audience, not just the statistical average.

‍

The Multiplier Effect: Why AI + Human Validation Outperforms Either Approach Alone

‍

The combination of AI-powered design and human validation creates a virtuous cycle that elevates both:

‍

• From Rapid Iteration to Deeper Insights AI allows teams to test more design variations than ever before, gathering richer comparative data through human testing. This breadth of exploration was previously impossible with human-only design processes.‍
• Continuous Learning Loop Human validation of AI designs creates feedback that improves future AI prompts. Over time, this creates a compounding advantage where AI tools become increasingly aligned with real user preferences.‍
• Scale + Depth AI provides the scale to generate numerous options, while human validation provides the depth of understanding required to select the right ones. This combination addresses both the breadth and depth dimensions of effective design.

‍

At Optimal, we're committed to helping you navigate this new landscape by providing the tools you need to ensure AI-generated designs truly resonate with the humans who will use them. Our human validation platform is the essential complement to AI's creative potential, turning promising designs into proven experiences.

‍

Introducing the Optimal + Lovable Integration: Bridging AI Innovation with Human Validation

‍

At Optimal, we've always believed in the power of human feedback to create truly effective designs. Now, with our new Lovable integration, we're making it easier than ever to validate AI-generated designs with real users.

‍

Here's how our integrated approach works:

‍

1. Generate Innovative Designs with Lovable

‍

Lovable allows you to:

‍

• Explore emotional dimensions of design through AI prompting
• Generate multiple design variations in minutes
• Create interfaces that feel aligned with your brand's emotional targets

‍

2. Validate Those Designs with Optimal

‍

Interactive Prototype Testing Our integration lets you import Lovable designs directly as interactive prototypes, allowing users to click, navigate, and experience your AI-generated interfaces in a realistic environment. This reveals critical insights about how users naturally interact with your design.

‍

Ready to Transform Your Design Process?

‍

Try our Optimal + Lovable integration today and experience the power of combining AI innovation with human validation. Your first study is on us! See firsthand how real user feedback can elevate your AI-generated designs from interesting to truly effective.

‍

Try the Optimal + Lovable Integration today

‍

‍

Recently, everyone in the design industry has been talking about design sprints. So, naturally, the team at Optimal Workshop wanted to see what all the fuss was about. I picked up a copy of The Sprint Book and suggested to the team that we try out the technique.

‍

In order to keep momentum, we identified a current problem and decided to run the sprint only two weeks later. The short notice was a bit of a challenge, but in the end we made it work. Here’s a run down of how things went, what worked, what didn’t, and lessons learned.

‍

A sprint is an intensive focused period of time to get a product or feature designed and tested with the goal of knowing whether or not the team should keep investing in the development of the idea. The idea needs to be either validated or not validated by the end of the sprint. In turn, this saves time and resource further down the track by being able to pivot early if the idea doesn’t float.

‍

If you’re following The Sprint Book you might have a structured 5 day plan that looks likes this:

• Day 1 - Understand: Discover the business opportunity, the audience, the competition, the value proposition and define metrics of success.
• Day 2 - Diverge: Explore, develop and iterate creative ways of solving the problem, regardless of feasibility.
• Day 3 - Converge: Identify ideas that fit the next product cycle and explore them in further detail through storyboarding.
• Day 4 - Prototype: Design and prepare prototype(s) that can be tested with people.
• Day 5 - Test: User testing with the product's primary target audience.

‍

When you’re running a design sprint, it’s important that you have the right people in the room. It’s all about focus and working fast; you need the right people around in order to do this and not have any blocks down the path. Team, stakeholder and expert buy-in is key — this is not a task just for a design team!After getting buy in and picking out the people who should be involved (developers, designers, product owner, customer success rep, marketing rep, user researcher), these were my next steps:

‍

Pre-sprint

1. Read the book
2. Panic
3. Send out invites
4. Write the agenda
5. Book a meeting room
6. Organize food and coffee
7. Get supplies (Post-its, paper, Sharpies, laptops, chargers, cameras)

‍

‍

The sprint

‍

Due to scheduling issues we had to split the sprint over the end of the week and weekend. Sprint guidelines suggest you hold it over Monday to Friday — this is a nice block of time but we had to do Thursday to Thursday, with the weekend off in between, which in turn worked really well. We are all self confessed introverts and, to be honest, the thought of spending five solid days workshopping was daunting. At about two days in, we were exhausted and went away for the weekend and came back on Monday feeling sociable and recharged again and ready to examine the work we’d done in the first two days with fresh eyes.

‍

Design sprint activities

‍

During our sprint we completed a range of different activities but here’s a list of some that worked well for us. You can find out more information about how to run most of these over at The Sprint Book website or checkout some great resources over at Design Sprint Kit.

‍

Lightning talks

‍

We kicked off our sprint by having each person give a quick 5-minute talk on one of these topics in the list below. This gave us all an overview of the whole project and since we each had to present, we in turn became the expert in that area and engaged with the topic (rather than just listening to one person deliver all the information).

‍

Our lightning talk topics included:

• Product history - where have we come from so the whole group has an understanding of who we are and why we’ve made the things we’ve made.
• Vision and business goals - (from the product owner or CEO) a look ahead not just of the tools we provide but where we want the business to go in the future.
• User feedback - what have users been saying so far about the idea we’ve chosen for our sprint. This information is collected by our User Research and Customer Success teams.
• Technical review - an overview of our tech and anything we should be aware of (or a look at possible available tech). This is a good chance to get an engineering lead in to share technical opportunities.
• Comparative research - what else is out there, how have other teams or products addressed this problem space?

‍

Empathy exercise

‍

I asked the sprinters to participate in an exercise so that we could gain empathy for those who are using our tools. The task was to pretend we were one of our customers who had to present a dendrogram to some of our team members who are not involved in product development or user research. In this frame of mind, we had to talk through how we might start to draw conclusions from the data presented to the stakeholders. We all gained more empathy for what it’s like to be a researcher trying to use the graphs in our tools to gain insights.

‍

How Might We

‍

In the beginning, it’s important to be open to all ideas. One way we did this was to phrase questions in the format: “How might we…” At this stage (day two) we weren’t trying to come up with solutions — we were trying to work out what problems there were to solve. ‘We’ is a reminder that this is a team effort, and ‘might’ reminds us that it’s just one suggestion that may or may not work (and that’s OK). These questions then get voted on and moved into a workshop for generating ideas (see Crazy 8s).Read a more detailed instructions on how to run a ‘How might we’ session on the Design Sprint Kit website.

‍

Crazy 8s

‍

This activity is a super quick-fire idea generation technique. The gist of it is that each person gets a piece of paper that has been folded 8 times and has 8 minutes to come up with eight ideas (really rough sketches). When time is up, it’s all pens down and the rest of the team gets to review each other's ideas.In our sprint, we gave each person Post-it notes, paper, and set the timer for 8 minutes. At the end of the activity, we put all the sketches on a wall (this is where the art gallery exercise comes in).

‍

‍

‍

Art gallery/Silent critique

‍

The art gallery is the place where all the sketches go. We give everyone dot stickers so they can vote and pull out key ideas from each sketch. This is done silently, as the ideas should be understood without needing explanation from the person who made them. At the end of it you’ve got a kind of heat map, and you can see the ideas that stand out the most. After this first round of voting, the authors of the sketches get to talk through their ideas, then another round of voting begins.

‍

‍

‍

Usability testing and validation

‍

The key part of a design sprint is validation. For one of our sprints we had two parts of our concept that needed validating. To test one part we conducted simple user tests with other members of Optimal Workshop (the feature was an internal tool). For the second part we needed to validate whether we had the data to continue with this project, so we had our data scientist run some numbers and predictions for us.

‍

‍

Challenges and outcomes

‍

One of our key team members, Rebecca, was working remotely during the sprint. To make things easier for her, we set up 2 cameras: one pointed to the whiteboard, the other was focused on the rest of the sprint team sitting at the table. Next to that, we set up a monitor so we could see Rebecca.

‍

Engaging in workshop activities is a lot harder when working remotely. Rebecca would get around this by completing the activities and take photos to send to us.

‍

‍

Lessons

‍

• Lightning talks are a great way to have each person contribute up front and feel invested in the process.
• Sprints are energy intensive. Make sure you’re in a good place with plenty of fresh air with comfortable chairs and a break out space. We like to split the five days up so that we get a weekend break.
• Give people plenty of notice to clear their schedules. Asking busy people to take five days from their schedule might not go down too well. Make sure they know why you’d like them there and what they should expect from the week. Send them an outline of the agenda. Ideally, have a chat in person and get them excited to be part of it.
• Invite the right people. It’s important that you get the right kind of people from different parts of the company involved in your sprint. The role they play in day-to-day work doesn’t matter too much for this. We’re all mainly using pens and paper and the more types of brains in the room the better. Looking back, what we really needed on our team was a customer support team member. They have the experience and knowledge about our customers that we don’t have.
• Choose the right sprint problem. The project we chose for our first sprint wasn’t really suited for a design sprint. We went in with a well defined problem and a suggested solution from the team instead of having a project that needed fresh ideas. This made the activities like ‘How Might We’ seem very redundant. The challenge we decided to tackle ended up being more of a data prototype (spreadsheets!). We used the week to validate assumptions around how we can better use data and how we can write a script to automate some internal processes. We got the prototype working and tested but due to the nature of the project we will have to run this experiment in the background for a few months before any building happens.

‍

Overall, this design sprint was a great team bonding experience and we felt pleased with what we achieved in such a short amount of time. Naturally, here at Optimal Workshop, we're experimenters at heart and we will keep exploring new ways to work across teams and find a good middle ground.

‍

Further reading

‍

• The Sprint Book - a book by Jake Knapp
• Design Sprint Kit — a whole bunch of presentation decks, and activities for sprints
• Sprint Stories — some case studies that show the processes and value of design sprints
• How to run a remote design sprint without going crazy - an article by Jake Knapp
• Choosing the right sprint problem - a blog by Lisa Jansen

The term ‘usability’ captures sentiments of how usable, useful, enjoyable, and intuitive a website or app is perceived by users. By its very nature, usability is somewhat subjective. But what we’re really looking for when we talk about usability is how well a website can be used to achieve a specific task or goal. Using this definition we can analyze usability metrics (standard units of measurement) to understand how well user experience design is performing.

‍

Usability metrics provide helpful insights before and after any digital product is launched. They help us form a deeper understanding of how we can design with the user front of mind. This user-centered design approach is considered the best-practice in building effective information architecture and user experiences that help websites, apps, and software meet and exceed users' needs.

‍

In this article, we’ll highlight key usability metrics, how to measure and understand them, and how you can apply them to improve user experience.

‍

Understanding Usability Metrics

‍

Usability metrics aim to understand three core elements of usability, namely: effectiveness, efficiency, and satisfaction. A variety of research techniques offer designers an avenue for quantifying usability. Quantifying usability is key because we want to measure and understand it objectively, rather than making assumptions.

‍

Types of Usability Metrics
‍

There are a few key metrics that we can measure directly if we’re looking to quantify effectiveness, efficiency, and satisfaction. Here are four common examples:

‍

• Success rate: Also known as ‘completion rate’, success rate is the percentage of users who were able to successfully complete the tasks.
• Time-based efficiency: Also known as ‘time on task’, time-based efficiency measures how much time a user needs to complete a certain task.
• Number of errors: Sounds like what it is! It measures the average number of times where an error occurred per user when performing a given task.
• Post-task satisfaction: Measures a user's general impression or satisfaction after completing (or not completing) a given task.

‍

How to Collect Usability Metrics

‍
Usability metrics are outputs from research techniques deployed when conducting usability testing. Usability testing in web design, for example, involves assessing how a user interacts with the website by observing (and listening to) users completing defined tasks, such as purchasing a product or signing up for newsletters.

‍

Conducting usability testing and collecting usability metrics usually involves:

‍

• Defining a set of tasks that you want to test
• Recruitment of test participants
• Observing participants (remotely or in-person)
• Recording detailed observations
• Follow-up satisfaction survey or questionnaire

‍

Tools such Reframer are helpful in conducting usability tests remotely, and they enable live collaboration of multiple team members. It is extremely handy when trying to record and organize those insightful observations! Using paper prototypes is an inexpensive way to test usability early in the design process.

‍

The Importance of Usability Metrics in User-Centered Design

‍

User-centered design challenges designers to put user needs first. This means in order to deploy user-centered design, you need to understand your user. This is where usability testing and metrics add value to website and app performance; they provide direct, objective insight into user behavior, needs, and frustrations. If your user isn’t getting what they want or expect, they’ll simply leave and look elsewhere.

‍

Usability metrics identify which parts of your design aren’t hitting the mark. Recognizing where users might be having trouble completing certain actions, or where users are regularly making errors, are vital insights when implementing user-centered design. In short, user-centered design relies on data-driven user insight.

‍

But why hark on about usability metrics and user-centered design? Because at the heart of most successful businesses is a well-solved user problem. Take Spotify, for example, which solved the problem of dodgy, pirated digital files being so unreliable. People liked access to free digital music, but they had to battle viruses and fake files to get it. With Spotify, for a small monthly fee, or the cost of listening to a few ads, users have the best of both worlds. The same principle applies to user experience - identify recurring problems, then solve them.

‍

Best Practices for Using Usability Metrics

‍

Usability metrics should be analyzed by design teams of every size. However, there are some things to bear in mind when using usability metrics to inform design decisions:

‍

• Defining success: Usability metrics are only valuable if they are being measured against clearly defined benchmarks. Many tasks and processes are unique to each business, so use appropriate comparisons and targets; usually in the form of an ‘optimized’ user (a user with high efficiency).
• Real user metrics: Be sure to test with participants that represent your final user base. For example, there’s little point in testing your team, who will likely be intimately aware of your business structure, terminology, and internal workflows.
• Test early: Usability testing and subsequent usability metrics provide the most impact early on in the design process. This usually means testing an early prototype or even a paper prototype. Early testing helps to avoid any significant, unforeseen challenges that could be difficult to rewind in your information architecture.
• Regular testing: Usability metrics can change over time as user behavior and familiarity with digital products evolve. You should also test and analyze the usability of new feature releases on your website or app.

‍

Remember, data analysis is only as good as the data itself. Give yourself the best chance of designing exceptional user experiences by collecting, researching, and analyzing meaningful and accurate usability metrics.

‍

Conclusion

‍

Usability metrics are a guiding light when it comes to user experience. As the old saying goes, “you can’t manage what you can’t measure”. By including usability metrics in your design process, you invite direct user feedback into your product. This is ideal because we want to leave any assumptions or guesswork about user experience at the door.

‍

User-centered design inherently relies on constant user research. Usability metrics such as success rate, time-based efficiency, number of errors, and post-task satisfaction will highlight potential shortcomings in your design. Subsequently, they identify where improvements can be made, AND they lay down a benchmark to check whether any resulting updates addressed the issues.

‍

Ready to start collecting and analyzing usability metrics? Check out our guide to planning and running effective usability tests to get a head start!