February 20, 2024
3 min

Meera Pankhania: From funding to delivery - Ensuring alignment from start to finish

It’s a chicken and egg situation when it comes to securing funding for a large transformation program in government. On one hand, you need to submit a business case and, as part of that, you need to make early decisions about how you might approach and deliver the program of work. On the other hand, you need to know enough about the problem you are going to solve to ensure you have sufficient funding to understand the problem better, hire the right people, design the right service, and build it the right way. 

Now imagine securing hundreds of millions of dollars to design and build a service, but not feeling confident about what the user needs are. What if you had the opportunity to change this common predicament and influence your leadership team to carry out alignment activities, all while successfully delivering within the committed time frames?

Meera Pankhania, Design Director and Co-founder of Propel Design, recently spoke at UX New Zealand, the leading UX and IA conference in New Zealand hosted by Optimal Workshop, on traceability and her learnings from delivering a $300 million Government program.

In her talk, Meera helps us understand how to use service traceability techniques in our work and apply them to any environment - ensuring we design and build the best service possible, no matter the funding model.

Background on Meera Pankhania

As a design leader, Meera is all about working on complex, purpose-driven challenges. She helps organizations take a human-centric approach to service transformation and helps deliver impactful, pragmatic outcomes while building capability and leading teams through growth and change.

Meera co-founded Propel Design, a strategic research, design, and delivery consultancy in late 2020. She has 15 years of experience in service design, inclusive design, and product management across the private, non-profit, and public sectors in both the UK and Australia. 

Meera is particularly interested in policy and social design. After a stint in the Australian Public Service, Meera was appointed as a senior policy adviser to the NSW Minister for Customer Service, Hon. Victor Dominello MP. In this role, she played a part in NSW’s response to the COVID pandemic, flexing her design leadership skills in a new, challenging, and important context.

Contact Details:

Email address: meera@propeldesign.com.au

Find Meera on LinkedIn  

From funding to delivery: ensuring alignment from start to finish 🏁🎉👏

Meera’s talk explores a fascinating case study within the Department of Employment Services (Australia) where a substantial funding investment of around $300 million set the stage for a transformative journey. This funding supported the delivery of a revamped Employment Services Model, which had the goal of delivering better services to job seekers and employers, and a better system for providers within this system. The project had a focus on aligning teams prior to delivery, which resulted in a huge amount of groundwork for Meera.

Her journey involved engaging various stakeholders within the department, including executives, to understand the program as a whole and what exactly needed to be delivered. “Traceability” became the watchword for this project, which is laid out in three phases.

  • Phase 1: Aligning key deliverables
  • Phase 2: Ensuring delivery readiness
  • Phase 3: Building sustainable work practices

Phase 1: Aligning key deliverables 🧮

Research and discovery (pre-delivery)

Meera’s work initially meant conducting extensive research and engagement with executives, product managers, researchers, designers, and policymakers. Through this process, a common theme was identified – the urgent (and perhaps misguided) need to start delivering! Often, organizations focus on obtaining funding without adequately understanding the complexities involved in delivering the right services to the right users, leading to half-baked delivery.

After this initial research, some general themes started to emerge:

  1. Assumptions were made that still needed validation
  2. Teams weren’t entirely sure that they understood the user’s needs
  3. A lack of holistic understanding of how much research and design was needed

The conclusion of this phase was that “what” needed to be delivered wasn’t clearly defined. The same was true for “how” it would be delivered.

Traceability

Meera’s journey heavily revolved around the concept of "traceability” and sought to ensure that every step taken within the department was aligned with the ultimate goal of improving employment services. Traceability meant having a clear origin and development path for every decision and action taken. This is particularly important when spending taxpayer dollars!

So, over the course of eight weeks (which turned out to be much longer), the team went through a process of combing through documents in an effort to bring everything together to make sense of the program as a whole. This involved some planning, user journey mapping, and testing and refinement. 

Documenting Key Artifacts

Numerous artifacts and documents played a crucial role in shaping decisions. Meera and her team gathered and organized these artifacts, including policy requirements, legislation, business cases, product and program roadmaps, service maps, and blueprints. The team also included prior research insights and vision documents which helped to shape a holistic view of the required output.

After an effort of combing through the program documents and laying everything out, it became clear that there were a lot of gaps and a LOT to do.

Prioritising tasks

As a result of these gaps, a process of task prioritization was necessary. Tasks were categorized based on a series of factors and then mapped out based on things like user touch points, pain points, features, business policy, and technical capabilities.

This then enabled Meera and the team to create Product Summary Tiles. These tiles meant that each product team had its own summary ahead of a series of planning sessions. It gave them as much context (provided by the traceability exercise) as possible to help with planning. Essentially, these tiles provided teams with a comprehensive overview of their projects i.e. what their user needs, what certain policies require them to deliver, etc.  

Phase 2: Ensuring delivery readiness 🙌🏻

Meera wanted every team to feel confident that we weren’t doing too much or too little in order to design and build the right service, the right way.

Standard design and research check-ins were well adopted, which was a great start, but Meera and the team also built a Delivery Readiness Tool. It was used to assess a team's readiness to move forward with a project. This tool includes questions related to the development phase, user research, alignment with the business case, consideration of policy requirements, and more. Ultimately, it ensures that teams have considered all necessary factors before progressing further. 

Phase 3: Building sustainable work practices 🍃

As the program progressed, several sustainable work practices emerged which Government executives were keen to retain going forward.

Some of these included:

  • ResearchOps Practice: The team established a research operations practice, streamlining research efforts and ensuring that ongoing research was conducted efficiently and effectively.
  • Consistent Design Artifacts: Templates and consistent design artifacts were created, reducing friction and ensuring that teams going forward started from a common baseline.
  • Design Authority and Ways of Working: A design authority was established to elevate and share best practices across the program.
  • Centralized and Decentralized Team Models: The program showcased the effectiveness of a combination of centralized and decentralized team models. A central design team provided guidance and support, while service design leads within specific service lines ensured alignment and consistency.

Why it matters 🔥

Meera's journey serves as a valuable resource for those working on complex design programs, emphasizing the significance of aligning diverse stakeholders and maintaining traceability. Alignment and traceability are critical to ensuring that programs never lose sight of the problem they’re trying to solve, both from the user and organization’s perspective. They’re also critical to delivering on time and within budget!

Traceability key takeaways 🥡

  • Early Alignment Matters: While early alignment is ideal, it's never too late to embark on a traceability journey. It can uncover gaps, increase confidence in decision-making, and ensure that the right services are delivered.
  • Identify and audit: You never know what artifacts will shape your journey. Identify everything early, and don’t be afraid to get clarity on things you’re not sure about.
  • Conducting traceability is always worthwhile: Even if you don’t find many gaps in your program, you will at least gain a high level of confidence that your delivery is focused on the right things.

Delivery readiness key takeaways 🥡

  • Skills Mix is Vital: Assess and adapt team member roles to match their skills and experiences, ensuring they are positioned optimally.
  • Not Everyone Shares the Same Passion: Recognize that not everyone will share the same level of passion for design and research. Make the relevance of these practices clear to all team members.

Sustainability key takeaways 🥡

  • One Size Doesn't Fit All: Tailor methodologies, templates, and practices to the specific needs of your organization.
  • Collaboration is Key: Foster a sense of community and collective responsibility within teams, encouraging shared ownership of project outcomes.

Share this article
Author
Optimal
Workshop

Related articles

View all blog articles
Learn more
1 min read

Designing for delight in the industry of fun: My story and observations

Emotional. Playful. Delightful.These words resonate with user experience (UX) practitioners. We put them in the titles of design books. We build products that move up the design hierarchy of needs, with the goal to go beyond just reliability, usability, and productivity. We want to truly delight the people who use our products.Designing for delight has parallels in the physical world. I see this in restaurants which offer not only delicious food but also an inviting atmosphere; in stores that don’t just sell clothes but also provide superior customer service. Whole industries operate on designing for delight.

The amusement industry has done this for over 500 years. The world’s oldest operating amusement park, Bakken, first opened for guests in 1583 – about 300 years before the first modern roller coaster. Amusement parks experienced a boom in growth in the US in the 1970s. As of November 2014, China had 59 new amusement parks under construction. Today, hundreds of millions of guests each year visit amusement parks throughout the world.I've been fortunate to work in the amusement industry as the owner of a digital UX design company called Thrill & Create. Here is my story of how I got to do this kind of work, and my observations as a UX practitioner in this market.

Making user-centred purveyors of joy

I've been a fan of amusement parks for most of my life. And it’s somewhat hereditary. Much of my family still lives in Central Florida, and several of them have annual passes to Walt Disney World. My mom was a Cast Member at the Magic Kingdom during its opening season.Although I grew up living east of Washington, DC, I spent most of my childhood waiting for our annual trip to an amusement park. I was a different kind of amusement enthusiast: scared to death of heights, loath to ride roller coasters, but so interested in water rides and swimming that my family thought I was a fish. The love for roller coasters would show up much later. But the collection of park maps from our annual trips grew, and in the pre-RollerCoaster Tycoon days, I would sketch designs for amusement parks.

In college, my interest in amusement received a healthy boost from the internet. In the mid-2000s, before fan communities shifted toward Facebook, unofficial websites were quite popular. My home park — a roller coaster enthusiasts’ term for the park that we visit most frequently, not necessarily the closest park to us — had several of these fansites.The fansites would typically last for a year or two and enjoy somewhat of a rivalry with other fansites before their creators would move on to a different hobby and close their sites. The fansites’ forums became an interesting place to share knowledge and learn history about my home park. They also gave us a place to discuss what we would do if we owned the park, to echo rumors we had heard, and start our own rumors. Some quite active forums still exist for this.

Of course, many of us on the forums wanted to be the first to hear a rumor. So we follow the industry blogs, which are typically the first sources of the news. Screamscape has been announcing amusement-industry rumors since the 1990s. And Screamscape and other sites like it announce news not only in the parks but around the industry. Regular Screamscape readers learn about ride manufacturers, trade names for each kind of ride, industry trade shows, and much more. And the International Association of Amusement Parks and Attractions (IAAPA) keeps an eye on the industry as well.Several years ago, I transitioned from being a software developer to starting a user experience design company. It is now called Thrill & Create. I was faced with a challenge: how to compete against the commoditization of freelance design services. Ultimately, selecting a niche was the answer. And seeing IAAPA’s iconic roller coaster sign outside the Orange County Convention Center during a trip to Central Florida was all I needed to shift my strategy toward the amusement industry.

What UX looks like in the Amusement Industry

Periodically, I see new articles about amusement parks in UX blogs. Here are my observations about how UX looks in the industry, both in the physical world and the digital world.

Parks are focused on interactive rides

The amusement industry is known for introducing rides that are bigger, taller, and faster. But the industry has a more interactive future. Using cleverly-designed shops which produce some of the longest waits in the park, Universal Studios has sold many interactive wands to give guests additional experiences in Hogsmeade and Diagon Alley.

amusement UX
Buzz Lightyear's Astro Blasters at Disneyland

At IAAPA, interactivity and interactive rides are very hot topics. Interactive shooting dark rides came to many parks in the early 2000s. Wonder Mountain’s Guardian, a 2014 addition to Canada’s Wonderland, features the world’s longest interactive screen and a ride program that changes completely for the Halloween season.Also coming in 2015 is the elaborately-themed Justice League: Battle for Metropolis rides at two Six Flags parks. And Wet‘n’Wild Las Vegas will debut 'Slideboarding', which allows riders to participate in a video game by touching targets on their way down the slide, and is marketed as 'the world's first waterslide gaming experience'.

UX design is well-established for the physical space

Terms like 'amusement industry', 'attractions industry', and 'themed entertainment industry' can be interchangeable, but they do have different focuses, and different user experience design needs. The amusement industry encompasses amusement parks, theme parks, zoos, aquaria, museums, and their suppliers; the attractions industry also includes other visitor attractions.Several experience design companies have worked extensively on user-centered themed entertainment.

Jack Rouse Associates, who see themselves as “audience advocates”, have worked with over 35 clients in themed entertainment, including Universal, Ocean Park, and LEGOLAND. Thinkwell Group, which touts a 'guest-centric approach to design', showcases 14 theme park and resort projects and attractions work in 12 countries.Consultants in the industry have been more intentionally user-centered. Sasha Bailyn and her team at Entertainment Designer write regularly about physical-world experience design, including UX, in themed entertainment. Russell Essary, owner of Interactive Magic, applies user-centered design to exhibit design, wayfinding, game design, and much more.

In-house teams, agencies, and freelancers are becoming more common

Several large park chains have in-house or contracted UX design teams. Most mid-sized parks work with in-house marketing staff or with outside design companies. One company I know of specializes in web design and development for the amusement industry. Smaller parks and ride companies tend to work with local web designers, or occasionally free website vendors.

amusement ux
Screenshot of the Memphis Zoo homepage, designed by Speak Creative

Some of the best redesigns in the amusement industry recently have involved UX designers. Parc Astérix, north of Paris, hired a UX designer for a redesign with immersive pictures, interesting shapes, and unique iconography. SeaWorld Parks & Entertainment, working with UX designers, unifies their brand strongly across their corporate site, the sites for SeaWorld and Busch Gardens, and individual park sites. The Memphis Zoo’s website, which showcases videos of their animals, was built by an agency that provides UI design and UX design among their other services.

amusement ux
Screenshot of the Extreme Engineering homepage, designed by Extreme Engineering

Sometimes, amusement sites with a great user experience are not made by UX practitioners. The website for Extreme Engineering has a very strong, immersive visual design which communicates their brand well.  When I contacted them to learn who designed their site, I was surprised to learn that their head of marketing had designed it.

What I think is going well

Several developments have encouraging me in my mission to help the amusement industry become more user-centered.

UX methods are producing clear wins for my clients and their users

My clients in the industry so far have had significant fan followings. Fans have seen my user-centered approach, and they have been eager to help me improve their favorite sites. I told a recent client that it would take a week to get enough responses from his site visitors on an OptimalSort study. Within a few hours, we exceeded our target number of responses.

amusement ux
The Explore the Park feature concept of this Busch Gardens Williamsburg

When I worked on redesign concepts for a network of park fansites, I ran separate OptimalSort studies for all 8 fansites in the network. They used comparable pages from each site as cards. We discovered that some parks’ attractions organized well by themed area, while others organized well by ride type. Based on this, we decided to let users find attractions using either way on every site. User testers received Explore the Park and the two other new features that emerged from our studies (Visit Tips and Fansite Community) very well.

Although not all of the features I designed for The Coaster Crew went live, the redesign of their official website produced solid results. Their Facebook likes increased over 50% within a year, and their site improved significantly in several major KPIs. Several site visitors have said that the Coaster Crew’s website’s design helped them choose to join The Coaster Crew instead of another club. So that's a big win.

In-park guest experiences and accessibility are hot right now

IAAPA offered over 80 education sessions for industry professionals at this past Attractions Expo. At least nine sessions discussed guest experience. Guest experience was also mentioned in several industry publications I picked up at the show, including one which interviewed The Experience Economy author B. Joseph Pine II.

And parks are following through on this commitment, even for non-riders. Parks are increasingly theming attractions in ways that allow non-riders to experience a ride’s theme in the ride’s environment. For example, Manta at SeaWorld Orlando is a flying roller coaster themed to a manta ray. The park realized that not all of its guests will want to ride a thrilling ride with four inversions. So, separate queues allow both riders and non-riders to see aquariums with around 3,000 sea creatures. And Manta becomes, effectively, a walkthrough attraction for guests who do not want to ride the roller coaster.

amuse7

The industry has also had encouraging innovations recently in accessibility. Attractions Management Magazine recently featured Morgan’s Wonderland, an amusement park geared toward people with physical and cognitive disabilities. Water parks are beginning to set aside times to especially cater to guests with autism. And at IAAPA, ride manufacturer Zamperla donated a fully-accessible ride to Give Kids the World, an amusement-industry charity.

Several successful consultancies are helping amusement parks and attractions deliver both a better guest experience in the park and better results on business metrics. And that's something I'm definitely keen to be a part of.

Amusement business factors with UX implications

While this is not an exhaustive list, here are some factors in the amusement industry which have UX implications.

Investment in improvements

To keep guests interested in returning, parks reinvest between 5-10% of their revenue into improvements. Park chains typically allocate one capital improvements budget — for new attractions and any other kind of improvement — throughout their entire chain. Reserving enough capital for guest experience improvements is a challenge, even when each touchpoint in a guest’s experience has make-or-break importance. Progress in this area has been slow.

A shift in focus away from high thrills

While ride manufacturers continue to innovate, they are starting to encounter limits on how much physical thrill the human body can handle. A new world’s tallest complete-circuit roller coaster should open in 2017. But that record, only broken one other time since 2003, was broken 5 times between 1994 and 2003. So the industry is shifting toward more immersive attractions and "psychological thriller" rides.

Increased reliance on intellectual property

While some parks still develop their own worlds and characters for attractions, parks today increasingly rely on third-party intellectual property (IP), such as movies, TV shows, and characters. Third-party IP provides guests with a frame of references for interpreting what they see in the park. For example, The Wizarding World of Harry Potter enjoyed a very positive reception from guests due to its faithfulness to the Harry Potter books and films. In the same way, fans will notice if a themed area or attraction is not faithful to the original, and will see it as a broken experience.

Bring your own device

Many guests now carry mobile devices with them in the parks. But so far, guests have not been able to use their mobile devices to trigger changes in a park’s environment. The closest this has come is the interactive wands in The Wizarding World of Harry Potter. Most parks ban mobile devices from most rides due to safety hazards. And amusement parks and museums are both beginning to ban selfie sticks.

Multiple target markets at the same time

Because parks look for gaps in their current offerings and customer bases, they very rarely add new attractions for similar audiences several years in a row. My regional parks tend to handle additions on a 5-10 year cycle. They alternate year by year with additions like a major roller coaster, one or more thrilling flat rides, a family ride, and at least one water ride, to appeal to different market segments, as regularly as possible.The same goes for in-park UX improvements. Themed environment upgrades in a kids’ area appeal to few people in haunted attractions’ target audiences, and vice versa.

Empathy

According to the Association of Zoos and Aquariums (AZA), the fact that most people today will never see sharks, elephants, or pandas in the wild is making conservation efforts more difficult. Zoos and aquaria, in particular, have a large opportunity to allow families to empathize with animals and efforts to conserve threatened species. But, according to an International Zoo Educators Association presentation, most zoo and aquarium guests go there primarily just to see the animals or please their kids.

Ideas for improving the user experience of websites and software in the amusement industry

My primary work interest is to design digital experiences in the amusement industry that support the goals of their target users. To understand where the industry currently stands, I have visited several thousand websites for amusement parks, ride companies, suppliers, zoos, aquaria, museums, and dolphinariums. Below, I've documented a few problems I've seen, and suggested ways companies can solve these problems.

Treat mobile as a top priority

Currently, over 500 websites in the industry are on my radar as sites to improve. At least 90 of them are desktop-only websites with no mobile presence. Several of these sites — even for major ride manufacturers — use Flash and cannot be viewed at all on a mobile device.The industry’s business-to-consumer (B2C) organizations, such as parks, realize that a great deal of traffic comes from mobile and that mobile users are more likely to leave a site that is desktop-only.

These organizations recognize the simple fact that going mobile means selling more tickets. IAAPA itself has capitalized on mobile for their trade show attendees for several years by making a quite resourceful mobile app available.However, at IAAPA, I asked people from several business-to-business (B2B) companies why their sites were not mobile yet. Several told me that they didn’t consider mobile a high priority and that they might start working on a mobile site “in about a year or so.” Thus, they don't feel a great deal of urgency — and I think it's time they did.

Bring design styles and technologies up to date

Many professional UX designers and web designers are well aware of 1990s-style web design artefacts like misused fonts (mainly Comic Sans and Papyrus), black text on a red background,obviously-tiled backgrounds, guestbooks, and splash screens. But I've seen more than a few live amusement websites that still use each of these.Sites that prompt users to install Flash (on mobile devices) or QuickTime increase users’ interaction cost with the site, because the flow of their tasks has been interrupted.

And as Jakob Nielsen says, "Unless everything works perfectly, the novice user will have very little chance of recovery."

If a website needs to use technologies such as Flash or features such as animation or video, a more effective solution would be progressive enhancement. Users whose devices lack the capability to work with these technologies would still see a website with its core features intact and no error messages to distract them from converting.

Organize website information to support user goals and knowledge

Creating an effective website involves much more than using up-to-date design styles. It also involves the following.

Understand why users are on the website

Businesses promote products and services that make them money. Many amusement parks now offer front-of-line passes, VIP tours, pay-per-experience rides, locker rentals, and water park cabana rentals, which are each an additional charge for admitted guests. And per-capita spending is very important to not only parks’ operations, but also their investor relations.Users come to websites with the question, “What’s in it for me?”, and their own sets of goals. Businesses need to know when these goals match their own goals and when they conflict.

The importance of each goal should also be apparent in the design. One water park promotes its changing rooms and locker rentals on its homepage. This valuable space dedicated to logistical information for guests already coming could be used for attracting prospective visitors. Analytics tools and the search queries that they show are helpful tools for understanding why users come to a website. Sites should supplement these by conducting usability studies with users outside their organization. These studies, in turn, could include questions allowing users to describe why would visit that website. The site could use this knowledge to make sure that its content speaks to users’ reasons for visiting.

Understand what users know

Non-technical users are bringing familiarity with how to use the internet when they visit a website. They quickly become perpetual intermediates on the internet, and don’t need to be told how buttons and links work. I've noticed amusement websites that currently label calls to action with “Click Here”, and some even do so on more than one link. This explicit instruction to people is no longer needed, and the best interfaces signal clickable elements in their visual design.

Similarly, people expect to find information in categories they understand and in language familiar to them. So it's important to not make assumptions that people who visit our websites think like us. For example, if a website is organized by model name, users will need to already be familiar with these products and the differences between them. Usability testing exercises, such as card sorting, would contribute to a better design and solve this problem.

Design websites that are consistent with users’ expectations

Websites that aim to showcase a company’s creativity and sense of fun sometimes lack features that people are used to when they visit websites (like easy-to-access menus, vertical scrolling, and so on). But it's important to remember that unconventional designs may lead to increased effort for visitors, which in turn may create a negative experience. A desire to come across as fun may conflict with a visitor's need for ease and simplicity.

The site organization needs to reflect users' goals with minimal barriers to entry. People will be frustrated with things like needing to log in to see prices, having to navigate three levels deep to buy tickets, and coming across unfamiliar or contradictory terms.

Design content for reading

Marketers have written about increased engagement and other benefits resulting from automatically playing videos, animated advertising, and rotating sliders or carousels (all of which usability practitioners have argued against). And this obsession with visual media has sometimes taken attention away from a feature people still want: easy-to-read text.There are still websites in the industry that show walls of text, rivers of text, very small text for main content, text embedded within images, and text written in all-capital letters. But telling clients, “Make the text bigger, higher-contrast, and sentence case”, can conflict with the increasing reliance on exciting visual and interactive design elements.

Goal-directed design shows us what that problem is.  For example, if people visit a website to learn more about a company, the website’s design should emphasize the content that gets that message across — in the format that users find most convenient.I recently worked with a leading themed entertainment blogger to improve his site’s usability. His site, Theme Park University, provides deep knowledge of the themed entertainment industry that readers cannot get anywhere else. It first came to my attention when he published a series of posts on why Hard Rock Park in Myrtle Beach, South Carolina – one of the most ambitious new theme park projects in the US in the 2000s — failed after only one season.

As a regular reader, I knew that TPU had truly fantastic content and an engaged community on social media.  But users commented to us that the site was cluttered, and so didn't spend much time on the website. We needed to give the website a more open look and feel — in other words, designed to be read.The project was not a full redesign. But by fitting small changes into the site’s existing design, we made the site more open and easier to read while retaining its familiar branding for readers.  We also made his site’s advertising more effective, even by having fewer ads on each page.As D. Bnonn Tennant says, “Readership = Revenue.… [A website] has to fulfill a revenue goal. So, every element should be designed to achieve that goal. Including the copy. Especially the copy — because the copy is what convinces visitors to do whatever it is you want them to do on the website".

Three final ideas for getting UX a seat at the table

In helping take UX methods to the amusement industry, I have learned several lessons which would help other practitioners pioneer UX in other industries:

1. Explain UX benefits without UX jargon

When I work for clients in the amusement industry, the biggest challenge that I face is unfamiliarity with UX.  Most other professionals in the industry do not know about UX design principles or practices. I have had to educate clients on the importance of giving me feedback early and testing with users often. And because most of my clients have not had a technical background, I have had to explain UX and its benefits in non-UX terms.

2. Be willing to do non-UX project work yourself in a team of one

As a business owner, I regularly prospect for new clients. The biggest challenge in landing projects here is trying to convince people that they should hire a bigger team than just me. A bigger team (and higher rates for a UXer versus a web designer) leads to bigger project costs and more reluctant approvals. So I have had to do development – and even some tech support – myself so far.

3. Realize clients have a lot on their plates — so learn patience

The other main challenge in selling UX to the amusement industry is project priority. Marketing departments that handle websites are used to seeing the website as one job duty out of many. Ride companies without dedicated IT staff tend to see the website as an afterthought, partially because they do most of their business at trade shows instead of online. This has led to several prospects telling me that they might pursue a redesign a year from now or later, but not in the near future. That's OK because I can be ready for them when they're ready for me.

Learn more
1 min read

6 things to consider when setting up a research practice

With UX research so closely tied to product success, setting up a dedicated research practice is fast becoming important for many organizations. It’s not an easy process, especially for organizations that have had little to do with research, but the end goal is worth the effort.

But where exactly are you supposed to start? This article provides 6 key things to keep in mind when setting up a research practice, and should hopefully ensure you’ve considered all of the relevant factors.

1) Work out what your organization needs

The first and most simple step is to take stock of the current user research situation within the organization. How much research is currently being done? Which teams or individuals are talking to customers on an ongoing basis? Consider if there are any major pain points with the current way research is being carried out or bottlenecks in getting research insights to the people that need them. If research isn't being practiced, identify teams or individuals that don't currently have access to the resources they need, and consider ways to make insights available to the people that need them.

2) Consolidate your insights

UX research should be communicating with nearly every part of an organization, from design teams to customer support, engineering departments and C-level management. The insights that stem from user research are valuable everywhere. Of course, the opposite is also true: insights from support and sales are useful for understanding customers and how the current product is meeting people's needs.

When setting up a research practice, identify which teams you should align with, and then reach out. Sit down with these teams and explore how you can help each other. For your part, you’ll probably need to explain the what and why of user research within the context of your organization, and possibly even explain at a basic level some of the techniques you use and the data you can obtain.

Then, get in touch with other teams with the goal of learning from them. A good research practice needs a strong connection to other parts of the business with the express purpose of learning. For example, by working with your organization’s customer support team, you’ll have a direct line to some of the issues that customers deal with on a regular basis. A good working relationship here means they’ll likely feed these insights back to you, in order to help you frame your research projects.

By working with your sales team, they’ll be able to share issues prospective customers are dealing with. You can follow up on this information with research, the results of which can be fed into the development of your organization’s products.

It can also be fruitful to develop an insights repository, where researchers can store any useful insights and log research activities. This means that sales, customer support and other interested parties can access the results of your research whenever they need to.

When your research practice is tightly integrated other key areas of the business, the organization is likely to see innumerable benefits from the insights>product loop.

3) Figure out which tools you will use

By now you’ve hopefully got an idea of how your research practice will fit into the wider organization – now it’s time to look at the ways in which you’ll do your research. We’re talking, of course, about research methods and testing tools.

We won’t get into every different type of method here (there are plenty of other articles and guides for that), but we will touch on the importance of qualitative and quantitative methods. If you haven’t come across these terms before, here’s a quick breakdown:

  • Qualitative research – Focused on exploration. It’s about discovering things we cannot measure with numbers, and often involves speaking with users through observation or user interviews.
  • Quantitative research – Focused on measurement. It’s all about gathering data and then turning this data into usable statistics.

All user research methods are designed to deliver either qualitative or quantitative data, and as part of your research practice, you should ensure that you always try to gather both types. By using this approach, you’re able to generate a clearer overall picture of whatever it is you’re researching.

Next comes the software. A solid stack of user research testing tools will help you to put research methods into practice, whether for the purposes of card sorting, carrying out more effective user interviews or running a tree test.

There are myriad tools available now, and it can be difficult to separate the useful software from the chaff. Here’s a list of research and productivity tools that we recommend.

Tools for research

Here’s a collection of research tools that can help you gather qualitative and quantitative data, using a number of methods.

  • Treejack – Tree testing can show you where people get lost on your website, and help you take the guesswork out of information architecture decisions. Like OptimalSort, Treejack makes it easy to sort through information and pairs this with in-depth analysis features.
  • dScout – Imagine being able to get video snippets of your users as they answer questions about your product. That’s dScout. It’s a video research platform that collects in-context “moments” from a network of global participants, who answer your questions either by video or through photos.
  • Ethnio – Like dScout, this is another tool designed to capture information directly from your users. It works by showing an intercept pop-up to people who land on your website. Then, once they agree, it runs through some form of research.
  • OptimalSort – Card sorting allows you to get perspective on whatever it is you’re sorting and understand how people organize information. OptimalSort makes it easier and faster to sort through information, and you can access powerful analysis features.
  • Reframer – Taking notes during user interviews and usability tests can be quite time-consuming, especially when it comes to analyze the data. Reframer gives individuals and teams a single tool to store all of their notes, along with a set of powerful analysis features to make sense of their data.
  • Chalkmark – First-click testing can show you what people click on first in a user interface when they’re asked to complete a task. This is useful, as when people get their first click correct, they’re much more likely to complete their task. Chalkmark makes the process of setting up and running a first-click test easy. What’s more, you’re given comprehensive analysis tools, including a click heatmap.

Tools for productivity

These tools aren’t necessarily designed for user research, but can provide vital links in the process.

  • Whimsical – A fantastic tool for user journeys, flow charts and any other sort of diagram. It also solves one of the biggest problems with online whiteboards – finicky object placement.
  • Descript – Easily transcribe your interview and usability test audio recordings into text.
  • Google Slides – When it inevitably comes time to present your research findings to stakeholders, use Google Slides to create readable, clear presentations.

4) Figure out how you’ll track findings over time

With some idea of the research methods and testing tools you’ll be using to collect data, now it’s time to think about how you’ll manage all of this information. A carefully ordered spreadsheet and folder system can work – but only to an extent. Dedicated software is a much better choice, especially given that you can scale these systems much more easily.

A dedicated home for your research data serves a few distinct purposes. There’s the obvious benefit of being able to access all of your findings whenever you need them, which means it’s much easier to create personas if the need arises. A dedicated home also means your findings will remain accessible and useful well into the future.

When it comes to software, Reframer stands as one of the better options for creating a detailed customer insights repository as you’re able to capture your sessions directly in the tool and then apply tags afterwards. You can then easily review all of your observations and findings using the filtering options. Oh, and there’s obviously the analysis side of the tool as well.

If you’re looking for a way to store high-level findings – perhaps if you’re intending to share this data with other parts of your organization – then a tool like Confluence or Notion is a good option. These tools are basically wikis, and include capable search and navigation options too.

5) Where will you get participants from?

A pool of participants you can draw from for your user research is another important part of setting up a research practice. Whenever you need to run a study, you’ll have real people you can call on to test, ask questions and get feedback from.

This is where you’ll need to partner other teams, likely sales and customer support. They’ll have direct access to your customers, so make sure to build a strong relationship with these teams. If you haven’t made introductions, it can helpful to put together a one-page sheet of information explaining what UX research is and the benefits of working with your team.

You may also want to consider getting in some external help. Participant recruitment services are a great way to offload the heavy lifting of sourcing quality participants – often one of the hardest parts of the research process.

6) Work out how you'll communicate your research

Perhaps one of the most important parts of being a user researcher is taking the findings you uncover and communicating them back to the wider organization. By feeding insights back to product, sales and customer support teams, you’ll form an effective link between your organization’s customers and your organization. The benefits here are obvious. Product teams can build products that actually address customer pain points, and sales and support teams will better understand the needs and expectations of customers.

Of course, it isn’t easy to communicate findings. Here are a few tips:

  • Document your research activities: With a clear record of your research, you’ll find it easier to pull out relevant findings and communicate these to the right teams.
  • Decide who needs what: You’ll probably find that certain roles (like managers) will be best served by a high-level overview of your research activities (think a one-page summary), while engineers, developers and designers will want more detailed research findings.

Continue reading

Learn more
1 min read

Moderated vs unmoderated research: which approach is best?

Knowing and understanding why and how your users use your product is invaluable for getting to the nitty gritty of usability. Delving deep with probing questions into motivation or skimming over looking for issues can equally be informative. 

Put super simply, usability testing literally is testing how usable your product is for your users. If your product isn’t usable users often won’t complete their task, let alone come back for more. No one wants to lose users before they even get started. Usability testing gets under their skin and really into the how, why and what they want (and equally what they don’t).

As we have been getting used to video calling regularly and using the internet for interactions, usability testing has followed suit. Being able to access participants remotely has allowed us to diversify the participant pool by not being restricted to those that are close enough to be in-person. This has also allowed an increase in the number of participants per test, as it becomes more cost-effective to perform remote usability testing.

But if we’re remote, does this mean it can’t be moderated? No - remote testing, along with modern technology, can mean that remote testing can be facilitated and moderated. But what is the best method - moderated or unmoderated?

What is moderated remote research testing?

In traditional usability testing, moderated research is done in person. With the moderator and the participant in the same physical space. This, of course, allows for conversation and observational behavioral monitoring. Meaning the moderator can note not only what the participant answers but how and even make note of the body language, surroundings, and other influencing factors. 

This has also meant that traditionally, the participant pool has been limited to those that can be available (and close enough) to make it into a facility for testing. And being in person has meant it takes time (and money) to perform these tests.

As technology has moved along and the speed of internet connections and video calling has increased, this has opened up a world of opportunities for usability testing. Allowing usability testing to be done remotely. Moderators can now set up testing remotely and ‘dial in’ to observe participants anywhere they are. And potentially even running focus groups or other testing in a group format across the internet. 

Pros of moderated remote research testing:

- In-depth gathering of insights through a back-and-forth conversation and observing of the participants.

- Follow-up questions don’t underestimate the value of being available to ask questions throughout the testing. And following up in the moment.

- Observational monitoring noticing and noting the environment and how the participants are behaving, can give more insight into how or why they choose to make a decision.

- Quick remote testing can be quicker to start, find participants, and complete than in-person. This is because you only need to set up a time to connect via the internet, rather than coordinating travel times, etc.

- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing. This broadens your ability to broaden the pool, and participants can be either within your country or global. 

Cons of moderated remote research testing:

- Time-consuming having to be present at each test takes time. As does analyzing the data and insights generated. But remember, this is quality data.

- Limited interactions with any remote testing there is only so much you can observe or understand across the window of a computer screen. It can be difficult to have a grasp on all the factors that might be influencing your participants.

What is unmoderated remote research testing?

In its most simple sense, unmoderated user testing removes the ‘moderated’ part of the equation. Instead of having a facilitator guide participants through the test, participants are left to complete the testing by themselves and in their own time. For the most part, everything else stays the same. 

Removing the moderator, means that there isn’t anyone to respond to queries or issues in the moment. This can either delay, influence, or even potentially force participants to not complete or maybe not be as engaged as you may like. Unmoderated research testing suits a very simple and direct type of test. With clear instructions and no room for inference. 

Pros of unmoderated remote research testing:

- Speed and turnaround,  as there is no need to schedule meetings with each and every participant. Unmoderated usability testing is usually much faster to initiate and complete.

- Size of study (participant numbers) unmoderated usability testing allows you to collect feedback from dozens or even hundreds of users at the same time. 


- Location (local and/or international) Testing online removes reliance on participants being physically present for the testing, which broadens your participant pool.  And unmoderated testing means that it literally can be anywhere while participants complete the test in their own time.

Cons of unmoderated remote research testing:

- Follow-up questions as your participants are working on their own and in their own time, you can’t facilitate and ask questions in the moment. You may be able to ask limited follow-up questions.

- Products need to be simple to use unmoderated testing does not allow for prototypes or any product or site that needs guidance. 

- Low participant support without the moderator any issues with the test or the product can’t be picked up immediately and could influence the output of the test.

When should you do moderated vs unmoderated remote usability testing?

Each moderated and unmoderated remote usability testing have its use and place in user research. It really depends on the question you are asking and what you are wanting to know.

Moderated testing allows you to gather in-depth insights, follow up with questions, and engage the participants in the moment. The facilitator has the ability to guide participants to what they want to know, to dig deeper, or even ask why at certain points. This method doesn’t need as much careful setup as the participants aren’t on their own. While this is all done online, it does still allow connection and conversation. This method allows for more investigative research. Looking at why users might prefer one prototype to another. Or possibly tree testing a new website navigation to understand where they might get lost and querying why the participant made certain choices.

Unmoderated testing, on the other hand, is literally leaving the participants to it. This method needs very careful planning and explaining upfront. The test needs to be able to be set and run without a moderator. This lends itself more to wanting to know a direct answer to a query. Such as a card sort on a website to understand how your users might sort information. Or a first click to see how/where users will click on a new website.

Planning your next user test? Here’s how to choose the right method

With the ability to expand our pool of participants across the globe with all of the advances (and acceptance of) technology and video calling etc, the ability to expand our understanding of users’ experiences is growing. Remote usability testing is a great option when you want to gather information from users in the real world. Depending on your query, moderated or unmoderated usability testing will suit your study. As with all user testing, being prepared and planning ahead will allow you to make the most of your test.

Seeing is believing

Explore our tools and see how Optimal makes gathering insights simple, powerful, and impactful.