Optimal Blog
Articles and Podcasts on Customer Service, AI and Automation, Product, and more

In our Value of UX Research report, nearly 70% of participants identified analysis and synthesis as the area where AI could make the biggest impact.
At Optimal, we're all about cutting the busywork so you can spend more time on meaningful insights and action. That’s why we’ve built automated Insights, powered by AI, to instantly surface key themes from your survey responses.
No extra tools. No manual review. Just faster insights to help you make quicker, data-backed decisions.
What You’ll Get with Automated Insights
- Instant insight discovery
Spot patterns instantly across hundreds of responses without reading every single one. Get insights served up with zero manual digging or theme-hunting. - Insights grounded in real participant responses
We show the numbers behind every key takeaway, including percentage and participant count, so you know exactly what’s driving each insight. And when participants say it best, we pull out their quotes to bring the insights to life. - Zoom in for full context
Want to know more? Easily drill down to the exact participants behind each insight for open text responses, so you can verify, understand nuances, and make informed decisions with confidence. - Segment-specific insights
Apply any segment to your data and instantly uncover what matters most to that group. Whether you’re exploring by persona, demographic, or behavior, the themes adapt accordingly. - Available across the board
From survey questions to pre- and post-study, and post-task questions, you’ll automatically get Insights across all question types, including open text questions, matrix, ranking, and more.
Automate the Busywork, Focus on the Breakthroughs
Automated Insights are just one part of our ever-growing AI toolkit at Optimal. We're making it easier (and faster) to go from raw data to real impact, such as our AI Simplify tool to help you write better survey questions, effortlessly. Our AI assistant suggests clearer, more effective wording to help you engage participants and get higher-quality data.
Ready to level up your UX research? Log into your account to get started with these newest capabilities or sign up for a free trial to experience them for yourselves.
Topics
Research Methods
Popular
All topics
Latest
"I need to upsell an item for charity during the online checkout process..."
"Dear Optimal WorkshopI need to add an upsell item for a charity group into the checkout process. This will include a 'Would you like to add this item?" question, and a tick box that validates the action in one click. Which step of the checkout process do you think this would be best on? Thanks in advance."— Mary
Dear MaryAbout a month ago, I found myself with some time to kill in Brisbane airport (Australia) before my flight home. I wandered on into a stationery store and it was seriously gorgeous, believe me. Its brick-walled interior had an astroturf floor and a back corner filled with design books — I could've spent hours in there.I selected a journal that facilitates daily ranting through guided questions, and made my way to the counter to pay.
Just as I was about to part with my hard-earned dollars, the sales assistant offered me a charity upsell in the form of a 600ml bottle of water for $2 (AUD). Now, I don’t know how familiar you are with Australian domestic airports, but a bottle of water bought from the airport costing less than an absurdly expensive $5 is something I’d written off as a unicorn!Yes, that’s right.
$5 for WATER is considered normal, and this nice man was offering it to me for $2, in a coloured bottle of my choice, complete with that feel-good feeling of giving to charity! I left the store feeling pretty proud of myself and silently had a giggle at people who were buying bottled water elsewhere.
Getting the balance right
Charity upselling at the check out can be a tricky thing. If we get it wrong, we not only fail to raise money for the charity, but we also risk annoying our customers at a moment we want them to be particularly happy with us. The experience I had at the airport is one that I would describe as near perfect. Not all experiences are as positive. It falls down when people start feeling pressured or tricked; when it turns into an ambush.I like the approach you’re looking at.
It’s non-threatening and seamless for the user. So, where should you position it in the checkout?Online checkout is a process like any other: it follows a uniform, often-linear order, and each step involves an action that moves things forward.From my vast online shopping experience (and I do mean vast), I've observed that the generic checkout process looks something like this:
- Add item to cart
- View cart (option to skip and proceed straight to checkout is also common)
- Proceed to checkout
- Sign in / Create account / Guest checkout
- Enter billing address
- Enter / Confirm shipping address
- Enter payment details
- Review purchase
- Make payment
- Confirmation / Payment decline screen
There are two ways that I would consider approaching this and it largely depends on the actual charity upsell item itself.
If you're offering a product to purchase for charity, offer early
If we're talking about the equivalent of my water bottle (offering a product), then I say introduce it earlyin the checkout process (either Step 1 or 2). Why? Because the customer is still in buying mode.
Once they click ‘Proceed to checkout’, they transition and theirfocus shifts tothe business end of their purchase. The stationery store at Brisbane Airport offered the water before telling me how much I owed them. I hadn't quite got to the real world version of checkout mode, which for me was Where did I put my debit card? mode. I quite like the way Oxfam Australia handles the charity upsell by including an 'Add to your gift' button (screen 1) which takes you to the charity upsell option (screen 2) and guides you back to the previous screen, all before checkout.


If you're asking for a donation to charity, ask later
Now, if we'retalking about a donation, such as rounding the purchase price up to the nearest dollar, or asking straight out, it’s a slightly different story. I'd say addthe charity upsell option when they first review their whole intended purchase. It might be before confirming the shipping address, or even just before confirming payment details. They've got money on the brain, and they haven't quite sealed the deal. And it can only be good for the charity if the customer can easily see how small the requested donation is compared to their entire purchase (ahh, the art of persuasion...).
Once they start typing in their payment details, they're essentially signing on the dotted line and entering into a contract with you. So there's no asking after that.David Moth published an interesting article discussing upselling and the online checkout process that's worth a look, so do check it out.In the end, this is something that you'll still want to testwith users, and ultimately it will be up to them. If you have scope, try out a few different options and see which results in more sales. Hopefully this post has given you a good place to start.
All the best Mary!
Are small links more attractive to people as icons or text?
"Dear Optimal Workshop
How do you make a small link attractive to people (icon vs. text)?"
— Cassie
Dear Cassie,
I'm going to dive straight into this interesting question with a good old game of Pros and Cons, and then offer a resolution of sorts, with a meandering thought or two along the way. Let's kick things off with Team Icon.
The good side of icons: A picture is worth a 1000 words
When shopping online, the number above the little shopping trolley icon tells me how badly behaved I’ve been, and if I click on it, I know I’ll get to gleefully review all the shoes I've selected so far. There’s a whole heap of icons out there like this that people have absorbed and can use without thinking twice. Marli Mesibov wrote a fantastic article on the use of icons for UX Booth on the use of icons that is well worth a look. Marli discusses how they work well on small screens, which is a definite bonus when you’re on the go! Young children who aren’t yet literate can easily figure out how to open and play Angry Birds on their parent’s smartphones thanks to icons. And icons also have a great capacity for bridging language barriers.
The not so good side of icons: We’re too old for guessing games
On the flipside, there are some issues that may huff and puff and blow that cute little home icon down. Starting with there being no consistent standard for them. Sure, there are a handful that are universal like home and print, but beyond that it seems to be a free-for-all. Icons are very much in the hands of the designer and this leaves a lot of room for confusion to grow like bacteria in a badly maintained office refrigerator. Difficult to understand icons can also seriously hinder a user’s ability to learn how to use your website or application. When icons don't communicate what they intend, well, you can guess what happens. In a great piece advocating for text over icons, Joshua Porter writes about an experience he had:
"I have used this UI now for a week and I still have do a double-take each time I want to navigate. I’m not learning what the icons mean. The folder icon represents 'Projects', which I can usually remember (but I think I remember it because it’s simply the first and default option). The second icon, a factory, is actually a link to the 'Manage' screen, where you manage people and projects. This trips me up every time."
If people can't pick up the meaning of your icons quickly and intuitively, they may just stop trying altogether. And now, over to Team Label.
The good side of text: What you see is what you get
Sometimes language really is the fastest vehicle you've got for delivering a message. If you choose the right words to label your links, you'll leave the user with very little doubt as to what lies beneath. It’s that simple. Carefully-considered and well-written labels can cut through the noise and leave minimal ambiguity in their wake. Quoting Joshua Porter again: "Nothing says 'manage' like 'manage'. In other words, in the battle of clarity between icons and labels, labels always win."
The not so good side of text: Your flat shoe is my ballet pump
Text labels can get messy and be just as confusing as unfamiliar icons! Words and phrases sometimes don’t mean the same thing to different people. One person’s flat enclosed shoe may be another person’s ballet pump, and the next person may be left scratching their head because they thought pumps were heels and all they wanted was a ballet flat! Text only labels can also become problematic if there isn’t a clear hierarchy of information, and if you have multiple links on one page or screen. Bombarding people with a page of short text links may make it difficult for them to find a starting point. And text may also hold back people who speak other languages.
The compromise: Pair icons up with text labels
Because things are always better when we work together! Capitalise on the combined force of text and icons to solve the dilemma. And I don’t mean you should rely on hovers — make both text and icon visible at all times. Two great examples are Google Apps (because nothing says storage like a weird geometric shape...) and the iPhone App store (because the compass and magnifying glass would pose an interesting challenge without text...):

So what comes next? (You can probably guess what I'm going to say)
Whatever you decide to run with, test it. Use whatever techniques you have on hand to test all three possibilities — icons only, text only, and icons and text — on real people. No Pros and Cons list, however wonderful, can beat that. And you know, the results will probably surprise you. I ran a quick study recently using Chalkmark to find out where people on the ASOS women's shoes page would click to get to the homepage (and yes, I can alway find ways to make shoe shopping an integral part of my job). 28 people responded, and...

...a whopping 89% of them clicked the logo, just 7% clicked the home icon, and just one person (the remaining 4%) clicked the label 'Home'. Enough said. Thanks for your question Cassie. To finish, here's some on-topic (and well-earned) comic relief (via @TechnicallyRon)


Card Sorting outside UX: How I use online card sorting for in-person sociological research
Hello, my name is Rick and I’m a sociologist. All together, “Hi, Rick!” Now that we’ve got that out of the way, let me tell you about how I use card sorting in my research. I'll soon be running a series of in-person, moderated card sorting sessions. This article covers why card sorting is an integral part of my research, and how I've designed the study toanswer specific questions about two distinct parts of society.
Card sorting to establish how different people comprehend their worlds
Card sorting,or pile sorting as it’s sometimes called, has a long history in anthropology, psychology and sociology. Anthropologists, in particular, have used it to study how different cultures think about various categories. Researchers in the 1970s conducted card sorts to understand how different cultures categorize things like plants and animals. Sociologists of that era also used card sorts to examine how people think about different professions and careers. And since then, scholars have continued to use card sorts to learn about similar categorization questions.
In my own research, I study how different groups of people in the United States imagine the category of 'religion'. Asthose crazy 1970s anthropologists showed, card sorting is a great way to understand how people cognitively understand particular social categories. So, in particular,I’m using card sorting in my research to better understand how groups of people with dramatically different views understand 'religion' — namely, evangelical Christians and self-identified atheists. Thinkof it like this. Some people say that religion is the bedrock of American society.
Others say that too much religion in public life is exactly what’s wrong with this country. What's not often considered is these two groups oftenunderstand the concept of 'religion' in very different ways. It’s like the group of blind men and the elephant: one touches the trunk, one touches the ears, and one touches the tail. All three come away with very different ideas of what an elephant is. So you could say that I study how different people experience the 'elephant' of religion in their daily lives. I’m doing so using primarily in-person moderated sorts on an iPad, which I’ll describe below.
How I generated the words on the cards
The first step in the process was to generate lists of relevant terms for my subjects to sort. Unlike in UX testing, where cards for sorting might come from an existing website, in my world these concepts first have to be mined from the group of people being studied. So the first thing I did was have members of both atheist and evangelical groups complete a free listing task. In a free listing task, participants simply list as many words as they can that meet the criteria given. Sets of both atheist and evangelical respondents were given the instructions: "What words best describe 'religion?' Please list as many as you can.” They were then also asked to list words that describe 'atheism', 'spirituality', and 'Christianity'.
I took the lists generated and standardizedthem by combining synonyms. For example, some of my atheists used words like 'ancient', 'antiquated', and 'archaic' to describe religion. SoI combined all of these words into the one that was mentioned most: 'antiquated'. By doing this, I created a list of the most common words each group used to describe each category. Doing this also gave my research another useful dimension, ideal for exploring alongside my card sorting results. Free lists can beanalyzed themselves using statistical techniques likemulti-dimensional scaling, so I used this technique for apreliminary analysis of the words evangelicals used to describe 'atheism':

Now that I’m armed with these lists of words that atheist and evangelicals used to describe religion, atheism etc., I’m about to embark on phase two of the project: the card sort.
Why using card sorting software is a no-brainer for my research
I’ll be conducting my card sorts in person, for various reasons. I have relatively easy access to the specific population that I’m interested in, and for the kind of academic research I’m conducting, in-person activities are preferred. In theory, I could just print the words on some index cards and conduct a manual card sort, but I quickly realized that a software solution would be far preferable, for a bunch of reasons.
First of all, it's important for me to conductinterviews in coffee shops and restaurants, and an iPad on the table is, to put it mildly, more practical than a table covered in cards — no space for the teapot after all.
Second, usingsoftwareeliminates the need for manual data entry on my part. Not only is manual data entry a time consuming process, but it also introduces the possibly of data entry errors which may compromise my research results.
Third, while the bulk of the card sorts are going to be done in person, having an online version will enable meto scale the project up after the initial in-person sorts are complete. The atheist community, in particular, has a significant online presence, making a web solution ideal for additional data collection.
Fourth, OptimalSort gives the option to re-direct respondents after they complete a sort to any webpage, which allows multiple card sorts to be daisy-chained together. It also enables card sorts to be easily combined with complex survey instruments from other providers (e.g. Qualtrics or Survey Monkey), so card sorting data can be gathered in conjunction with other methodologies.
Finally, and just as important, doing card sorts on a tablet is more fun for participants. After all, who doesn’t like to play with an iPad? If respondents enjoy the unique process of the experiment, this is likely to actually improve the quality of the data, andrespondents are more likely to reflect positively on the experience, making recruitment easier. And a fun experience also makes it more likely that respondents will complete the exercise.
What my in-person, on-tablet card sorting research will look like
Respondents will be handed an iPad Air with 4G data capability. While the venues where the card sorts will take place usually have public Wi-Fi networks available, these networks are not always reliable, so the cellular data capabilities are needed as a back-up (and my pre-testing has shown that OptimalSort works on cellular networks too).
The iPad’s screen orientation will be locked to landscape and multi-touch functions will be disabled to prevent respondents from accidentally leaving the testing environment. In addition, respondents will have the option of using a rubber tipped stylus for ease of sorting the cards. While I personally prefer to use a microfiber tipped stylus in other applications, pre-testing revealed that an old fashioned rubber tipped stylus was easier for sorting activities.

When the respondent receives the iPad, the card sort first page with general instructions will already be open on the tablet in the third party browser Perfect Web. A third party browser is necessary because it is best to run OptimalSort locked in a full screen mode, both for aesthetic reasons and to keep the screen simple and uncluttered for respondents. Perfect Web is currently the best choice in the ever shifting app landscape.

I'll give respondents their instructions and then go to another table to give them privacy (because who wants the creepy feeling of some guy hanging over you as you do stuff?). Altogether, respondents will complete two open card sorts and a fewsurvey-style questions, all chained together by redirect URLs. First, they'll sort 30 cards into groups based on how they perceive 'religion', and name the categories they create. Then, they'll complete a similar card sort, this time based on how they perceive 'atheism'.
Both atheist and evangelicals will receive a mixture of some of the top words that both groups generated in the earlier free listing tasks. To finish, they'll answer a few questions that will provide further data on how they think about 'religion'. After I’ve conducted these card sorts with both of my target populations, I’ll analyze the resulting data on its own and also in conjunction with qualitative data I’ve already collected via ethnographic research and in-depth interviews. I can't wait, actually. In a few months I’ll report back and let you know what I’ve found.

Behind the scenes of UX work on Trade Me's CRM system
We love getting stuck into scary, hairy problems to make things better here at Trade Me. One challenge for us in particular is how best to navigate customer reaction to any change we make to the site, the app, the terms and conditions, and so on. Our customers are passionate both about the service we provide — an online auction and marketplace — and its place in their lives, and are rightly forthcoming when they're displeased or frustrated. We therefore rely on our Customer Service (CS) team to give customers a voice, and to respond with patience and skill to customer problems ranging from incorrectly listed items to reports of abusive behavior.
The CS team uses a Customer Relationship Management (CRM) system, Trade Me Admin, to monitor support requests and manage customer accounts. As the spectrum of Trade Me's services and the complexity of the public website have grown rapidly, the CRM system has, to be blunt, been updated in ways which have not always been the prettiest. Links for new tools and reports have simply been added to existing pages, and old tools for services we no longer operate have not always been removed. Thus, our latest focus has been to improve the user experience of the CRM system for our CS team.
And though on the surface it looks like we're working on a product with only 90 internal users, our changes will have flow on effects to tens of thousands of our members at any given time (from a total number of around 3.6 million members).
The challenges of designing customer service systems
We face unique challenges designing customer service systems. Robert Schumacher from GfK summarizes these problems well. I’ve paraphrased him here and added an issue of my own:
1. Customer service centres are high volume environments — Our CS team has thousands of customer interactions every day, and and each team member travels similar paths in the CRM system.
2. Wrong turns are amplified — With so many similar interactions, a system change that adds a minute more to processing customer queries could slow down the whole team and result in delays for customers.
3. Two people relying on the same system — When the CS team takes a phone call from a customer, the CRM system is serving both people: the CS person who is interacting with it, and the caller who directs the interaction. Trouble is, the caller can't see the paths the system is forcing the CS person to take. For example, in a previous job a client’s CS team would always ask callers two or three extra security questions — not to confirm identites, but to cover up the delay between answering the call and the right page loading in the system.
4. Desktop clutter — As a result of the plethora of tools and reports and systems, the desktop of the average CS team member is crowded with open windows and tabs. They have to remember where things are and also how to interact with the different tools and reports, all of which may have been created independently (ie. work differently). This presents quite the cognitive load.
5. CS team members are expert users — They use the system every day, and will all have their own techniques for interacting with it quickly and accurately. They've also probably come up with their own solutions to system problems, which they might be very comfortable with. As Schumacher says, 'A critical mistake is to discount the expert and design for the novice. In contact centers, novices become experts very quickly.'
6. Co-design is risky — Co-design workshops, where the users become the designers, are all the rage, and are usually pretty effective at getting great ideas quickly into systems. But expert users almost always end up regurgitating the system they're familiar with, as they've been trained by repeated use of systems to think in fixed ways.
7. Training is expensive — Complex systems require more training so if your call centre has high churn (ours doesn’t – most staff stick around for years) then you’ll be spending a lot of money. …and the one I’ve added:
8. Powerful does not mean easy to learn — The ‘it must be easy to use and intuitive’ design rationale is often the cause of badly designed CRM systems. Designers mistakenly design something simple when they should be designing something powerful. Powerful is complicated, dense, and often less easy to learn, but once mastered lets staff really motor.
Our project focus
Our improvement of Trade Me Admin is focused on fixing the shattered IA and restructuring the key pages to make them perform even better, bringing them into a new code framework. We're not redesigning the reports, tools, code or even the interaction for most of the reports, as this will be many years of effort. Watching our own staff use Trade Me Admin is like watching someone juggling six or seven things.
The system requires them to visit multiple pages, hold multiple facts in their head, pattern and problem-match across those pages, and follow their professional intuition to get to the heart of a problem. Where the system works well is on some key, densely detailed hub pages. Where it works badly, staff have to navigate click farms with arbitrary link names, have to type across the URL to get to hidden reports, and generally expend more effort on finding the answer than on comprehending the answer.
Groundwork
The first thing that we did was to sit with CS and watch them work and get to know the common actions they perform. The random nature of the IA and the plethora of dead links and superseded reports became apparent. We surveyed teams, providing them with screen printouts and three highlighter pens to colour things as green (use heaps), orange (use sometimes) and red (never use). From this, we were able to immediately remove a lot of noise from the new IA. We also saw that specific teams used certain links but that everyone used a core set. Initially focussing on the core set, we set about understanding the tasks under those links.
The complexity of the job soon became apparent – with a complex system like Trade Me Admin, it is possible to do the same thing in many different ways. Most CRM systems are complex and detailed enough for there to be more than one way to achieve the same end and often, it’s not possible to get a definitive answer, only possible to ‘build a picture’. There’s no one-to-one mapping of task to link. Links were also often arbitrarily named: ‘SQL Lookup’ being an example. The highly-trained user base are dependent on muscle memory in finding these links. This meant that when asked something like: “What and where is the policing enquiry function?”, many couldn’t tell us what or where it was, but when they needed the report it contained they found it straight away.
Sort of difficult
Therefore, it came as little surprise that staff found the subsequent card sort task quite hard. We renamed the links to better describe their associated actions, and of course, they weren't in the same location as in Trade Me Admin. So instead of taking the predicted 20 minutes, the sort was taking upwards of 40 minutes. Not great when staff are supposed to be answering customer enquiries!
We noticed some strong trends in the results, with links clustering around some of the key pages and tasks (like 'member', 'listing', 'review member financials', and so on). The results also confirmed something that we had observed — that there is a strong split between two types of information: emails/tickets/notes and member info/listing info/reports.
We built and tested two IAs

After card sorting, we created two new IAs, and then customized one of the IAs for each of the three CS teams, giving us IAs to test. Each team was then asked to complete two tree tests, with 50% doing one first and 50% doing the other first. At first glance, the results of the tree test were okay — around 61% — but 'Could try harder'. We saw very little overall difference between the success of the two structures, but definitely some differences in task success. And we also came across an interesting quirk in the results.
Closer analysis of the pie charts with an expert in Trade Me Admin showed that some ‘wrong’ answers would give part of the picture required. In some cases so much so that I reclassified answers as ‘correct’ as they were more right than wrong. Typically, in a real world situation, staff might check several reports in order to build a picture. This ambiguous nature is hard to replicate in a tree test which wants definitive yes or no answers. Keeping the tasks both simple to follow and comprehensive proved harder than we expected.
For example, we set a task that asked participants to investigate whether two customers had been bidding on each other's auctions. When we looked at the pietree (see screenshot below), we noticed some participants had clicked on 'Search Members', thinking they needed to locate the customer accounts, when the task had presumed that the customers had already been found. This is a useful insight into writing more comprehensive tasks that we can take with us into our next tests.
What’s clear from analysis is that although it’s possible to provide definitive answers for a typical site’s IAs, for a CRM like Trade Me Admin this is a lot harder. Devising and testing the structure of a CRM has proved a challenge for our highly trained audience, who are used to the current system and naturally find it difficult to see and do things differently. Once we had reclassified some of the answers as ‘correct’ one of the two trees was a clear winner — it had gone from 61% to 69%. The other tree had only improved slightly, from 61% to 63%.
There were still elements with it that were performing sub-optimally in our winning structure, though. Generally, the problems were to do with labelling, where, in some cases, we had attempted to disambiguate those ‘SQL lookup’-type labels but in the process, confused the team. We were left with the dilemma of whether to go with the new labels and make the system initially harder to use for staff but easier to learn for new staff, or stick with the old labels, which are harder to learn. My view is that any new system is going to see an initial performance dip, so we might as well change the labels now and make it better.
The importance of carefully structuring questions in a tree test has been highlighted, particularly in light of the ‘start anywhere/go anywhere’ nature of a CRM. The diffuse but powerful nature of a CRM means that careful consideration of tree test answer options needs to be made, in order to decide ‘how close to 100% correct answer’ you want to get.
Development work has begun so watch this space
It's great to see that our research is influencing the next stage of the CRM system, and we're looking forward to seeing it go live. Of course, our work isn't over— and nor would we want it to be! Alongside the redevelopment of the IA, I've been redesigning the key pages from Trade Me Admin, and continuing to conduct user research, including first click testing using Chalkmark.
This project has been governed by a steadily developing set of design principles, focused on complex CRM systems and the specific needs of their audience. Two of these principles are to reduce navigation and to design for experts, not novices, which means creating dense, detailed pages. It's intense, complex, and rewarding design work, and we'll be exploring this exciting space in more depth in upcoming posts.

"Could I A/B test two content structures with tree testing?!"
"Dear Optimal Worshop
I have two huge content structures I would like to A/B test. Do you think Treejack would be appropriate?"
— Mike
Hi Mike (and excellent question)!
Firstly, yes, Treejack is great for testing more than one content structure. It’s easy to run two separate Treejack studies — even more than two. It’ll help you decide which structure you and your team should run with, and it won’t take you long to set them up.
When you’re creating the two tree tests with your two different content structures, include the same tasks in both tests. Using the same tasks will give an accurate measure of which structure performs best. I’ve done it before and I found that the visual presentation of the results — especially the detailed path analysis pietrees — made it really easy to compare Test A with Test B.
Plus (and this is a big plus), if you need to convince stakeholders or teammates of which structure is the most effective, you can’t go past quantitative data, especially when its presented clearly — it’s hard to argue with hard evidence!
Here’s two example of the kinds of results visualizations you could compare in your A/B test: the pietree, which shows correct and incorrect paths, and where people ended up:

And the overall Task result, which breaks down success and directness scores, and has plenty of information worth comparing between two tests:

Keep in mind that running an A/B tree test will affect how you recruit participants — it may not be the best idea to have the same participants complete both tests in one go. But it’s an easy fix — you could either recruit two different groups from the same demographic, or test one group and have a gap (of at least a day) between the two tests.
I’ve one more quick question: why are your two content structures ‘huge’?
I understand that sometimes these things are unavoidable — you potentially work for a government organization, or a university, and you have to include all of the things. But if not, and if you haven’t already, you could run an open card sort to come up with another structure to test (think of it as an A/B/C test!), and to confirm that the categories you’re proposing work for people.
You could even run a closed card sort to establish which content is more important to people than others (your categories could go from ‘Very important’ to ‘Unimportant’, or ‘Use everyday’ to ‘Never use’, for example). You might be able to make your content structure a bit smaller, and still keep its usefulness. Just a thought... and of course, you could try to get this information from your analytics (if available) but just be cautious of this because of course analytics can only tell you what people did and not what they wanted to do.
All the best Mike!

Designing for delight in the industry of fun: My story and observations
Emotional. Playful. Delightful.These words resonate with user experience (UX) practitioners. We put them in the titles of design books. We build products that move up the design hierarchy of needs, with the goal to go beyond just reliability, usability, and productivity. We want to truly delight the people who use our products.Designing for delight has parallels in the physical world. I see this in restaurants which offer not only delicious food but also an inviting atmosphere; in stores that don’t just sell clothes but also provide superior customer service. Whole industries operate on designing for delight.
The amusement industry has done this for over 500 years. The world’s oldest operating amusement park, Bakken, first opened for guests in 1583 – about 300 years before the first modern roller coaster. Amusement parks experienced a boom in growth in the US in the 1970s. As of November 2014, China had 59 new amusement parks under construction. Today, hundreds of millions of guests each year visit amusement parks throughout the world.I've been fortunate to work in the amusement industry as the owner of a digital UX design company called Thrill & Create. Here is my story of how I got to do this kind of work, and my observations as a UX practitioner in this market.
Making user-centred purveyors of joy
I've been a fan of amusement parks for most of my life. And it’s somewhat hereditary. Much of my family still lives in Central Florida, and several of them have annual passes to Walt Disney World. My mom was a Cast Member at the Magic Kingdom during its opening season.Although I grew up living east of Washington, DC, I spent most of my childhood waiting for our annual trip to an amusement park. I was a different kind of amusement enthusiast: scared to death of heights, loath to ride roller coasters, but so interested in water rides and swimming that my family thought I was a fish. The love for roller coasters would show up much later. But the collection of park maps from our annual trips grew, and in the pre-RollerCoaster Tycoon days, I would sketch designs for amusement parks.
In college, my interest in amusement received a healthy boost from the internet. In the mid-2000s, before fan communities shifted toward Facebook, unofficial websites were quite popular. My home park — a roller coaster enthusiasts’ term for the park that we visit most frequently, not necessarily the closest park to us — had several of these fansites.The fansites would typically last for a year or two and enjoy somewhat of a rivalry with other fansites before their creators would move on to a different hobby and close their sites. The fansites’ forums became an interesting place to share knowledge and learn history about my home park. They also gave us a place to discuss what we would do if we owned the park, to echo rumors we had heard, and start our own rumors. Some quite active forums still exist for this.
Of course, many of us on the forums wanted to be the first to hear a rumor. So we follow the industry blogs, which are typically the first sources of the news. Screamscape has been announcing amusement-industry rumors since the 1990s. And Screamscape and other sites like it announce news not only in the parks but around the industry. Regular Screamscape readers learn about ride manufacturers, trade names for each kind of ride, industry trade shows, and much more. And the International Association of Amusement Parks and Attractions (IAAPA) keeps an eye on the industry as well.Several years ago, I transitioned from being a software developer to starting a user experience design company. It is now called Thrill & Create. I was faced with a challenge: how to compete against the commoditization of freelance design services. Ultimately, selecting a niche was the answer. And seeing IAAPA’s iconic roller coaster sign outside the Orange County Convention Center during a trip to Central Florida was all I needed to shift my strategy toward the amusement industry.
What UX looks like in the Amusement Industry
Periodically, I see new articles about amusement parks in UX blogs. Here are my observations about how UX looks in the industry, both in the physical world and the digital world.
Parks are focused on interactive rides
The amusement industry is known for introducing rides that are bigger, taller, and faster. But the industry has a more interactive future. Using cleverly-designed shops which produce some of the longest waits in the park, Universal Studios has sold many interactive wands to give guests additional experiences in Hogsmeade and Diagon Alley.

At IAAPA, interactivity and interactive rides are very hot topics. Interactive shooting dark rides came to many parks in the early 2000s. Wonder Mountain’s Guardian, a 2014 addition to Canada’s Wonderland, features the world’s longest interactive screen and a ride program that changes completely for the Halloween season.Also coming in 2015 is the elaborately-themed Justice League: Battle for Metropolis rides at two Six Flags parks. And Wet‘n’Wild Las Vegas will debut 'Slideboarding', which allows riders to participate in a video game by touching targets on their way down the slide, and is marketed as 'the world's first waterslide gaming experience'.
UX design is well-established for the physical space
Terms like 'amusement industry', 'attractions industry', and 'themed entertainment industry' can be interchangeable, but they do have different focuses, and different user experience design needs. The amusement industry encompasses amusement parks, theme parks, zoos, aquaria, museums, and their suppliers; the attractions industry also includes other visitor attractions.Several experience design companies have worked extensively on user-centered themed entertainment.
Jack Rouse Associates, who see themselves as “audience advocates”, have worked with over 35 clients in themed entertainment, including Universal, Ocean Park, and LEGOLAND. Thinkwell Group, which touts a 'guest-centric approach to design', showcases 14 theme park and resort projects and attractions work in 12 countries.Consultants in the industry have been more intentionally user-centered. Sasha Bailyn and her team at Entertainment Designer write regularly about physical-world experience design, including UX, in themed entertainment. Russell Essary, owner of Interactive Magic, applies user-centered design to exhibit design, wayfinding, game design, and much more.
In-house teams, agencies, and freelancers are becoming more common
Several large park chains have in-house or contracted UX design teams. Most mid-sized parks work with in-house marketing staff or with outside design companies. One company I know of specializes in web design and development for the amusement industry. Smaller parks and ride companies tend to work with local web designers, or occasionally free website vendors.

Some of the best redesigns in the amusement industry recently have involved UX designers. Parc Astérix, north of Paris, hired a UX designer for a redesign with immersive pictures, interesting shapes, and unique iconography. SeaWorld Parks & Entertainment, working with UX designers, unifies their brand strongly across their corporate site, the sites for SeaWorld and Busch Gardens, and individual park sites. The Memphis Zoo’s website, which showcases videos of their animals, was built by an agency that provides UI design and UX design among their other services.

Sometimes, amusement sites with a great user experience are not made by UX practitioners. The website for Extreme Engineering has a very strong, immersive visual design which communicates their brand well. When I contacted them to learn who designed their site, I was surprised to learn that their head of marketing had designed it.
What I think is going well
Several developments have encouraging me in my mission to help the amusement industry become more user-centered.
UX methods are producing clear wins for my clients and their users
My clients in the industry so far have had significant fan followings. Fans have seen my user-centered approach, and they have been eager to help me improve their favorite sites. I told a recent client that it would take a week to get enough responses from his site visitors on an OptimalSort study. Within a few hours, we exceeded our target number of responses.

When I worked on redesign concepts for a network of park fansites, I ran separate OptimalSort studies for all 8 fansites in the network. They used comparable pages from each site as cards. We discovered that some parks’ attractions organized well by themed area, while others organized well by ride type. Based on this, we decided to let users find attractions using either way on every site. User testers received Explore the Park and the two other new features that emerged from our studies (Visit Tips and Fansite Community) very well.
Although not all of the features I designed for The Coaster Crew went live, the redesign of their official website produced solid results. Their Facebook likes increased over 50% within a year, and their site improved significantly in several major KPIs. Several site visitors have said that the Coaster Crew’s website’s design helped them choose to join The Coaster Crew instead of another club. So that's a big win.
In-park guest experiences and accessibility are hot right now
IAAPA offered over 80 education sessions for industry professionals at this past Attractions Expo. At least nine sessions discussed guest experience. Guest experience was also mentioned in several industry publications I picked up at the show, including one which interviewed The Experience Economy author B. Joseph Pine II.
And parks are following through on this commitment, even for non-riders. Parks are increasingly theming attractions in ways that allow non-riders to experience a ride’s theme in the ride’s environment. For example, Manta at SeaWorld Orlando is a flying roller coaster themed to a manta ray. The park realized that not all of its guests will want to ride a thrilling ride with four inversions. So, separate queues allow both riders and non-riders to see aquariums with around 3,000 sea creatures. And Manta becomes, effectively, a walkthrough attraction for guests who do not want to ride the roller coaster.

The industry has also had encouraging innovations recently in accessibility. Attractions Management Magazine recently featured Morgan’s Wonderland, an amusement park geared toward people with physical and cognitive disabilities. Water parks are beginning to set aside times to especially cater to guests with autism. And at IAAPA, ride manufacturer Zamperla donated a fully-accessible ride to Give Kids the World, an amusement-industry charity.
Several successful consultancies are helping amusement parks and attractions deliver both a better guest experience in the park and better results on business metrics. And that's something I'm definitely keen to be a part of.
Amusement business factors with UX implications
While this is not an exhaustive list, here are some factors in the amusement industry which have UX implications.
Investment in improvements
To keep guests interested in returning, parks reinvest between 5-10% of their revenue into improvements. Park chains typically allocate one capital improvements budget — for new attractions and any other kind of improvement — throughout their entire chain. Reserving enough capital for guest experience improvements is a challenge, even when each touchpoint in a guest’s experience has make-or-break importance. Progress in this area has been slow.
A shift in focus away from high thrills
While ride manufacturers continue to innovate, they are starting to encounter limits on how much physical thrill the human body can handle. A new world’s tallest complete-circuit roller coaster should open in 2017. But that record, only broken one other time since 2003, was broken 5 times between 1994 and 2003. So the industry is shifting toward more immersive attractions and "psychological thriller" rides.
Increased reliance on intellectual property
While some parks still develop their own worlds and characters for attractions, parks today increasingly rely on third-party intellectual property (IP), such as movies, TV shows, and characters. Third-party IP provides guests with a frame of references for interpreting what they see in the park. For example, The Wizarding World of Harry Potter enjoyed a very positive reception from guests due to its faithfulness to the Harry Potter books and films. In the same way, fans will notice if a themed area or attraction is not faithful to the original, and will see it as a broken experience.
Bring your own device
Many guests now carry mobile devices with them in the parks. But so far, guests have not been able to use their mobile devices to trigger changes in a park’s environment. The closest this has come is the interactive wands in The Wizarding World of Harry Potter. Most parks ban mobile devices from most rides due to safety hazards. And amusement parks and museums are both beginning to ban selfie sticks.
Multiple target markets at the same time
Because parks look for gaps in their current offerings and customer bases, they very rarely add new attractions for similar audiences several years in a row. My regional parks tend to handle additions on a 5-10 year cycle. They alternate year by year with additions like a major roller coaster, one or more thrilling flat rides, a family ride, and at least one water ride, to appeal to different market segments, as regularly as possible.The same goes for in-park UX improvements. Themed environment upgrades in a kids’ area appeal to few people in haunted attractions’ target audiences, and vice versa.
Empathy
According to the Association of Zoos and Aquariums (AZA), the fact that most people today will never see sharks, elephants, or pandas in the wild is making conservation efforts more difficult. Zoos and aquaria, in particular, have a large opportunity to allow families to empathize with animals and efforts to conserve threatened species. But, according to an International Zoo Educators Association presentation, most zoo and aquarium guests go there primarily just to see the animals or please their kids.
Ideas for improving the user experience of websites and software in the amusement industry
My primary work interest is to design digital experiences in the amusement industry that support the goals of their target users. To understand where the industry currently stands, I have visited several thousand websites for amusement parks, ride companies, suppliers, zoos, aquaria, museums, and dolphinariums. Below, I've documented a few problems I've seen, and suggested ways companies can solve these problems.
Treat mobile as a top priority
Currently, over 500 websites in the industry are on my radar as sites to improve. At least 90 of them are desktop-only websites with no mobile presence. Several of these sites — even for major ride manufacturers — use Flash and cannot be viewed at all on a mobile device.The industry’s business-to-consumer (B2C) organizations, such as parks, realize that a great deal of traffic comes from mobile and that mobile users are more likely to leave a site that is desktop-only.
These organizations recognize the simple fact that going mobile means selling more tickets. IAAPA itself has capitalized on mobile for their trade show attendees for several years by making a quite resourceful mobile app available.However, at IAAPA, I asked people from several business-to-business (B2B) companies why their sites were not mobile yet. Several told me that they didn’t consider mobile a high priority and that they might start working on a mobile site “in about a year or so.” Thus, they don't feel a great deal of urgency — and I think it's time they did.
Bring design styles and technologies up to date
Many professional UX designers and web designers are well aware of 1990s-style web design artefacts like misused fonts (mainly Comic Sans and Papyrus), black text on a red background,obviously-tiled backgrounds, guestbooks, and splash screens. But I've seen more than a few live amusement websites that still use each of these.Sites that prompt users to install Flash (on mobile devices) or QuickTime increase users’ interaction cost with the site, because the flow of their tasks has been interrupted.
And as Jakob Nielsen says, "Unless everything works perfectly, the novice user will have very little chance of recovery."
If a website needs to use technologies such as Flash or features such as animation or video, a more effective solution would be progressive enhancement. Users whose devices lack the capability to work with these technologies would still see a website with its core features intact and no error messages to distract them from converting.
Organize website information to support user goals and knowledge
Creating an effective website involves much more than using up-to-date design styles. It also involves the following.
Understand why users are on the website
Businesses promote products and services that make them money. Many amusement parks now offer front-of-line passes, VIP tours, pay-per-experience rides, locker rentals, and water park cabana rentals, which are each an additional charge for admitted guests. And per-capita spending is very important to not only parks’ operations, but also their investor relations.Users come to websites with the question, “What’s in it for me?”, and their own sets of goals. Businesses need to know when these goals match their own goals and when they conflict.
The importance of each goal should also be apparent in the design. One water park promotes its changing rooms and locker rentals on its homepage. This valuable space dedicated to logistical information for guests already coming could be used for attracting prospective visitors. Analytics tools and the search queries that they show are helpful tools for understanding why users come to a website. Sites should supplement these by conducting usability studies with users outside their organization. These studies, in turn, could include questions allowing users to describe why would visit that website. The site could use this knowledge to make sure that its content speaks to users’ reasons for visiting.
Understand what users know
Non-technical users are bringing familiarity with how to use the internet when they visit a website. They quickly become perpetual intermediates on the internet, and don’t need to be told how buttons and links work. I've noticed amusement websites that currently label calls to action with “Click Here”, and some even do so on more than one link. This explicit instruction to people is no longer needed, and the best interfaces signal clickable elements in their visual design.
Similarly, people expect to find information in categories they understand and in language familiar to them. So it's important to not make assumptions that people who visit our websites think like us. For example, if a website is organized by model name, users will need to already be familiar with these products and the differences between them. Usability testing exercises, such as card sorting, would contribute to a better design and solve this problem.
Design websites that are consistent with users’ expectations
Websites that aim to showcase a company’s creativity and sense of fun sometimes lack features that people are used to when they visit websites (like easy-to-access menus, vertical scrolling, and so on). But it's important to remember that unconventional designs may lead to increased effort for visitors, which in turn may create a negative experience. A desire to come across as fun may conflict with a visitor's need for ease and simplicity.
The site organization needs to reflect users' goals with minimal barriers to entry. People will be frustrated with things like needing to log in to see prices, having to navigate three levels deep to buy tickets, and coming across unfamiliar or contradictory terms.
Design content for reading
Marketers have written about increased engagement and other benefits resulting from automatically playing videos, animated advertising, and rotating sliders or carousels (all of which usability practitioners have argued against). And this obsession with visual media has sometimes taken attention away from a feature people still want: easy-to-read text.There are still websites in the industry that show walls of text, rivers of text, very small text for main content, text embedded within images, and text written in all-capital letters. But telling clients, “Make the text bigger, higher-contrast, and sentence case”, can conflict with the increasing reliance on exciting visual and interactive design elements.
Goal-directed design shows us what that problem is. For example, if people visit a website to learn more about a company, the website’s design should emphasize the content that gets that message across — in the format that users find most convenient.I recently worked with a leading themed entertainment blogger to improve his site’s usability. His site, Theme Park University, provides deep knowledge of the themed entertainment industry that readers cannot get anywhere else. It first came to my attention when he published a series of posts on why Hard Rock Park in Myrtle Beach, South Carolina – one of the most ambitious new theme park projects in the US in the 2000s — failed after only one season.
As a regular reader, I knew that TPU had truly fantastic content and an engaged community on social media. But users commented to us that the site was cluttered, and so didn't spend much time on the website. We needed to give the website a more open look and feel — in other words, designed to be read.The project was not a full redesign. But by fitting small changes into the site’s existing design, we made the site more open and easier to read while retaining its familiar branding for readers. We also made his site’s advertising more effective, even by having fewer ads on each page.As D. Bnonn Tennant says, “Readership = Revenue.… [A website] has to fulfill a revenue goal. So, every element should be designed to achieve that goal. Including the copy. Especially the copy — because the copy is what convinces visitors to do whatever it is you want them to do on the website".
Three final ideas for getting UX a seat at the table
In helping take UX methods to the amusement industry, I have learned several lessons which would help other practitioners pioneer UX in other industries:
1. Explain UX benefits without UX jargon
When I work for clients in the amusement industry, the biggest challenge that I face is unfamiliarity with UX. Most other professionals in the industry do not know about UX design principles or practices. I have had to educate clients on the importance of giving me feedback early and testing with users often. And because most of my clients have not had a technical background, I have had to explain UX and its benefits in non-UX terms.
2. Be willing to do non-UX project work yourself in a team of one
As a business owner, I regularly prospect for new clients. The biggest challenge in landing projects here is trying to convince people that they should hire a bigger team than just me. A bigger team (and higher rates for a UXer versus a web designer) leads to bigger project costs and more reluctant approvals. So I have had to do development – and even some tech support – myself so far.
3. Realize clients have a lot on their plates — so learn patience
The other main challenge in selling UX to the amusement industry is project priority. Marketing departments that handle websites are used to seeing the website as one job duty out of many. Ride companies without dedicated IT staff tend to see the website as an afterthought, partially because they do most of their business at trade shows instead of online. This has led to several prospects telling me that they might pursue a redesign a year from now or later, but not in the near future. That's OK because I can be ready for them when they're ready for me.