Testing FAQs with people who don’t use your site

“Questions are never indiscreet, answers sometimes are.”Oscar Wilde

‍

Frequently asked question pages. Love them or hate them, I don’t think they’re going anywhere anytime soon. This debate has been going on for quite some time and there is an equal number of opinions on both sides of the FAQ fence. Nielsen Norman Group’s Susan Farrell says FAQs can still add value to a website when done properly, and Gerry McGovern says FAQs are the dinosaurs of web navigation.

‍

So, how do we really know for sure if they will or won’t add value to a design? Like anything in UX, you have to test it! I don’t know about you, but I’m a shake-it-and-see-what-falls-out kind of UXer, so naturally I decided to run a Treejack study. Scouring the web one fine day, I came across Sainsbury’s Active Kids. Its FAQ page was unlike any I had ever seen and I knew I’d found the one. I was also curious to see how it would test with people who don’t use the website — after all, anyone should be able to use it. Since Active Kids is an active lifestyle program for UK schools and sports clubs, I recruited my participants entirely from the US.

‍

Pull up a chair and get comfy because what I found out should serve as a lesson to us all.

‍

Why Active Kids? 🤸🏼

‍

First of all, why did I choose this in the first place? The Active Kids FAQ page caught my attention for three main reasons:

‍

• structure
• labels
• content

‍

The structure of this FAQs page is quite deep, complex and very different from the rest of the site — almost like another information architecture (IA) had been built within the main structure. Imagine you have a large warehouse with hundreds of shelves, and then somewhere in the middle of it, someone builds a house — that’s how it felt to me.

‍

There are two ways to get to it: through the “Help” label on the top navigation bar and the “FAQ” label in the footer. It also uses a combination of drop-down filters that the user needs to apply, but it also has automatic filter options and confusing labels that can send you down a path you don’t necessarily want to take.

‍

I also found it very interesting that most of the information contained within the FAQs section cannot be located anywhere else on the website and most of this is essential to gaining a fundamental understanding of what Active Kids actually does. Adding to the house in the warehouse analogy, it’s like the house holds all the key information the warehouse needs to function, but no one knows which room it’s kept in.

‍

‍

‍

Setting up the study 🛠️

‍

Treejack was the perfect choice for testing the findability of information on the Active Kids FAQ page and I decided to test the IA of the website as a whole — this means both the warehouse and the house. I couldn’t just test the house in isolation because that’s not how a user would interact with it. The test needed the context of the whole site to gain an understanding of what’s going on. Creating a Treejack study is quick and easy and all you have to do is build the structure out in a basic Excel spreadsheet and then copy and paste it into Treejack.

‍

My next job was to determine the task based scenarios that my participants would use during the study. I decided to choose nine and all were derived from content located in the FAQs section and related to tasks a user might carry out when investigating or participating in the program. Once I had my tree and my tasks, all I had to do was set the correct answers based on where the information currently sits on the Active Kids website and I was ready to launch.

‍

Recruiting participants for the study🙋🏾

‍

In my experience,recruiting participants for a Treejack study is quick and easy. All you have to do is determine the screener criteria for your participants and Optimal Workshop takes care of the rest. For this study I requested 30 participants and they all had to reside in the US. I ended up with 31 completed responses and it was all over in less than two hours.

‍

Treejack results 🌲

‍

So, what fell out of that tree when I tested a website aimed at parents and teachers of kids in the UK with 31 Americans? I’ll be honest with you: it wasn’t pretty. Here’s what I discovered in this study:

‍

‍

• 81 per cent were unable to find out if home educators were eligible to apply (number 1 on the graph)
• 65 per cent were unable to find out what a Clubmark accreditation is (number 2 on the graph)
• 68 per cent were unable to find out how to share their wishlist with friends and family (number 3 on the graph)
• 64 per cent could not find the information that would explain the purpose of the £1 fee mentioned in the terms and conditions (number 4 on the graph)
• 97 per cent could not locate the information that would tell them if they could use a voucher from 2014 in 2015 (number 5 on the graph)
• No participant was able to determine if students from a middle school would be able to participate in Active Kids (number 8 on the graph)
• 58 per cent of participants in this study were unable to find out what the program is even about (number 9 on the graph)

‍

‍

On the flip side, 68 per cent of participants in this study were able to locate a phone number to contact Active Kids directly (number 6 on the graph) and 97 per cent were successfully able to work out how to redeem vouchers (number 7). Overall, it wasn’t great.In addition to some very useful quantitative data, Treejack also provides detailed information on the pathways followed by each participant.

‍

Understanding the journey they took is just as valuable as discovering how many found their way to the correct destination. This additional level of granularity will show you where and when your user is getting lost in your structure and where they went next. It’s also handy for spotting patterns (e.g., multiple participants navigating to the same incorrect response).

‍

I always set my studies to collect responses anonymously and when this occurs, Treejack assigns each participant a numerical identifier to help keep track of their experience without the participant having to share his or her personal details. For task 6, the paths chart below shows that participants numbered eight to 20 were able to navigate directly to the correct answer without deviating from the correct path I defined during setup.

‍

‍

‍

For Task 3 (below) , the story told by the paths was quite different. Participant number five navigated back and forth several times through the structure in their attempt to locate information on how to share a wishlist. After all that effort, they were unable to find the information they needed to complete the task and nominated to contact Active Kids directly. Not only is this a bad experience for the user but it also puts unnecessary pressure on the call centre because the information should be readily available on the website.

‍

‍

‍

Treejack also provides insights into where participants started their journey by recording first click data. Just like Chalkmark, this functionality will tell you if your users are starting out on the right foot from that all important first click.In this study I found it interesting that when looking for information regarding the eligibility of home educators in the Active Kids program, 42 per cent of participants clicked on “Schools & Groups” and 19 per cent clicked on “Parents & Community” for their first click. Only 6 per cent clicked on “Help”, which happens to be the only place this information can be found.

‍

‍

‍

‍

‍

I also found the first click results for Task 9 to be very interesting. When looking for basic information on the program, more than half (52 per cent) of the participants in this study went straight to “Help”. This indicates that, for these participants, none of the other options were going to provide them the information they needed.

‍

‍

‍

‍

What can be learned from this study? 🎓

‍

I mentioned earlier there was a lesson in this for everyone, and rather than dwell on how something tested, it’s time to move on to some lessons learned and constructive ideas for improvement. Based on the results of this Treejack study, here are my top three recommendations for improving the Active Kids website:

‍

Rethink the content housed in the FAQs section

‍

Most of the key information required to master the basics of what Active Kids is all about is housed entirely in the FAQs section. FAQs should not be the only place a user can find out basic information needed to understand the purpose of a product, program or service. I believe this website would benefit from some further thinking around what actually belongs in the FAQs section and what could be surfaced much higher.

‍

Another idea would be to follow the lead of the Government Digital Service and remove the FAQs section altogether — food for thought. Frequently asked questions would not be frequently asked questions if people could actually find the information on your site in the first place. Figure out where the answers to these questions really belong.

‍

If you’re using Treejack, just look at the fails in your results and figure out where people went first. Is there a trend? Is this the right place? Maybe think about putting the answer the user is looking for there instead.

‍

Restructure the FAQs section

‍

If you must have an FAQs section (and believe me I do understand that they don’t just disappear overnight! Just try to keep it as an interim solution only) please consider streamlining the way they are presented to the user. Ditch the filtering and display the list on one page only. Users should not have to drill down through several layers of content and then navigate through each category. For further reading on getting your FAQs straight, this Kissmetrics article is well worth a read.

‍

Review the intent of the website

‍

Looking at the Active Kids website and the results from this study, I feel the intent of this website could use some refining. If we come back to my warehouse and house analogy, the main chunk of the website (the warehouse) seems to be one giant advertisement, while the house (the FAQs) is where the action-oriented stuff lies. The house seems to hold the key information that people need to use the program and I think it could be displayed better. Don’t get me wrong, Active Kids does some incredibly good work for the community and should absolutely shout its achievements from the rooftops, however a sense of balance is required here. I think it’s time for the house and the warehouse to join forces into a solution that offers both rooftop shouting and usable information that facilitates participation.

‍

The value of fresh eyes 👀

‍

This study goes to show that regardless of where you are in your design process, whether that’s at the very beginning or a few years post-implementation, there is value to be gained from testing with a fresh set of eyes. I’m still undecided on which side of the FAQs debate I belong to — I’m going to sit on the fence and stand by the “if in doubt — test it” school of thought.

‍

Further reading:

• "Strategic design for frequently asked questions" - This ebook from Nielsen Norman Group explains how to design world-class FAQs.
• "Infrequently asked questions of FAQs" - An article from R. Stephen Gracey published on A List Apart, discussing FAQs and their place on the web.
• "Anatomy of a website: website architecture" - An article on our blog explaining website architecture and how to test your own structure.

Thanks to an ever-growing digital world, businesses are spoiled for information and data. The more complex the business, the more information there is and the more complicated the business requirements are. But where there are challenges, there is opportunity. That’s where information architecture and data architecture come in.

‍

Information and data architecture both seek to make sense of the plethora of information a business handles. However, the two have different roles to play in the way businesses use, move, maintain, and present data - both to internal and external stakeholders. So what are they and why should businesses take note?

‍

Defining Information Architecture 🗺️

‍

Information architecture is the structure used to organize and label content on websites, mobile applications and other digital environments. Its primary purpose is to enhance user experience by ensuring information is structured in an accessible, usable and relevant way.

‍

Information architecture seeks to understand user needs and goals by analyzing both existing and required information, then building an information framework in a logical and user-friendly way. It deals with three main components:

‍

• Labels: How information is represented
• Navigation: How users make their way through the information
• Search: How users look for information

‍

Whilst this information sits in the background, it’s the layer upon which you build the design of your digital products.

‍

Information architects bring data from file systems and databases to life by building meaningful narratives and stories. Outputs can include site-mapping, information architecture diagrams and content inventories. These outputs are supported by user research techniques such as card sorting, tree testing, user surveys and first-click testing.

‍

Defining Data Architecture 💻

‍

Data architecture bridges the gap between business needs, goals, and system requirements related to data handling. It sets out a framework for managing data assets, the flow of data and the maintenance of data systems. As such, it has a slightly more macro view than information architecture and concerns itself with emerging technologies such as artificial intelligence, machine learning, and blockchain.

‍

Where information architecture centers around the end-user interaction, data architecture centers around practical handling and operation of data processes i.e. collection through to transformation, distribution, and consumption. Because of this, data architecture must take into account the businesses ability to scale operations, integrate with third party systems, support real-time data processes and the reduction of operating costs. Modern data architecture may point to artificial intelligence to tackle some of these challenges.

‍

The Importance of Enterprise Architects in Information and Data Architecture 🏗

‍

Enterprise architects are big-picture people. Data architecture and information architecture both fall within their remit, and they often oversee other data management job specialities within an IT department.

‍

As a leader (and often, visionary) within a business, enterprise architects shoulder the responsibility of ‘mission critical’ projects. As a result, they tend to have several years experience with IT systems, backed by a bachelor’s degree in computer science, IT management, data science or similar. Many will hold a master’s degree and specialty certifications.

‍

The role involves collaborating with senior business leaders, solution-delivery teams and external stakeholders, and requires creative problem solving and excellent communication skills. Therefore, enterprise architects very much steer the ship when it comes to information and data architecture. Combining high-level business strategy with knowledge of ‘the nuts and bolts’ of IT data systems and processes, they command an annual salary in New Zealand between $150,000 and $200,000 per annum..

‍

Continuous improvement within any business that has substantial IT infrastructure calls for serious investment in enterprise architecture.

‍

Designing and Implementing an Effective Information and Data Architecture 𝞹📈🧠📚

‍

Once overarching business goals are aligned with the scope of data and system requirements, information and data architecture design (or redesign) can begin.

‍

Crucial to the design and implementation process is developing an architecture framework. This is a set of guidelines that lays out principles, practices, tools and approaches required to complete the design. It supports system design decisions, assigns key tasks and provides project guidance throughout the design process. The framework essentially aims to unite disparate teams and maintain business and IT alignment.

‍

The choice of architecture design is also critical. It should consider scalability, performance, maintainability and adaptability to emerging technology. Which is why cloud platforms feature so heavily in modern data architecture. Cloud architects will navigate the architecture design and technical requirements of cloud-based delivery models, which offer the solution to those scalability and adaptability challenges. They are responsible for bridging the gaps between complex business problems and solutions in the cloud. Modern data architecture tends to involve some form of cloud delivery component.

‍

Throughout implementation, data and information architects will work closely with designers and engineers until testable architecture is ready. User research and testing will be carried out, and a feedback loop will commence until requirements are met. Users, as always, should be at the center of your digital product.

‍

Summing Up the Complexities of Information and Data Architecture 🧮

‍

Whilst the difference between information and data architecture can appear nuanced on the surface, they hold unique roles when delivering a cohesive, user-friendly digital product.

‍

Think of a sliding scale where business operations sit at one end, and users sit at the other. Data architecture addresses challenges closer to the business: aligning business requirements and goals with how data flows through the system. On the other hand, information architecture addresses the challenges related to how this data is organized and interpreted for the end user.

‍

At the end of the day, both information and data architecture need to work in harmony to satisfy the user and the business.

What is Information Architecture?

‍

Information architecture (IA) is a term used to describe how content or information is organised and arranged. This can relate to a website, a retail store or an app. And you could even consider the way a library is sorted to be IA. 

‍

For the purposes of this we will be focussed on websites. IA is fundamentally important to the success of your website. It determines how your users will access the information and the success of their user experience (UX) whilst on your website. And ultimately if you can keep users on your website long enough to complete their task.

‍

IA can be broken down into 3 main areas to consider when building great user experience:

‍

• Navigation: How people make their way through information (website content)
• Labels: How information is named and represented.
• Search: How people will look for information (keywords, categories)

‍

When put like this it does seem pretty straightforward. Maybe even simple? But these tasks need to be straightforward for your users. Putting thought, time and research in at the front of your design and build can mean an intuitive website is built. But at any point in your websites life cycle it can be of value to test and review. IA is the structure that sits in behind and allows the design to tell the story and the content to be found in an easy way.

‍

Why is Information Architecture important to web design? 🏗️

‍

If you’ve ever tried to use something and thought, “where am I supposed to go next?” or “this doesn’t make any sense,” you are encountering an issue with an information ‍architecture.

The Information Architecture Institute

‍

The way in which your users will use your website depends largely on how the information is presented and organised. By following through the tasks that you expect your users to undertake you can better understand the user experience. If the user can easily flow from point to point, finding what they need in a quick and efficient (and ideally intuitive manner) they are far more likely to stick around. And return when they need to.

‍

The opposite is definitely true also. If users find your website difficult to follow, hard to navigate and get lost or confused. They will not stick around to find out more. They will move on, and swiftly, to your competition. Frustrated, and disengaged. You will find it difficult to win them back.

‍

What does good information architecture look like for my users? 👀

‍

By providing a simple, clear and straightforward path users can stay focussed on their task, removing overwhelm and confusion. How often do you disappear ‘down a rabbit hole’ when on the internet? Confusing paths or overwhelming options may mean users move off on tangents, and become less likely to complete their initial task. Ultimately the best user experience is one that delivers the right information at the right time. Not too slow and not too complicated. 

‍

Always keep in mind that a great IA is:

‍

• Navigation: Always think straightforward, simple and intuitive.  Keep the navigation menu clean, clear and brief. Content and information where it’s expected to be. No point putting dog collars under dog food.
• Labels: Consider how pages, content and information is named. This needs to be direct and simple to understand. If you want people to find your store label the page ‘location’ or ‘find us’ or even ‘find our store’.
• Search: Most of your users will use search as a last resort. They will try to navigate their way through your website before resorting to a search option. Considering carefully the keywords for information that support the search tool. If they have already failed to find the information through your navigation, don’t let them down now.

‍

With all of these lined up in behind great web design, which is clear, bright and attractive. Along with language which appeals to your user.  You are providing a UX that will entice, engage and ultimately keep them on your website and converting.

‍

What does great information architecture look like for my organization?

‍

Great IA goes beyond simply being about your user experience. Your organization can benefit hugely with testing, research and insights put into your website IA. 

‍

With an IA that creates an easy navigable and engaging website your users are less likely to move off to your competitors. You’ve worked hard to get them to your website, through marketing and SEO. Delivering what your user expects and making it easy to find, means they will complete their task, and are far more likely to return. 

‍

By finding what they need quickly, and intuitively, users are more likely to be converted and generate leads or sales. Delivering and answering questions can also reduce the need for support. If you can, your organization's website should answer your users questions, before they complete. This means they are less likely to need to email and /or call for support, reducing overheads and time lag before conversion.

‍

Your organization's reputation is so important, your website may be your only interaction with users. If they have an easy user experience, their questions answered, and are able to complete what they need simply they leave with a great impression of your organization. They are more likely to return and their overall takeaway is that your organization is trustworthy, organised and easy to deal with. The opposite is quite possible with poor IA and design. You get but one chance to grab their attention and keep them. Do it badly and you may never get them back.

‍

Creating great information architecture 👷🏻

‍

User research with OptimalSort

‍

Of course the best website IA is based on your users experience. And there is no better way to get a full understanding of your users than by conducting research. At any point in your website's life cycle it can be beneficial to undertake research such as card sorting. At the beginning stages of your website build is best, but your website should be evolving as your organization does, therefore any time there are shifts in what you do or offer is a great time to revisit your UX and how best to deliver this. OptimalSort tests users on how they intuitively would like to see information sorted on your website. Building IA based on data, rather than assumptions, will mean that content and information can be sorted in a way that truly delivers a simple and intuitive experience.

‍

Maintaining your website with Treejack

‍

With a great IA, based on card sorting user research, your website content needs to be maintained. Tree testing allows you to see where your users are getting lost in your website navigation. And also how they expect to look for key information. The Treejack tool provides real user insights on how your website navigation is working, how it can work better, and ultimately how to fix paths that don’t work. Providing hard data to inform an intuitive IA.

‍

Wrap Up 🌮

‍

So, information architecture is fundamental to your website and how it operates. Want to learn more about information architecture? Take a look at our article, or download 'The Actionable IA Guide'.

‍

Supporting your website with user research can mean you build and design a intuitive website that simply rocks! 

‍

Learn more about card sorting with our 101 guide. And more on tree testing.