Treejack Webinar Downloads
Posted 6 days, 20 hours ago at 8:34 pm. 0 comments
4Q - are you satisfied?
Posted 1 month ago at 4:38 pm. 0 comments
Over the last three months, we have been asking ourselves - 4Q? Aside from the rather unfortunate name, 4Q is simple way to get a gauge on the happiness of your website visitors.
4Q is a free online survey that you can place on your website to ask 4 questions (hence the name). The questions are:
- What are my visitors at my website to do?
- Are they completing what they set out to do?
- If not, why not?
- How satisfied are my visitors?
So how did we do? Well, it appears we’re a bit rubbish at satisfaction.
Our task completion scores appear to be pretty good. Here are the types of tasks that visitors to our website complete, 83% of the time:
As an aside, a drawback of the tool is that these tasks cannot be customised. You have to choose from a long and somewhat random list of tasks.
The obvious questions that arise are - which tasks did people fail on and what were they dissatisfied with? With a little digging you can find out:
It appears that learning about our products is gradually diminishing in success and rated the lowest for satisfaction. It’s clear that we need to do something about that. Then again, we knew that which is why we have been working on a new site - but it’s always good (even if humbling) to get that validation.
Lastly, one of the most helpful things about the tool are the comments. As an example, here are some comments from people who were successful at the task “Learn about our products”:
Here are some comments from people who were unsuccessful at the task “Learn about our products”:
It appears we have a bit of work to do. Thanks 4Q. Sign up for your free account and get started. It’s worth your effort. Just keep in mind the golden rule of user experience - it’s not what they say that matters, it’s what they do.
Lastly, there are some drawbacks. The way the survey interrupts the website visitor is quite disruptive and it is not possible to tailor the questions you ask. The survey itself is also not going to win beauty awards. If you work in an environment where these things matter, this may be a show stopper.
Picking the right tool for the job
Posted 1 month, 3 weeks ago at 3:59 pm. 0 comments
Life as a UX professional is busy. One day you’re trawling through pages of content, then you’re doing wireframes, next you’re user testing and not to mention presenting results to clients.
We built our tools to help our consultants work more effectively. We have been asked about when the best time is to use which tool. So here are our answers. In general it pays to keep in mind that our tools can be used for generative purposes or evaluative purposes.
We have categorised our recommendations into how you may use our tools depending on what you’re doing. They include:
- Information architecture (IA) overhaul
- Intranet
- Navigation
- Redesigns
- A/B split testing
Information architecture overhaul
Primary recommendation/s: Start with OptimalSort then run a Treejack study. In some cases, the order maybe reversed.
When questions loom about how to best organise and structure content, we normally use OptimalSort as a tool to understand how people think about information. Sometimes the process of running the card sort can be as useful as the results themselves.
More recently we have also been using Treejack as a way to challenge whether the existing IA (if there is one) really needs to change. A quick Treejack survey will provide empirical evidence on whether people are truly struggling to find information. If participants perform well, the issue may not be a bad IA but perhaps bad user interface or interaction design. Go easy on the IA. It’s not always the culprit.
OptimalSort helps you facilitate a ground zero understanding of how people think about the information that you’re responsible for categorising. If you already have candidate structures (or it has been decided for you), your best bet is to skip OptimalSort and do several rounds of tree testing instead with Treejack.
Intranet
Primary recommendation/s: Treejack and Chalkmark
Intranets more than most websites stand to benefit tremendously from better information architecture. However, it is common to be faced with numerous constraints - a difficult content management system (or no/too many content management systems!), unchangeable mandates from certain business units, content owners writing for print.. and the list goes on.
Obviously what we would recommend depends what you’re working on within the intranet. Refer to the relevant recommendation here - eg, Navigation or IA overhaul. Typically however, the best bang for buck appears to come from involving as many people as possible. With this objective, Treejack and Chalkmark are great because it is easy for people to take part, the analysis is simple and we give you good head start with reporting the results.
Pick from any of these suggestions:
- Run a Treejack survey on the current intranet information architecture as a baseline for performance and link that to productivity. Eg, if you score 60% overall, what does the 40% failure cost the business in lost time?
- Make that change that business unit x really wants but before you deploy it, run a Chalkmark survey to see if anyone would click in the right place. Use the results to constructively agree on a new solution.
- Run an “open” Chalkmark survey to see what type of content people are attracted to. Mock up your most popular landing pages with proposed content and set open ended tasks for your participants. For example, “Click on the first thing you’re likely to read on this page. or “Imagine you’ve just started at ACME Inc. What grabs your attention?”
- Run a card sort with OptimalSort but invite key stakeholders to take part as a group. Book a meeting room and get key people from different business units try to agree on how to organise the cards. You could either do it with real cards, or load it into OptimalSort and get them to do it on screen (this way they have to fight for the mouse!).
Navigation
Primary recommendation/s: Chalkmark
If you have settled on what the labels are for your navigation, you should test the effectiveness of your proposed design. In our experience, visual design and interaction design can have a significant impact on a user’s ability to find information. It pays to know if it’s the user interface or the information architecture (or in this case the label) that is responsible for poor find-ability.
You can set up several tests:
Test navigation in the full context of the page. Where possible, it’s best to have the full context of the page available to a participant. Create a standard set of “find it” tasks and see if people would use the navigation, and if they do, whether they’re using the right labels. Run a small survey, make one or two changes then run another. That way you know what is affecting success and you’re not dealing with too many variables.
Test navigation by itself. Occasionally it’s useful to test just the navigation itself without the distractions of other elements on the page. Take a screenshot of the navigation and set the same “find it” type tasks. Results will clearly show whether people are clicking where you expect them to. You can always make it easier or harder by having the navigation expanded in the right areas in the screenshot you’re using.
Test interaction. To test a workflow simply set multiple tasks in Chalkmark. For example, if I want to know if users are getting to the right pages to get customer support, I would set the same task three times but with different screenshots. Screenshot #1 would be the top level navigation. Screenshot #2 would be the page where they should have clicked. Screenshot #3 would be where they should have clicked in Screenshot #2. For an example, try this Chalkmark demo.
Test navigation variants. Create two separate surveys, each with the same tasks. In survey #1, use design version 1 and in survey #2, use design version 2. Use the timing information and results from Chalkmark to conclude which version works better.
Redesigns
Primary recommendation/s: All three tools in the following order OptimalSort, Treejack, Chalkmark.
Redesigns of information intensive websites typically involve an information overhaul of some sort. In this scenario, we typically use the whole hog of tools in small but frequent iterations. The order can vary depending on which stage of the project you are in. Here is how they are normally used:
Definition phase: Use OptimalSort and in person card sorting combined to get good user research in the context of the card sorting exercise. Involving stakeholders in the card sort is incredibly valuable. Consider using screen sharing tools combined with OptimalSort for remote participants. We recommend you sit and watch at least three people do the card sort and get their thoughts before you send the card sort far and wide. You could also use Chalkmark on the existing website and moderate the sessions. If you are short on participants, combine the card sort with the Chalkmark survey. If you are moderating, you can always just ask participants to tell you what their goals may be for a given page and see where they would click. Many of these tests will help shape and define the user requirements. Create a baseline score for the current user experience by running Treejack and Chalkmark surveys. This will be invaluable once the website has been deployed because the same surveys can be run and compared to this baseline score. To do this, simply create a set of standard user tasks that you expect will remain constant for some time to come. Load these tasks into Treejack and Chalkmark and run the test with at least 30 people. Compose a success score for the overall survey/s and for individual tasks.
Design phase: Test, refine, test, refine, test refine. We tend to find that getting out of the abstract as soon as possible and refining it through rapid iterations of testing works remarkably well. Don’t over-engineer your IA. You don’t need to get it right first time. The time spent trying to get it right is better spent testing and refining it based on real feedback. Pull together your IA and put it into Treejack and run a few people through it. Create a new variation of your IA and do it again. Tweak a label, try a structure that is broader or deeper. This process will give you much greater confidence and iron out bad choices much earlier on. If your designers have already started wireframing the screens - test them in Chalkmark. Doing so forces them to think about the key tasks that users will want to accomplish on any given page - that in itself is invaluable. Upload a screenshot of your paper or wireframe mockup. Don’t wait till your design gets to a high fidelity stage! Test, refine, test, refine! If you have not done any split or A/B testing - do it. It’s simple and can save weeks of meetings that try to decide which option is better. See the A/B section below.
Develop phase: Test the prototypes while they’re being released. Here, you would benefit from good old fashioned user testing. Again, make the most of remote screen sharing tools like Webex, Gotomeeting or Adobe Connect and get regular feedback. You may also choose to focus on areas of the website that you have concerns over. Get some data to facilitate a constructive conversation. We have seen numerous Chalkmark surveys created to accomplish this.
Deploy phase: Run the same Treejack and Chalkmark surveys described in the Define phase. Use the same set of standard tasks but with the new IA and web pages. Hopefully, your results will show a considerable improvement! If it hasn’t already been defined, set measurable and objective criteria to define the quality of user experience of the new website. There are numerous ways to do this with plenty of online tools that can help. We use a simple online (and free) tool called 4Q. This has its drawbacks as it cannot be customised in any way and the experience is less than ideal - but it provides invaluable raw data on the experience over time.
A/B split testing
Primary recommendation/s: Chalkmark or Treejack depending on what you want to test. Chalkmark for user interface variations and Treejack for information architecture variations.
A/B testing or split testing is most commonly done with websites that are live on the internet. It’s a simple but powerful concept. Visitor 1 comes to your website and sees Homepage-Design-A.html. Visitor 2 comes to your website and sees Homepage-Design-B.html. Of course it can be any page, not just your home page or a website.
While there are numerous tools that can help you do A/B testing once you have a live website, there are few that help you do A/B testing of yet-to-be-built prototypes.
This is where Chalkmark and Treejack excel. Before you even think about the user interface or interaction model you can be testing your Information Architecture designs. Customers tell us about the endless meetings where Person X, Y or Z will be advocating for a particular design or label. The same applies to user interfaces, or even what colour something should be! It can be incredibly hard to come to a consensus quickly in these situations. Enter A/B testing.
To set up a A/B test in Chalkmark or Treejack, simply create two surveys. Survey 1 uses design A (be that a home page interface design or an information architecture design). Survey 2 uses design B. Send participant 1 to survey 1 and participant 2 to survey 2. Tasks need to be worded identically for both surveys. So long as you have an equal participation across both surveys you can begin comparing results.
Userfly - how much can you learn from a silent movie?
Posted 2 months ago at 2:00 pm. 0 comments
We’re on a mission to learn about how our website is being used. We’re about to embark on a redesign, resources are tight and time is short so we’ve been using a range to online tools to accelerate the process.
Userfly is a tool that records, stores and replays live website browsing sessions. While we don’t believe it’s anywhere near as useful as a real user test session, the idea did intrigue us.
After a painless sign up, here is the only piece of setup required:
Instructions on how to setup Userfly on your website
We installed this on our key landing pages which are the home page and all our product pages. Within minutes, we got our first recording. Cool.
Our first Userfly recording
This is where it gets interesting. Clicking on the link lets you watch the video. Watch our first recording below and decide for yourself:
Video of our first Userfly recording on Optimal Workshop.com
It almost feels like we’re invading someone’s privacy! It certainly is interesting to see how people browse and what they might be reading or ignoring. Aside from this however, we didn’t really learn anything else. As we have largely a static website, we didn’t really get much from this. Subsequent recordings helped us understand how people browse between pages but we didn’t discover anything new.
Given the speed of the setup and result, the payback is pretty good. It’s hard to deduce anything conclusive about the recordings however. If we can get it working inside the Optimal Workshop application itself - now that would be a different proposition!
Have you experienced cute design?
Posted 3 months ago at 12:33 pm. 0 comments
One of the guys was installing Microsoft Office on his home notebook that he brought in today. The DVDs came in a fancy new plastic case with soft rounded corners. Its contents gracefully swivel out courtesy of an innovative corner hinge. The catch: you have to get it open first.
Scissors? Teeth? Hammer? Just about everyone in the office had a go and wound up wanting to smash it into a thousand tiny pieces. To me, this is an example of cute design. An over-enthusiastic designer had obviously thought that this infuriating piece of packaging was going to take the world by storm. The cuteness won out and suddenly it was a great idea.
Is it just me, or would a simple usability test have shown that this DVD case was a bad idea?
What are you working on that has cute design? Chances are, you already know about it but the others can only see how cute it is. Test it and get some data to support your case.
A trusty steed
Posted 5 months ago at 7:44 am. 0 comments
Over the past month or two we have concentrated almost all our efforts on improving the stability and reliability of Treejack and Chalkmark. I’m pleased to say that I think we’ve made great headway so far! Of course this wasn’t an initiative pulled out of the air.. we were having some significant growing pains. We hit technical limitations on all fronts, almost all at once. Our database server ran out of space, our automated backups were taking too long, we found some things in the database that really needed indexing, we were critically low on memory at busy times, all your Chalkmark images were becoming a big storage issue, our monitoring services weren’t behaving as expected… the list does go on and gets increasingly technical.
So we’ve taken action on all these fronts. Servers have been upgraded, code has been made more efficient and monitoring has been improved. There is now very little chance that I can possibly sleep through a server crash (awesome!), but fortunately there’s substantially fewer ways that could happen (awesome twice!). I hope too that you’ll be noticing a substantial performance improvement; things should generally be snappier for both survey administrators and participants in both Treejack and Chalkmark. Please let me know how you’re getting on- we’re a small team here and we take your feedback straight to heart. You will be heard and your comments are extremely useful to us.
Thanks to everyone who let me know when things weren’t quite right over the past couple of months. We’ve maintained a great percentage of uptime, but even 99.5% uptime allows for 43.8 hours downtime, to which we haven’t even come close, but it has made me think about people who boast their 99% uptime…!
New OptimalSort features released
Posted 6 months, 1 week ago at 4:00 pm. 0 comments
One of the most commonly requested features for OptimalSort is the ability to change the way participants identify themselves. We used to collect email addresses as the primary way of identifying participants to card sort administrators. This can be a big problem for some organisations’ privacy policies.
So, OptimalSort now gives you the ability to:
- Provide any label to identify participants for your card sorts. Eg, you could use Full name, employee number, favourite food etc.
- Allow participants to remain anonymous. They will not be asked for any form of identification and still be able to complete the card sort.
- Continue to use email as a way to identify participants.
Other new features in this release include:
- Automatically end a card sort after a certain number of participant responses has been reached.
- Option to receive daily summaries by email on the total number of card sort responses.
A large number of fixes were also deployed, including:
- Improved cookie detection scripts to ensure that the “No Cookies” page is presented appropriately.
- Fixes to ensure projects end automatically after a certain date.
- Correct rendering of apostrophes and commas in a card sort.
New project setting options in OptimalSort
Card sorting Q&A discussion with Donna Spencer this Friday
Posted 6 months, 1 week ago at 3:30 pm. 0 comments
Each month we host an hour long question and answer session with Donna Spencer, author of the book “Card Sorting: Defining Usable Categories“.
This is an opportunity for you to bring your tricky questions to the (virtual) table and also share advice with other Optimal Workshop customers. It’s free to join in the discussion and all you need is a phone (yes, it’s pretty primitive technology isn’t it?!).
The next 1 hour session will be on:
Thursday 6th August 2009 at 2pm (Pacific Daylight Time, GMT - 7)
This corresponds to Friday 7th August at 7am in Sydney, Australia and Friday 7th August at 9am in Auckland, New Zealand. To check what time this is in your area, try this link.
The session is a chance for you to ask questions of Donna and learn from other participants. What we cover is completely up to you. We suggest you bring questions about card sorting and think about things you can share with others.
To join the discussion, download this PDF document to find out what local phone number you can ring, and then dial in at the scheduled time. When prompted, enter the PIN 266905.
For our sanity, it would be nice if you could let Michelle know if you do intend to join us. Hope to talk to you then!
Card sorting Q&A discussion with Donna Spencer next Friday
Posted 7 months, 1 week ago at 10:47 am. 0 comments
Each month we host an hour long question and answer session with Donna Spencer, author of the book “Card Sorting: Defining Usable Categories“.
This is an opportunity for you to bring your tricky questions to the (virtual) table and also share advice with other Optimal Workshop customers. It’s free to join in the discussion and all you need is a phone (yes, it’s pretty primitive technology isn’t it?!).
The next 1 hour session will be on:
Thursday 9th July 2009 at 2pm (Pacific Daylight Time, GMT - 7)
This corresponds to Friday 10th July at 7am in Sydney, Australia and Friday 10th July at 9am in Auckland, New Zealand. To check what time this is in your area, try this link.
Sorry to those in Europe and the United Kingdom as this time will clearly not suit! (We will run one more convenient for you next month).
The session is a chance for you to ask questions of Donna and learn from other participants. What we cover is completely up to you. We suggest you bring questions about card sorting and think about things you can share with others.
To join the discussion, download this PDF document to find out what local phone number you can ring, and then dial in at the scheduled time. When prompted, enter the PIN 266905.
For our sanity, it would be nice if you could let Michelle know if you do intend to join us. Hope to talk to you then!
Friends from competitors
Posted 7 months, 2 weeks ago at 3:20 pm. 1 comment
While we were at the UPA conference we met up with Jed Wood from Websort - the main competitor to OptimalSort.
We had a blast.
OptimalSort and Websort meetup
It’s odd that as competitors there are a bunch of unspoken rules. Like how you should never talk to each other, hang out together or openly share information. When we started our first business 6 years ago, we told ourselves that not having a business background can work to our advantage. We will try things that “smart” business people probably won’t do because that’s not what you do. Making friends with our competitors is possibly one of those things.
We thoroughly enjoyed meeting Jed. It was uncanny how similar we were, and how many things we had in common. We shared ideas with each other, lamented about facing the same types of problems and even gave each other useful advice. There was no ulterior motive or hidden agenda. We just wanted to say hi and make a friend.
If you are in the position to do so, we encourage you to try it. Call a competitor and offer to take them out for coffee. If you’re a competitor, please talk to us - we would love to shout you coffee.