How Andy is using card sorting to prioritize our product improvements
There has been a flurry of new faces in the Optimal Workshop office since the beginning of the year, myself included! One of the more recent additions is Andy (not to be confused with our CEO Andrew) who has stepped into the role of product manager. I caught up with Andy to hear about how he’s making use of OptimalSort to fast-track the process of prioritizing product improvements.
I was also keen to learn more about how he ensures our users are at the forefront throughout the prioritization process.Only a few weeks in, it’s no surprise that the current challenges of the product manager role are quite different to what they’ll be in a year or two. Aside from learning all he can about Optimal Workshop and our suite of tools, Andy says that the greatest task he currently faces is prioritizing the infinite list of things that we could do. There's certainly no shortage of high value ideas!
Product improvement prioritization: a plethora of methods
So, what’s the best approach for prioritization, especially when everything is brand new to you? Andy says that despite his experience working with a variety of people and different techniques, he’s found that there’s no single, perfect answer. Factors that could favor a particular technique over another range from company strategy, type of product or project, team structure, and time constraints. Just to illustrate the range of potential approaches, this guide by Daniel Zacarias, a freelance product management consultant, discusses no less than 20 popular product prioritization techniques! Above all, a product manager should never make decisions in isolation; you can only be successful if you bring in experts on the business direction and the technical considerations — and of course your users!
Fact-packed prioritization with card sorting
For his first pass at tackling the lengthy list of improvements, Andy settled on running a prioritization exercise in OptimalSort. As an added benefit, this gave him the chance to familiarize himself with one of Optimal Workshop’s tools from a user’s perspective.In preparation for the sort, Andy ran quick interviews with members across the Optimal Workshop team in order to understand what they saw as the top priority features. The Customer Success and User Research teams, in particular, were encouraged to contribute suggestions directly from the wealth of user feedback that they receive.
From these suggestions, Andy eliminated any duplicates and created a list of 30 items that covered the top priority features. He then created a closed card sort with these items and asked the whole team to to rank cards as ‘Most important’, ‘Very important’, and ‘Important’. He also added the options ‘Not sure what these cards mean’ and ‘No opinion on these cards’.
He provided descriptions to give a short explanation of each feature, and set the survey options so that participants were required to sort all cards. Although this is not compulsory for an internal prioritization sort such as this, particularly if your participants are motivated to provide feedback, it can ensure that you gather as much feedback as possible.
The benefit of using OptimalSort to prioritize product improvements was that it allowed Andy to efficiently tap into the collective knowledge of the whole team. He admits that he could have approached the activity by running a series of more focussed, detailed meetings with key decision makers, but this wouldn’t have allowed him to engage the whole team and may have taken him longer to arrive at similar insights.
Ranking the results of the prioritization sort 🥇
Following an initial review of the prioritization sort results, there were some clear areas of agreement across the team. Topping the lot was implementing the improvements to Reframer that our research has identified as critical. Other clear priorities were increasing the functionality of Chalkmark and streamlining the process of upgrading surveys, so that users can carry this out themselves.Outside of this, the other priorities were not quite as evident. Andy decided to apply a two-tiered approach for ranking the sorted cards by including:
- any card that was placed in the ‘Most important’ group by at least two people,
- and any card whose weighted priority was 20 or greater. (He calculated the weighted priority by multiplying the total of each card placed in ‘Most important’, ‘Very important’ and ‘Important’ by four, two and one, respectively.)
By applying the following criteria to the sort results, Andy was left with a solid list of 15 priority features to take forward. While there’s still more work to be done in terms of integrating these priorities into the product roadmap, the prioritization sort got Andy to the point where he could start having more useful conversations. In addition, he said the exercise gave him confidence in understanding the areas that need more investigation.
Improving the process of prioritizing with card sorting 🃏
Is there anything that we’d do differently when using card sorting for future prioritization exercises? For our next exercise, Andy recommended ensuring each card represented a feature of a similar size. For this initial sort, some cards described smaller, specific features, while others were larger and less well-defined, which meant it could be difficult to compare them side by side in terms of priority.
Thinking back, a ‘Not important’ category could also have been useful. He had initially shied away from doing this, as each card had come from at least one team member’s top five priorities. Andy now recognizes this could have actually encouraged good debate if some team members thought a particular feature was a priority, whereas others ranked it as ‘Not important’.
For the purposes of this sort, he didn’t make use of the card ranking feature which shows the order in which each participant sorted a card within a category. However, he thinks this would be invaluable if he was looking to carry out finer analysis for future prioritization sorts.
Prioritizing with a public roadmap 🛣️
While this initial prioritization sort included indirect user feedback via the Customer Success and User Research teams, it would also be invaluable to run a similar exercise with users themselves. In the longer-term, Andy mentioned he’d love to look into developing a customer-facing roadmap and voting system, similar to those run by companies such as Atlassian.
"It’s a product manager’s dream to have a community of highly engaged users and for them to be able to view and directly feedback on the development pipeline. People then have visibility over the range of requests, can see how others’ receive their requests and can often answer each other’s questions," Andy explains.
Have you ever used OptimalSort for a prioritization exercise? What other methods do you use to prioritize what needs to be done? Have you worked somewhere with a customer-facing product road map and how did this work for you? We’d love to learn about your ideas and experience, so leave us a comment below!
