Welcome to our second speaker interview for UX New Zealand 2019 (check out our first interview with Gregg Bernstein). In the lead up to the conference, we’re catching up with the people who’ll be sharing their stories with you at the conference in October. Today, we chat with Nate Foulds, the product researcher for Stories at Instagram.

Thanks for chatting with us Nate. To start off, would you mind telling me a little bit about your history and how you got started in design?

Nate: Yeah, so I took a pretty non-traditional path. When I was in college I never really thought about design or technology at all. I knew a couple people in computer science and digital design, but it wasn’t really on my radar. I studied political science and art history, and I really wanted to go into art law. But it was senior year and I got cold feet, so I decided to scrap that idea and spend a year teaching English abroad, just to take some time to figure out what to do next.

After a year I moved to San Francisco without a job or anything – just a connection. I had a friend working at an agency, Beyond, that was just starting out and needed some help with some pretty basic marketing-type stuff. Things like light data analysis and social listening, which was big at the time, basically analyzing what people are saying about your company on social media.

And so I started doing that and it turned out to be a pretty good fit. I liked working with so many different clients, getting the inside scoop on how their customers felt and then delivering recommendations for design and marketing. Over time, that work turned more and more into original user research with customers rather than just social listening.

I want to circle back to your comments on working in an agency, but let’s first dive into your work at the New York Times. What was that like?

Nate: I started at the Times after being at the agency for 5 years, and it was my first proper in-house role. At the Times, I led research for news products, which are basically the main website and news app. Projects I worked on included the redesign of The New York Times home page and the mobile app, including the concepting of a personalized news section called ‘For You’.

It was a really interesting time to be there since it was during the 2016 election cycle in the US. We witnessed the field of candidates and then the election itself where Donald Trump won, and then the post-election wake-up call that everyone had. Subscriptions grew an insane amount, just between the few quarters before the election to after the election itself, something like 30 percent, which was about 5 times more than growth periods prior.

And so all of a sudden we had this massive amount of people who were wanting to pay more attention to the news. It was really exciting for us to think about the sorts of features we could offer them to start and keep on subscribing. Like, how much are people willing to pay for the news in the first place? How much can we offer additional news value versus what we think of as complementary features? We found that podcasts and newsletters were really popular, as well as the cooking app and the crossword app. Some of these are complementary businesses that are value-adds for people once they’re in the door with the main news, or for those who don’t like the main news but value the rest.

A special thing about being there is the fact that you're surrounded by some of the greatest journalists in the country. There were times when I led research engagements that involved journalists as partners, and that inevitably resulted in some funny moments. I was once conducting interviews with an observer who was herself a Pulitzer Prize winning investigative journalist. I remember being in the room with her, Jodi Kantor, and I was leading the interview, but I felt so nervous in front of her as someone who's devoted her life to doing this as a skill.

And so after The New York Times you obviously moved over to Instagram, where you’re based now. What’s that change been like?

Nate: Well in some ways Instagram is similar to the Times since it's still a consumer product. I feel comfortable working on those sorts of products where the goal is that anyone can pick up and use them. But as a company, it's pretty different, and a lot of that is just because it was born out of the tech world, versus the Times, which is a journalism company first. A lot of the resources and the infrastructure at Instagram allow you to move fast, test things, get feedback quickly and that sort of stuff. As a researcher, it really unlocks a lot of potential for coming up with ideas, getting feedback on them quickly, testing them, and seeing the results.

And can you talk about what you’ve been working on recently?

Nate: The whole time I've been at Instagram, since January 2018, I've been on the stories engagement team. It’s part of the home team, which is the home tab on Instagram that includes the feed, stories, comments, likes – that sort of stuff.

The research I focus on is how stories fits within the ecosystem of Instagram, thinking about where they appear, how people interact with them, the order in which they appear, how people react to different types of content, etc. Some of the work that we've been doing recently is about how to make stories better for people newer to Instagram, who could be in different markets, or who aren’t so digital-first.

Circling back to what you were talking about earlier, how do you find working at places like Instagram and The New York Times versus the agency environment where you started out?

Nate: There are some similarities, but at the same time it's so different. People usually say that in-house, you have one product and you feel an ownership over it, which I really value personally. At brands like Instagram and The New York Times I’ve enjoyed working on the core pieces – those companies are never going to outsource the core code for the main part of the experience. So I think it's cool to be on the inside and have the ability and influence to affect the product experience.

I’m also surprised by how much depth people can devote to a single feature. At an agency, every 3 to 6 months you're changing your focus completely, in a totally new context with a new audience and a new client. When I found out I was going to be on the stories team at Instagram, I first thought, how could I possibly spend this much time doing research for stories, how could it be a full-time focus for someone?

Soon I realized the depth of the experience, thinking about things like the transition when you swipe from one person's story to the next, understanding what that best experience feels like. The ability to focus completely and go deep on these micro-interactions is a major difference from my agency experience.

A major similarity, though, are those skills you also need in an agency, like pitching and selling ideas and projects, having well-designed presentations, and keeping a large network of people that you're constantly having coffee with. They’re useful skills that will never go out of style no matter where you work.

Would you say there’s been a person that’s influenced your approach as a researcher or your approach to design?

Nate: There’s this one person who comes to mind, Tomer Sharon, you might have heard of him. He was a UX researcher at Google for a long time, and he's this incredible thought leader on research and design. Basically every time I had to pick up something new I would google his writing and speaking. I had this master doc in Google Docs that was just called UX, most of it was derived from Tomer, and it evolved over the years to be something I would use every time I had to go to an interview. I've never met him, though he also lives in New York; my study of his work might creep him out. He's had a huge influence on my career. One day I’ll hopefully get to tell him that.

On a related note, what’s the best piece of advice that you like to repeat to others?

Nate: I know it’s a really common one, especially in UX, but, ‘You are not the user’. I think it's technically called the false-consensus effect, where people tend to design with themselves in mind. A lot of the time this can be great, intuition is a skill that designers have developed. But at the same time it's important to call out our biases.

One example at Instagram is that everyone who works here tends to follow each other, so you might have 50 or more people on your personal Instagram account that are co-workers. And a lot of the time, my co-workers produce pretty good content because they know what creative tools are available, or maybe they're on work trips or offsites. So as a way to remind myself what the experience is like for someone who doesn't have automatic access to this type of content, I basically mute co-workers as soon as I follow them so they don't show up in my stories section. It shows me the normal experience for people who don’t necessarily have that content in their ecosystem.

Do you have anything right now that's currently fascinating you, or that's feeding into your work?

Nate: At Facebook we talk about communities a lot, so lately I've been reading about how communities are formed, the types of relationships between people in communities, hierarchical roles within communities, feelings of belonging, being in multiple communities at once, how people express their identities in communities. And especially how you begin to become a member of a community, and also leave that community.

What does it mean to step into a community for a week or for a month? How can I engage with something or someone that might be interesting now, but won’t be relevant at a certain point in time? How can we make the process of going in and out of these communities as easy as possible? There’s a lot to think about in the future when it comes to mapping online community dynamics to the real world.

What are you looking forward to about speaking at UX New Zealand, or just visiting New Zealand in general?

Nate: I’m excited to come to New Zealand in general because I’ve never been before, and I’m excited for UX New Zealand because it’s a multi-disciplinary, cross-functional conference, focusing on design, product managers, research – I'm sure there will be so many different roles there. For me that's a lot more exciting than just a research-focused conference. I'm really excited to meet people across so many different roles, working at agencies, working in-house, working solo, and to hear their different perspectives.

I didn't know this at first, but I read that Wellington is the culinary capital of New Zealand, so I've been reading about the coffee and the craft beer and all the good food there. I wish I had more time in Wellington, but I'm going to be driving from Auckland to Wellington and stopping at Tongariro National Park where I’m looking forward to doing the crossing!

Thanks for your time Nate, and see you at UX New Zealand!

UX New Zealand is just around the corner. Whether you're new to UX or a seasoned professional, you'll gain valuable insights and inspiration - and have fun along the way! Learn more on the UX New Zealand website.

Welcome to our first UX New Zealand 2019 speaker interview. In the lead up to the conference (which is just around the corner!), we’re catching up with the people who’ll be sharing their stories with you in October.

Today, we chat to Gregg Bernstein, the Senior Director of User Research at Vox Media.

I appreciate you taking the time to chat with us today Gregg. First of all, I just want to say I’m a huge fan of The Verge and the whole Vox Media network.

Gregg: Yeah, I'm a big fan too. It's a treat to get to work with them.

Let’s start off at the beginning. What got you into user research in the first place?

Gregg: So what got me into user research is that I was actually a designer for a number of years. And, after a while, I got pretty tired of design. I used to do a lot of album covers and posters for punk rock bands and independent bands and things like that. And I just felt like I was doing the same thing over and over.

I decided to go to graduate school because, after teaching design at a university for a couple of years, I wanted to teach design full time, instead of doing design work. And it was in grad school that I realized that I liked understanding the information that informs the design in the first place, right? I was fascinated by exploring what the opportunities were and who would consume the final product.

And then I realized what I was really interested in was actually UX research, a term which I didn't even know existed at the time. And then once I realized that this was an entire area of study, it made it clear to me that that's where I wanted to go with my career. So I ended up turning my master's degree in graphic design into a more encompassing study of user experience and UX research. And fortunately ended up getting to do that work at MailChimp just a year after graduating with my MFA.

That actually leads into my next question. I hear you got the original user research practice at MailChimp off the ground?

Gregg: Not exactly. I was given the opportunity to scale up the team and scale up the research practices.

When I first started, all of our work was in service of the UX team. So it was a lot of interviews and usability tests and competitive analyses that were solely to make the MailChimp app better. But over time, as my team shared our work in presentations and in internal newsletters, the rest of the company started asking us questions and it wasn't coming from our traditional UX partners. It wasn't coming from engineering, it was coming from the accounting team or the marketing team and all of this demand for research was evidence that we needed to hire more people and become more of a consultancy to the entire organization.

So I was able to scale up what we were doing in that sense, to serve not just our product and our application, but the entire organization. And really think about what are the questions that are going to help us as a business and help us make smarter decisions.

That must've been quite gratifying to see that payoff though, to see the requests for research data from throughout the organization?

Gregg: I think in hindsight it's more gratifying. When you're in the thick of it, it's, "wow, there's so much demand, how are we going to satisfy everyone?" It becomes a prioritization challenge to try to figure out, which work do we take on now versus what's nice to know but isn't going to help us with either building the right product or marketing in the right way, increasing revenue.

So I was gratified to be put in a position to hire people and try to answer more questions. But when you're in the thick of it's also just a whole lot of, "Oh gosh, how do I do this?"

How do you find leading the research practice at Vox Media versus the practice at MailChimp?

Gregg: It's a lot different at Vox. There is a product team and that's where I live and that's where my team lives. We work within our product organization. But media is so different because you don't (at least in our case) require anybody to sign up or pay for the product. Anybody can read The Verge, anybody can listen to a Vox.com podcast. Anybody can keep up with Polygon wherever they keep up with Polygon. So there's not a true exchange of money for products, so the whole idea of there being a product changes.

One of my roles at Vox is really to help us understand how we can make it easier for journalists to write their stories. So we have a content management system we call Chorus, all of our different networks, whether it's Vox or The Verge or Eater, they use Chorus to write their stories. And then that sends their stories to our websites, but also to Apple news, to Google News, newsletters and Facebook and Twitter. Wherever the stories need to go.

There's the research into, how do we make that experience of writing the news better? How do we make the experience of consuming the news better? What would make a podcast listener have a better experience and find more podcasts? How does somebody who watches us only on YouTube discover other YouTube channels that we create content on?

So it's a very different type of research. I try to help all of our teams make better decisions, whether it's the podcast team with how to market the podcast, or our product team with how to make it easier to write a story. And now I’m working on a new line of business which is how do we sell our content management system to other newsrooms? So, I don't know if you're familiar with Funny Or Die or The Ringer, those are other media companies, but they’re running on our CMS. And so there's research into how do we make our products usable for other organizations that we don't work with day to day.

Is research centralized at Vox or do each of the websites/sub-brands have their own teams and do their own research?

Gregg: They don't have their own research teams. I mean they are all journalists, they all know how to conduct their own investigations. But when it comes to the user experience research, I was the first hire in a company with that skillset and I still help all of our different sub brands when they have questions. Let's say we’re interested in starting up a new newsletter focused on a very specific topic. What they might come to me to understand is the context around that topic. So how do people currently satisfy their need to get information on that topic? Where do they go? Do they pay for it? At what time of day do they read it or watch it or consume it. Those are the types of studies where I will partner with The Verge or Vox or Curbed or whoever it is, and help them get that information.

My primary research audience is our product teams. There are always questions around how can we make the editorial or audience experience better. That's always going to be my first responsibility, but that's 70% of the work. The other 30% is how do I help our other colleagues around the company that are in these sub-brands get answers to their questions too.

Would you say you prefer this type of work that you do at Vox to what you were doing at MailChimp?

Gregg: I prefer any type of job where I'm helping people make better decisions. I think that's really the job of the researcher is to help people make better decisions. So whether it's helping people understand what the YouTube audience for vox.com looks like, or how we make MailChimp easier to use for a small business owner? That doesn't really matter as long as I feel like I’m giving people better information to make better decisions.

That ties nicely into the topic of your UX New Zealand talk, which is research being everyone's job. Do you feel like this is starting to gain traction? Does it feel like this is the case at Vox?

Gregg: It does because there are only 4 researchers at Vox right now, soon to be 3 because one is returning to graduate school. So there's few researchers, but there's no shortage of questions, which means part of the job of research is to help everyone understand where they can get information to make better decisions. If you look at LinkedIn right now, you'll see that there's something like 30,000 UX engineer positions open, but only 4,000 UX research positions open.

There's a shortage of researchers. There's not a lot of demand for the role, but there is a demand for information. So you kind of have to give people the skills or a playbook to understand, there's information out there, here's where you can find it. But not only that, you have to give them the means to get that information in a way where it's not going to disrupt their normal deadlines. So research can't be some giant thing that you're asking people to adopt. You have to give people the skills to become their own researchers.

At Vox we've put together a website that has examples of the work we've done, resources on how to do it and how somebody can do it themselves. A form people can fill out if they need help with a project.

So we're really trying to be as transparent as possible and saying, "these are things that you could do. Here are examples of things that we've done. Here are people you can talk to." There's also Slack channels that we host where anybody can ask us questions. So if I can't do the work myself or if my team can't do it, people will still know that there are options available to them.

What would your advice be for researchers who need to foster a research culture if they're in a very small team or even if they’re by themselves?

Gregg: The first thing you can do is go on a listening tour and just understand how people make decisions now. What information they use to make those decisions and what the opportunities are. Just get that context.

That's step 1, step 2 is to pick one small tightly scoped project that is going to be easy to accomplish but also is going to be meaningful to a lot of people. So what's the one thing that everybody's confused about in your product? Quickly do that research to help illuminate the context of that problem space and offer some scenarios.

And the reason you pick one tightly scoped project is then you can point to it and say, this is what user research can do. This didn't take long, it didn't cost a lot, but we've learned a ton. So I think the starting point is just creating that evidence that people can point to and say, "Hey, look what we did. We could be doing this every day." So you just have to make the case that research is achievable and prove that it's not impossible to put into place.

Do you see this culture taking hold at Vox?

Gregg: I think I'm making progress within Vox. I think people are seeing that research is not hard to incorporate, that it should be a consideration for any project.

I think once people see that they can do their own research, that's step one of a longer process. Like you want everyone to be aware of research and starting to do their own research, but that's a stopgap. Ideally, you want it to get to the point where everyone is saying we need more research and then you can hire dedicated experts who can do the research all the time. And that's where we got to at Vox a year ago where I was able to hire more people, or a year and a half ago, I could hire more people because there was a demand for it and I couldn't be in every meeting and I couldn't take on every project. But the projects were important and we were going to make big decisions based on research. We needed to have more people who were experts doing this work.

So I think everyone being a researcher is the first of a long process to get to having a dedicated research staff. But you have to start with something small, which is everyone could do their own research.

Last question. What are you looking forward to about the conference and/or New Zealand?

Gregg: The thing I'm most looking forward to about the conference itself is I get so much out of meeting attendees and hearing what challenges they're facing. Whether they're a designer or developer or just somebody who works in user experience in any capacity. I want to hear what work looks like for them and how their teams are growing or how their organizations are growing.

In addition to the speakers, that's what I want to hear, is the audience. And then Wellington, I've never been there. I'm super excited to spend a day just walking around and seeing everything and eating some food and having a good time. It doesn't take much to satisfy me so just being there is going to be enough.

Thanks for your time Gregg, and see you at UX New Zealand!

UX New Zealand is just around the corner. Whether you're new to UX or a seasoned professional, you'll gain valuable insights and inspiration - and have fun along the way! Learn more on the UX New Zealand website.

Time, money, people, access, reach. The resources we have at our disposal can also become constraints. In the real world, research projects don’t always follow a perfect plan. There are times when we have to be pragmatic and work with what we have, but there are limits. Knowing where those limits are and when to push back can be really challenging. If we don’t push back in the right way, our research results and design decisions could be compromised and if we push back in the wrong way, we may be inviting a whole host of new resourcing constraints that might just make life harder for us.

Let’s take a look at some research approach compromises that you should push back on, some examples of useful workarounds that will still allow you to gain the insights you need for your project and some constructive ways to lead those push back conversations.

4 research depth compromises that you should push back on

When you’re asked (or told) to talk to experts and frontline staff instead of users or customers

We know we’re not our users and this is definitely one of those moments where we have a responsibility to speak up and try to find a better way. Experts and frontline staff who interact with users or customers all day long certainly have a lot of value to contribute to the design process, however you really do need to gather insights from the people you’re designing for. Failing to include users or customers in your research has a high likelihood of coming back to bite you in the form of poorly designed products, services and experiences that will need to be redesigned costing you more time and money. If you do happen to get away with it and produce something that is fit for purpose, it’s because you were lucky. Don’t base your design decisions on luck and don’t let your stakeholders and team do it either.

When you’re told to just run a focus group (and nothing else)

Focus groups are a pain for a number of reasons, but one of the biggest issues is that the information that you’ll gather through them more often than not lacks depth, context and sometimes even authenticity. When you bring a group of people together into a room, instead of useful and useable insights, you’re more likely to end up with a pile of not-so-helpful opinions and you open your research up to a delightful thing called groupthink where your participants may say they agree with something when they actually don’t. Also, the things that people say they do in a focus group might not align to what they actually do in reality. It’s not their fault – they most likely think they’re being helpful but they’re really just giving you a bunch data you can’t be sure of.

When you’re told to just run a survey (and nothing else)

There’s a time and a place for when a survey might be appropriate, but a standalone research study isn’t it. A survey on its own isn’t enough to gain an appropriate level of depth to inform complex design decisions – it’s more of a starting point or a study to complement a round of user interviews. Surveys don’t allow you to dig deeper into participant responses – you can’t ask follow up questions in that moment and keep asking questions until you get the insight you need. You also don’t know what they’re doing or where they are when they complete your survey. You have no context or control over their environment – they might not complete the whole thing in one sitting and may leave it open on their device while they go off and complete other non-survey related tasks.

Surveys function best when they’re brief and don’t take up too much of your participant’s time because if they’re too long or require in-depth detail to be shared, people might just start providing quick or less than helpful responses just to get through it and finish. If there’s an incentive on offer, you also run the risk of participants providing nonsense responses just to complete the study to obtain the reward or they might just tell you what they think you want to hear so they don’t miss out.

When you’re told to skip discovery research

Skipping this very important early step in the design process in the hopes of saving time or money can end up being quite costly. If you launch into the design stage of a new product or a major redesign of an existing product without conducting UX research upfront, you’ll likely end up designing something that isn’t needed, wanted or fit for purpose. When this happens, all that time and money you apparently ‘saved’ – and then some – will get spent anyway trying to clean up the mess like I mentioned earlier. Start your design journey out on the right foot and work with your team and stakeholders to find a way to not skip this critical piece of research.

4 research depth compromises that won’t kill your project

Talking to a smaller group of users when the only other alternative is doing no research at all

If you have to choose between talking to 5 users or customers or no one at all, always pick the former. Talking to a smaller group is far better than talking to absolutely no one and essentially designing off your and your team’s opinion and not much else. Research is scalable. You don’t have to run 20+ user interviews to gather useful and deep insights – in many cases patterns tend to appear around the 5-10 participants mark. You can run your research in smaller bites and more often to save on time and keep your project moving along. If you’re short on time or money or your customers are hard to reach location wise, run your user interviews over the phone!

Guerrilla research

I’ve met people who aren’t a fan of the term ‘guerilla research’. I’ve been told it’s a negative term that can imply that you’re doing something you don’t have permission to be doing. Well guess what? Sometimes you are! We’re not all in privileged positions where UX research is an accepted and willingly supported practice. UX maturity comes in many shapes and sizes and some of us still need to prove the value of UX research to our stakeholders and organisations.

Hitting the streets or a customer facing environment (e.g., a store) with a mobile device for a few hours one afternoon is a good way to gather research insights quickly. While you will have to limit your interactions with participants to under 3 to 5 minutes, it can be a good way to get a lot of responses to a handful of big burning questions that you might be tackling during your discovery research.

As always, research begets research and this approach might give you the insights you need to secure buy in for a much larger piece of research. You might also use this technique to gather quantitative data or run a quick usability test a new feature. First-click testing tools like Chalkmark for example, are great for this because all the participant has to do is click on an image on a screen. It takes seconds for them to complete and you can include post study questions in the tool for them to answer or you can just have a conversation with them then and there.

Remote research

When it comes to remote research there are a lot of different methods and techniques covering the entire design process from start to finish. It’s super flexible and scalable and the level of depth you can achieve in a short space of time and effort can be significant. The depth compromise here is not being in the same room as your participants. For example if you’re running a remote card sort with OptimalSort, you won’t get to hear a conversation about why certain cards were placed where they were, however you will gather a decent amount of solid quantitative data quickly and most of the analysis work is done for you saving even more time. You can also fill in any qualitative research gaps by including pre and post study questions and you could also use your findings to help prove the need for resources to conduct face to face research to complement your remote study.

Live A/B testing

Also called split testing, live A/B testing on a website or app is a quick way to test out a new feature or an idea for a new feature. Much like with remote research, you won’t get to ask why your research participants did what they did, but you will obtain quantitative evidence of what they did in real time while attempting to complete a real task. It’s a quick and dirt cheap way to find out what does and doesn’t work. You could always ask your website visitors to complete a quick exit survey when they leave your website or app or you could consider positioning a quick poll that appears in the moment that they’re completing the task e.g., during checkout. You can test anything from a whole page to the language used in a Call to Action (CTA), and while the results are largely quantitative, you’ll always learn something new that you can use to inform your next iterative design decision.

How to constructively push back

When approaching push back conversations it can be helpful to try to understand where these requests or constraints are coming from and why. Why are you being told to just run a focus group? Why isn’t there any funding for participant recruitment or a reasonable amount of time for you to complete the research? Why has it been suggested that you skip talking to actual users or customers? And so on. Talk to your stakeholders. Consider framing it as you trying to understand their needs and goals better so that you can help them achieve them – after all, that is exactly what you’re trying to do.

Talk to your team and colleagues as well. If you can find out what is driving the need for the research depth compromise, you might just be able to meet any constraints halfway. For example, maybe you could pitch the option of running a mixed methods research approach to bridge any resourcing gaps. You might run a survey and 5 x 20 minute user interviews over the phone or a video call if you’re short on time for example. It’s also possible that there might be a knowledge gap or a misunderstanding around how long research takes and how much it costs. A little education can go a very long way in convincing others of the importance of UX research. Take your stakeholders along for the journey and do research together where possible. Build those relationships and increased UX maturity may follow.

Pushing back might feel intimidating or impossible, but it’s something that every UX researcher has had to do in their career. User and research advocacy is a big part of the job. Have confidence in your abilities and view these conversations as an opportunity to grow. It can take some practice to get it right, but we have a responsibility to our users, customers, team, stakeholders and clients to do everything we can to ensure that design decisions are supported by solid evidence. They’re counting on us to gather and provide the insights that deliver amazing experiences and it’s not unreasonable to have a conversation about how we can all work better together to achieve awesome things. It’s not about ensuring your research follows a pitch perfect plan. Compromise and pragmatism are completely normal parts of the process and these conversations are all about finding the right way to do that for your project.

Summary: You’ve run some user interviews and a couple of cards sorts, now it’s time to learn how to make the case for larger research projects.

In many ways, the work you do as a researcher is the fuel that product and design teams use to build great products. Or, as writer Gene Luen Yang once put it: “Creativity requires input, and that's what research is. You're gathering material with which to build”.

One of the toughest challenges for a user researcher is making the case for a bigger project. That is, one potentially involving more participants, a larger number of research methods, more researchers and even new tools. Sure, you’ve done small studies, but now you’ve identified a need for some bigger (and likely more expensive) research. So how exactly do you make a case for a larger project? How do you broach the subject of budget and possibly even travel, if it’s required? And, perhaps most importantly, who do you make the case to?

By understanding how to pitch your case, you can run the research project that needs to be run – not whatever you’re able to scrape together.

What’s your research question?

You know how important the research question is. After all, this is what you center your research around. It’s the clear, concise, focused and yet also complex heart of any research project. The importance of a good research question holds true from the smallest of studies right through to the massive research projects that require large teams of researchers and involve usability tests in real-world locations.

We’ve written about user research questions before, but needless to say, keep your question top-of-mind as you think about scaling your research. While most other aspects of your research will get bigger the larger your research project grows (think things like budget, number of participants and possibly even locations), your research question should remain concise and easy to understand. ‘Say it in a sentence’ – This is a good rule to keep in mind as you start to meet with stakeholders and other interested parties. Always have the detail ready if people want it, but your question is basically an elevator pitch.

Your research question will also form an important part of your pitch document – something we’ll come back to later on in this article.

Why do you need to scale up your research?

With your research question in hand (or more likely in a Google Drive document), you have to start asking yourself the tough questions. It’s time to map out the why of this entire process. Why exactly do you need to run a larger research project? Nailing all of this detail is going to be critical to getting the support you need to actually scale up your research.

To help get you thinking about this, we’ve put together some of the most common reasons to scale a user research project. Keep in mind that there’s a lot of crossover in the below sections simply due to the fact that methods/tools and participants are essentially 2 sides of the same coin.

You need more participants

Recruiting participants can be quite expensive, and it’s often a limiting factor for many researchers. In many cases, the prospect of remunerating even something like 10 participants can blow out a small research budget. While certain types of testing have ideal maximums for the number of participants you should use, scaling up the number of people feeding into your research can be quite fruitful. This could either running more tests with a lower number of participants or running a few tests with more.

By bringing in more people, you’re able to run more tests over the course of your project. For example, you could run 5 card sorts with different groups. Or, you could carry out a larger series of usability tests and user interviews with groups of different demographics and in different locations.

It’s easy to see how useful a larger or even unrestricted recruitment budget can be. You can go from needing to scrounge up whoever you can find to recruiting the ideal participants for your particular project.

You want to investigate new tools and methods

User research has been around in one form or another for decades (and possibly even longer if you loosen your definition of ‘user research’). With this in mind, the types of tools available to researchers have come along way since the days of paper prototypes – and even paper card sorts. Now, a cursory Google search will throw up dozens of UX tools tailored for different research methods and parts of the research process. There are tools for validating prototypes, getting fast feedback on UX copy and running complex, branching surveys. As just one example, we (at Optimal Workshop) offer a number of tools as part of our platform, focusing on areas including:

• Tree testing: Tree testing is a usability technique that can help you evaluate the findability of topics on a website.
• Card sorting: A research technique that can show you how people understand and categorize information.

You can read more about the Optimal Workshop platform here on our features page if you’re interested in learning more.

Integrating more tools is a strong reason to scale a research project, as tools can both lighten your workload and open up entirely new avenues of research. Plus, you’ll often find tools will automate difficult parts of the research process, like recruitment and analysis. Using one of our tools as an example, with Reframer, you can take notes during a user interview, tag them, and then Reframer will help you pull out various themes and insights for you to review.

A bigger project usually means a bigger budget, allowing you to spend time investigating possible new methods, tools and research techniques. Always wanted to investigate tree testing but could never quite find the time to sit down and assess the method and the various tools available? Now could be the perfect time.

You want to do on-location research

Remote testing has fast become one of the most practical options for user researchers who are short on time and budget. Instead of needing to go out and physically sit down with participants, researchers can instead recruit, run tests and analyze results as long as they have a decent internet connection. It makes sense – on-location research is expensive. But there are definitely some major advantages.

It is possible to conduct user interviews and usability tests remotely, but the very nature of these types of research means you’ll get a lot more out of doing the work in person. Take a user interview as just one example. By sitting down face to face with someone, you can read their facial expressions, better pick up on their tone and establish a stronger rapport from the outset.

Being able to go to wherever your users are means you’re not constrained by technology. If you need to study people living in rural villages in China, for example, you’re unlikely to find many of these people online. Furthermore, in this example, you could bring along a translator and actually get a real feel for your audience. The same applies to countless other demographics all over the world. Scaling up your research project means you can start to look at traveling to the people who can give you the best information.

Who has a stake in this project?

One of the most important (and potentially difficult) parts of scaling a research project is getting buy-in from your stakeholders. As we’ve mentioned previously, your pitch document is an essential tool in getting this buy-in, but you also need to identify all of your stakeholders. Knowing who all of your stakeholders are will mean you get input from every relevant area in your organization, and it also means you’ll likely have a larger support base when making your pitch to scale your research project.

Start by considering the wider organization and then get granular. Who is your research project likely to impact? Consider more than just product and design teams – how is your larger project likely to impact the budget? Again, capturing all of this detail in the pitch document will help you to build a much stronger case when it comes to convincing the people who have the final say.

Note: Building strong relationships with C-level and other people with influence is always going to be useful – especially in a field like UX where many people are still unsure of the value. Investing time into both educating these people (where appropriate) and creating a strong line of communication will likely pay dividends in future.

Create your pitch document

If you’re from the world of academia, the idea of pitching your research is likely second nature. But, to a user researcher, the ‘pitch’ often takes many forms. In ideal circumstances, there’s a good enough relationship between researchers and UX teams that research is just something that’s done as part of the design process. In organizations that haven’t fully embraced research, the process is often much more stilted and difficult.

When you’re trying to create a strong case for scaling your research project, it can help to consolidate all of the detail we’ve covered above (research question, high-level reasons for a running a larger project, etc.) and assemble this information in the form of a pitch document. But what exactly should this document look like? Well, borrowing again from the world of institutional research (but slightly tweaked), the main purpose of a research proposal is to explain that:

• The research question is of significance
• The planned methods are appropriate
• The results will make a useful contribution to the organization

With this in mind, it’s time to take a look at what you should include in your user research pitch document:

• Your research question: The core of any research project, your research question should query a problem that needs to be solved.
• The key stakeholders: Here’s where you list out every team, agency and individual with a stake in your research, as well as what this involvement entails.
• Data: What information do you currently have on this subject? Pull in any relevant details you can find by talking to other teams in your organization. If you’re researching your customers, for example, have a chat to sales and customer support staff.
• Tools/methods: How will you execute your research? List all of the tools and research methods you plan to use. If you can see that you’re going to need new tools, list these here. This leads on nicely to...
• Budget: What are the financial implications of your larger research project? List the new tools you’ll need and your estimates for recruitment costs.

You don’t necessarily need to take your pitch document and present it to stakeholders, but as a tool for ensuring you’ve covered all your bases, it’s invaluable. Doing this exercise means you’ll have all of the relevant information you require in one place.

Happy researching!

Read more elsewhere

• Asking the right questions during user research, interviews and testing – This is a great guide for crafting research questions from UX Collective. It’s full of helpful tips and considerations.
• How to create use cases – Use cases are one of the best tools to map out particular tasks your users perform and find problems they might encounter along their journey.
• How to become a more valuable UX professional: 3 factors to increase your worth – One way to have more influence and better effect change is to increase your value as a UX professional.
• Do I need a degree to become a good UX Designer? – You can have a wonderful career in the field of user experience without a degree in a related area of study.

With UX research so closely tied to product success, setting up a dedicated research practice is fast becoming important for many organizations. It’s not an easy process, especially for organizations that have had little to do with research, but the end goal is worth the effort.

‍

But where exactly are you supposed to start? This article provides 6 key things to keep in mind when setting up a research practice, and should hopefully ensure you’ve considered all of the relevant factors.

‍

1) Work out what your organization needs

‍

The first and most simple step is to take stock of the current user research situation within the organization. How much research is currently being done? Which teams or individuals are talking to customers on an ongoing basis? Consider if there are any major pain points with the current way research is being carried out or bottlenecks in getting research insights to the people that need them. If research isn't being practiced, identify teams or individuals that don't currently have access to the resources they need, and consider ways to make insights available to the people that need them.

‍

2) Consolidate your insights

‍

UX research should be communicating with nearly every part of an organization, from design teams to customer support, engineering departments and C-level management. The insights that stem from user research are valuable everywhere. Of course, the opposite is also true: insights from support and sales are useful for understanding customers and how the current product is meeting people's needs.

‍

When setting up a research practice, identify which teams you should align with, and then reach out. Sit down with these teams and explore how you can help each other. For your part, you’ll probably need to explain the what and why of user research within the context of your organization, and possibly even explain at a basic level some of the techniques you use and the data you can obtain.

‍

Then, get in touch with other teams with the goal of learning from them. A good research practice needs a strong connection to other parts of the business with the express purpose of learning. For example, by working with your organization’s customer support team, you’ll have a direct line to some of the issues that customers deal with on a regular basis. A good working relationship here means they’ll likely feed these insights back to you, in order to help you frame your research projects.

‍

By working with your sales team, they’ll be able to share issues prospective customers are dealing with. You can follow up on this information with research, the results of which can be fed into the development of your organization’s products.

‍

It can also be fruitful to develop an insights repository, where researchers can store any useful insights and log research activities. This means that sales, customer support and other interested parties can access the results of your research whenever they need to.

‍

When your research practice is tightly integrated other key areas of the business, the organization is likely to see innumerable benefits from the insights>product loop.

‍

3) Figure out which tools you will use

‍

By now you’ve hopefully got an idea of how your research practice will fit into the wider organization – now it’s time to look at the ways in which you’ll do your research. We’re talking, of course, about research methods and testing tools.

‍

We won’t get into every different type of method here (there are plenty of other articles and guides for that), but we will touch on the importance of qualitative and quantitative methods. If you haven’t come across these terms before, here’s a quick breakdown:

‍

• Qualitative research – Focused on exploration. It’s about discovering things we cannot measure with numbers, and often involves speaking with users through observation or user interviews.
• Quantitative research – Focused on measurement. It’s all about gathering data and then turning this data into usable statistics.

‍

All user research methods are designed to deliver either qualitative or quantitative data, and as part of your research practice, you should ensure that you always try to gather both types. By using this approach, you’re able to generate a clearer overall picture of whatever it is you’re researching.

‍

Next comes the software. A solid stack of user research testing tools will help you to put research methods into practice, whether for the purposes of card sorting, carrying out more effective user interviews or running a tree test.

‍

There are myriad tools available now, and it can be difficult to separate the useful software from the chaff. Here’s a list of research and productivity tools that we recommend.

‍

Tools for research

‍

Here’s a collection of research tools that can help you gather qualitative and quantitative data, using a number of methods.

‍

• Treejack – Tree testing can show you where people get lost on your website, and help you take the guesswork out of information architecture decisions. Like OptimalSort, Treejack makes it easy to sort through information and pairs this with in-depth analysis features.
• dScout – Imagine being able to get video snippets of your users as they answer questions about your product. That’s dScout. It’s a video research platform that collects in-context “moments” from a network of global participants, who answer your questions either by video or through photos.
• Ethnio – Like dScout, this is another tool designed to capture information directly from your users. It works by showing an intercept pop-up to people who land on your website. Then, once they agree, it runs through some form of research.
• OptimalSort – Card sorting allows you to get perspective on whatever it is you’re sorting and understand how people organize information. OptimalSort makes it easier and faster to sort through information, and you can access powerful analysis features.
• Reframer – Taking notes during user interviews and usability tests can be quite time-consuming, especially when it comes to analyze the data. Reframer gives individuals and teams a single tool to store all of their notes, along with a set of powerful analysis features to make sense of their data.
• Chalkmark – First-click testing can show you what people click on first in a user interface when they’re asked to complete a task. This is useful, as when people get their first click correct, they’re much more likely to complete their task. Chalkmark makes the process of setting up and running a first-click test easy. What’s more, you’re given comprehensive analysis tools, including a click heatmap.

‍

Tools for productivity

‍

These tools aren’t necessarily designed for user research, but can provide vital links in the process.

‍

• Whimsical – A fantastic tool for user journeys, flow charts and any other sort of diagram. It also solves one of the biggest problems with online whiteboards – finicky object placement.
• Descript – Easily transcribe your interview and usability test audio recordings into text.
• Google Slides – When it inevitably comes time to present your research findings to stakeholders, use Google Slides to create readable, clear presentations.

‍

4) Figure out how you’ll track findings over time

‍

With some idea of the research methods and testing tools you’ll be using to collect data, now it’s time to think about how you’ll manage all of this information. A carefully ordered spreadsheet and folder system can work – but only to an extent. Dedicated software is a much better choice, especially given that you can scale these systems much more easily.

‍

A dedicated home for your research data serves a few distinct purposes. There’s the obvious benefit of being able to access all of your findings whenever you need them, which means it’s much easier to create personas if the need arises. A dedicated home also means your findings will remain accessible and useful well into the future.

‍

When it comes to software, Reframer stands as one of the better options for creating a detailed customer insights repository as you’re able to capture your sessions directly in the tool and then apply tags afterwards. You can then easily review all of your observations and findings using the filtering options. Oh, and there’s obviously the analysis side of the tool as well.

‍

If you’re looking for a way to store high-level findings – perhaps if you’re intending to share this data with other parts of your organization – then a tool like Confluence or Notion is a good option. These tools are basically wikis, and include capable search and navigation options too.

‍

5) Where will you get participants from?

‍

A pool of participants you can draw from for your user research is another important part of setting up a research practice. Whenever you need to run a study, you’ll have real people you can call on to test, ask questions and get feedback from.

‍

This is where you’ll need to partner other teams, likely sales and customer support. They’ll have direct access to your customers, so make sure to build a strong relationship with these teams. If you haven’t made introductions, it can helpful to put together a one-page sheet of information explaining what UX research is and the benefits of working with your team.

‍

You may also want to consider getting in some external help. Participant recruitment services are a great way to offload the heavy lifting of sourcing quality participants – often one of the hardest parts of the research process.

‍

6) Work out how you'll communicate your research

‍

Perhaps one of the most important parts of being a user researcher is taking the findings you uncover and communicating them back to the wider organization. By feeding insights back to product, sales and customer support teams, you’ll form an effective link between your organization’s customers and your organization. The benefits here are obvious. Product teams can build products that actually address customer pain points, and sales and support teams will better understand the needs and expectations of customers.

‍

Of course, it isn’t easy to communicate findings. Here are a few tips:

‍

• Document your research activities: With a clear record of your research, you’ll find it easier to pull out relevant findings and communicate these to the right teams.
• Decide who needs what: You’ll probably find that certain roles (like managers) will be best served by a high-level overview of your research activities (think a one-page summary), while engineers, developers and designers will want more detailed research findings.

‍

Continue reading

‍

• Selling your design recommendations to clients and colleagues – Guest writer Jeff Tang outlines some of his techniques for presenting UX recommendations and answering objections.
• Quantifying the ROI of UX – Ashlea McKay delves into one of the toughest UX questions to answer: “What do we get for our money?”
• How do I explain what UX is? – Ashlea McKay covers how you can explain UX to people who may not necessarily have familiarity with the field.

In this post, Daniel Szuc describes why it’s important to go beyond how we may see our roles traditionally when only focusing on job titles. By exploring other roles, as outlined in the post that follows, we can all play a part in helping to glue a team together, making project work easier for all and creating a more positive environment to help in making meaningful project experiences.

‍

Collaboration is hard and needs practice 🙂↔️

‍

“Collaboration” is a term that gets thrown around in workplaces to encourage people to work together better. Sometimes, though, the people using the term may not understand the range of skills required to make collaboration work well, including (but not limited to) listening, expression, empathy, and curiosity.

‍

‍Each of these skills requires practice.

‍

‍So asking people to simply collaborate, without understanding the skills required nor the necessary spaces to practice these skills, may well frustrate people more than it helps.

‍

Misalignment 😤

‍

As work hums along in a team, it’s easy for misalignment to creep in. Misalignments are caused by a lack of communication, limited time, poor project management, and micro/macro issues that are addressed too late, causing friction between people. If specific roles are not put in place, these frictions can create difficult work environments, making coming to work unpleasant.

‍

Teams may lack common artifacts to help them communicate with a shared language, which in turn helps connect a project and business narrative together. Importantly, this helps aggregate what a team learns together from customer interviews to help improve a product or service.In effect, there is no light leading the way, so people can get lost in details that have nothing to do with a common and well understood purpose.

‍

Roles beyond a job title 👔

‍

When we speak about roles, we are not referring to traditional job titles such as project manager, developer, and designer, for example. Rather, we mean roles that everyone can play at various points in a project, helping others do their job well and the team deliver on making meaningful experiences.Roles, beyond job titles or the tasks inherent in those titles, help people think in integrated and holistic ways beyond their official job title.

‍

At times, our work requires that we delve deeply into design details; in other situations, we are required to step back and see how all the elements of our work connect in delivering solutions that are part of a broader narrative.As members of teams, we can work more effectively – whether it’s by advancing ideas or in recognizing when it’s time to consider alternative approaches.

‍

Four roles for making meaningful experiences 🎢

‍

We have identified four roles to encourage making meaningful experiences for the team and customers, as well as to encourage integrated ways of working:

1. Facilitators can define approaches that guide the process of informing, sense-making, and evaluating. They can craft agendas for working sessions and identify what problems need attention. Facilitators can also manage interactions between functions, aggregate a team’s learnings, and map these learnings to shared artifacts. They identify themes that require further study and set goals for the team’s next sessions.

2. Mentors need to be aware of approaches and skills that require ongoing development and practice, and organize safe spaces in which people can practice, using them over and over during working sessions and across projects. Mentors should work closely with facilitators and custodians to identify the knowledge that the team has captured and map it to a learning program for team members, with a focus on informing, sense-making, and evaluating.

3. Connectors create artifacts that help bridge gaps and make interactions between people feel more fluid, connecting people’s skills and roles.

4. Custodians maintain the knowledge base that forms over time and leverage it in creating approaches and courses that help our project teammates to improve at what they do.

‍

Practicing shared skills within roles ⚙️

‍

Independent of whether a person works in management, engineering, product management, design, user research, or some other function, there is a common set of skills of which people need to remain aware: skills that help make our project teams’ collective efforts better.Because there is an intention to integrate ways of working, collective learning makes teamwork effective and results in more meaningful experiences. Working sessions, in which people from different teams or functions come together to solve a problem, provide a common space to focus on that problem, define approaches to help solve the problem, and work through issues together.

‍

A team can identify the skills they practice, reflect on any gaps that may require them to expand their practice, and aggregate their learnings in common artifacts. These then help form and guide a project narrative with which the team resonates or can critique.In understanding the ways in which we work together – in essence, developing empathy for each other – we may see other benefits in addition to the work we produce.One benefit could be to move away from a blind focus on just tools and processes towards a primary focus on how we approach our work together or how we think about problems within the context of a project.

‍

The ways in which we interact with each other suggest that we should look at the following roles, again independent of function or job title:

1. Informing a problem - What evidence or learnings have we gained to date? What outstanding questions do we need to answer? How would the answers inform the solution to a problem we’re solving now or over time?
2. Making sense of the data we have - How can we make sense of our learnings as they pertain to specific questions or larger themes that we need to understand and for which we need to design solutions over time?
3. Evaluating designs - How can we evaluate designs and iteratively improve a product or service and its positioning over time?

‍

Questions for future consideration 💭

‍

• What roles resonate with you more?
• What roles do you think are missing?
• What skills do you need to practice in order to help your team make more meaningful experiences?
• What skills do you think are missing?
• What gaps, if any, do you recognize between roles on project teams?
• What frictions exist on a team and why do you think they occur?
• How can customer interviews – as one approach to understanding customer stories – encourage constant cycles of informing, sense-making and learning in the spirit of the learning organisation, so to help glue team practices together and create integrated ways of work?

‍

Acknowledgements

‍

Thanks to Josephine Wong for contributing to this piece. For more, see Integrated Approaches to Constant Personal Learning, Improvement, and Maturity.