5 common mistakes we have all made with screening questions
This is a guest post from our friends over at Askable. Check out their blog.
Writing screening questions is an everyday part of life as a UXer or researcher of any kind, really. And at first glance, they seem straightforward enough. Draft up some questions that help to either qualify or disqualify people from taking part in your research, whether that’s a survey, an interview or something in between.
At Askable, we have seen thousands and thousands of screening questions. Some horrible and some amazing - and everything in between.
So here we go – 5 of the most common mistakes made when writing screening questions – oh and how to avoid them.
- Using closed-ended questions too often
What’s the quickest way of knowing if someone went on a holiday in the last 6 months… You ask them, right? “Have you been on a holiday in the last 6 months – Yes or No?”. Duh.
But actually, a question worded in this way is signposting the answer you’re looking for, which may lead to false answers! And also, the answer doesn’t give you any extra information about that person’s travel habits, etc.
So, perhaps a better way to ask the question would be: “When was the last time you went on a holiday?” Provide multiple choices. This also gives you that added info, like if it was a month ago or 5 months ago, in this case.
- Using open-ended questions at the wrong time
Open-ended screening questions can be great, but only for certain info. Avoid using them when you have strict criteria. But instead, use them for getting inside your applicant’s head a bit more. An example would be to ask as a follow up to the example above “Tell me about where you went on your last holiday”.
Open-ended questions are also fine when the answers could vary wildly. A good example is “What is your occupation”. There are simply way too many possible responses to have as a multi-choice.
- Using industry jargon
How many people in the general public know what EV stands for? It’s Electric Vehicle by the way.
Or how about the term ‘Financial Services’? Are we talking about a bank or payments company or an accountant?
Work off the lowest common denominator, assume the applicant doesn’t know anything about your industry. Because often, they don’t or they think of it differently to you. When we live and breathe a topic, it’s all too easy to forget that others do not.
- Too many screening questions
We often write too many screening questions for a number of reasons. Sometimes we do it because we forget that screening questions are just that – to screen. Not to survey! Don’t start adding questions in there that are actually part of your research.
Other times it can be because our criteria is just way too narrow. Whatever the reason, a good rule of thumb is to never have more than 15 and the less the better.
- Not trusting the majority
We have learned this time and time again at Askable – most people are good and honest! We even have a saying now for it – “default to honesty”.
Don’t get overly concerned that your screening questions give too much away. Of course, keep it vague, but don’t go crazy. The 99% of people in our experience won’t take advantage of you. So serve the 99 and not the 1.
Wrap Up
Think about these next time you are writing up some screening questions, setting up your research or trying to figure out who it is you really want to talk with. Do this and you will be on your way to some seriously awesome and accurate insights!
