How to use open-ended questions in user testing

We’re doing a series of posts on asking the right questions to get the right answers when you’re conducting user testing or usability testing. Previously we looked at multiple-choice questions. If you missed that post, you can find it here. In this post, we’ll be looking at open-ended questions. 

Open-ended questions are 

Questions that do not have predetermined possible answers. Instead, the person answering the question is free to give any answer. Answers can be a word, a sentence, a paragraph or even a page long. There will always be participants who give one-word answers, and there will be those who write essays when they answer the same question. That’s okay. If you combine the questionnaire with an interview, you can always ask a follow-up question where you think you need more information. 

Open-ended questions are great when 

• You want respondents to think and respond freely without restrictions.  
• You want detailed, qualitative data. 
• You want to test experience. Respondents will be able to give nuanced feedback on how they experienced your communication.  
• You have more time to analyse the data. Don’t underestimate how much longer it takes to go through each person’s answer; record what they have said; and analyse it. 
• You want to draw out the “voice” of respondents.  

Avoid these mistakes 

• Don’t make open-ended questions too broad. Make sure that the words that you use in the question will keep respondents on the right track in their answers.  
• Don’t ask closed questions in open-ended format. Instead of asking “Do you like this document?” ask “What do you like or dislike about this document?”  
• Don’t lead respondents with your question. Be neutral in how you ask the question. Instead of ‘what in particular catches your eye in this design’ (objection, your honour! Leading!), ask ‘describe how you feel about or experience the design of this document’. 

Want us to chime in on your usability testing project? Give us a shout!