Words are open to interpretation.
If we ask in a survey, do you feel supported by the association? What does that mean exactly? As a respondent, I might deliberate for a while. Is the survey asking me about the time five years ago when a staff member connected me with another member and eventually I got a great job through that connection? Or should I respond with the experience I had two months ago where I called the association, left a voicemail, and no one got back to me?
In a survey, we don’t know how participants are interpreting the questions, so we do not understand the background that influenced their answers.
Pick almost any question on a survey, from satisfaction, to need, to value, and think about how many different ways members could interpret these questions. When we collect the results, each respondent’s interpretation is hidden to us which means we are in jeopardy of solving the wrong problem. I feel supported because the staff takes the time to connect me to other members and the resources that I need is very different than I feel supported because I know when I call someone will answer the phone.
The interpretation problem is one reason why I like qualitative member research so much. Participants tell me through their responses how they are interpreting the question. I have the opportunity to ask to follow up questions to better understand the background influencing their responses. As a result, we have the whole story and all the information we need to take action today to solve the right problem.
Related: