The next time someone on your committee says, “let’s survey members” consider this:
Member surveys are the most often used method of member research. While easy, cheap, and fast survey results for associations can be wildly inaccurate. Inaccurate and incomplete results can introduce a high level of risk into our decision making.
Forced-choice is only as good as the answers
Have you ever taken a survey where you select the best of the 5 answers but even that answer was not your answer? This happens all the time. When we write surveys it is very difficult to anticipate the likely answers. This is a huge challenge of the survey instrument. Interested in the details? Read Why Survey Monkey is Dangerous.
Surveys miss context
There are two problems with the fill-in-your-own-answer survey text box. Almost everyone won’t take the time to write a thoughtful answer. They race to complete the survey and will instead find the force-choice answer is most accurate (but still often inaccurate). Also, coding hundreds or thousands of text responses, when many use different language to say the same thing, becomes hard and time consuming.
Surveys are not a great choice for detail or context. But most often it is the context we need. Context helps us create better marketing. Context helps us solve member problems. Context leads us to the right innovations. Context is the building blocks for good strategy. Surveys miss context, the context we need.
Respondents can lie
Our most loyal members make great respondents. Unengaged members, disgruntled members and past members, not so much. In exit surveys past members will always tell you that there was not enough value or the price of membership is too high. But that is not the whole reason, it is merely an easy answer. And there is no incentive for them to give you the real reason. Member exit surveys don’t work, here’s why.
Are you attending the Mid-Atlantic Society of Association Executives Annual Conference today? At 11AM we will be talking about what member surveys are great for and why the results are so often highly risky. In addition we will also discuss other methods that are more accurate and actionable and not as costly as you think. Check out this session, What You Need To Know About Member Surveys.
Bonus links: Wait! Don’t Conduct That Survey! and 3 Ways to get the Attendee and Member Data You Need [two guest posts on the Bear Smarts blog.]