We all want to find the benefit or set of benefits that grow membership, increase retention and improve member satisfaction. This elusive goal has us wondering where to turn. So with each new association benchmarking survey report released we eagerly scan the results. Why do members join, we ask. Each time the results are hardly surprising. On the list: professional development, networking, content, annual conference, chapter events, advocacy and more. This is not exactly a new brew with which to catapult our members and association into the future.
Why do so many benchmarking surveys tell us what we already know?
We are Providing the Wrong Answers
Ask a respondent to pick the best of 5 incorrect answers and they will pick the best of 5 incorrect answers. This is what I think we are doing when we design benchmarking surveys. When we ask why do you stay a member and then we provide a list of tangible benefits that the association provides respondents will dutifully sort through the list and rank their top choices even though these answers may not be correct for them. I think we are providing the wrong answers in these surveys because in qualitative member research members talk far more about outcomes than benefits. Unaided members will often highlight: belonging, friendships, support, time to contemplate the big issues, competence and being up-to-date with the latest industry issues, trends and threats. Even more importantly they talk in depth about the special somethings that their beloved association does that keep them coming back, engaged and wanting to contribute.
We Ask The Wrong Respondents
Surveying members is a lot different than surveying association professionals asked to guess why members join. Surveying non-members is very different than surveying an association’s most loyal members (the exact type of members we want more of). Respondents that take the time to fill out the survey in exchange for a gift card are very different than respondents that answer because they want to give back. The data is only valuable if the responses come from the members we most need to hear from. The trouble with many benchmarking surveys is in order to get the volume we ask for answers from everyone.
We Apply Generic Insights to Our Unique Association
Each piece of custom qualitative member research like unwrapping a one-of-a-kind treasure. No two associations are alike. Not even associations serving the same industry and this is because no member group is like another member group. Association benchmarking surveys take responses from very disparate and diverse populations, like from associations that serve hair stylists as well as associations that serve airline pilots and mash the results together which generalizes the data. Just because it is quantified for the association industry does not mean that it is correct or even right for our associaiton.
Surveys as a member research methodology are naturally and intentionally constrained. Because they are constrained, surveys do a fantastic job of reporting some types of data but they do a poor job of accurately reporting other types of data. Trying to answer questions about retention, satisfaction or experience? Trying to plan organization strategy, market or innovate? Benchmarking surveys can’t help you much, in fact surveys can’t help you much but a special qualitative member research method can.
Related: