Smooth The Path

Member Informed Strategy, Innovation & Marketing

  • Home
  • Speaking
    • Presentations
    • Upcoming Events
    • Happy Event Organizers
    • Happy Participants
  • Resources
    • Virtual Networking Incubator Roundup
    • Associations of the Future
    • New Member Engagement Study
    • Association Industry Innovation Research Study
    • New Strategies for Member Onboarding, Engagement, and Retention E-book
  • Blog
  • About
  • Contact

August 13, 2015 By Amanda Kaiser

3 Reasons We Should Be Skeptical of Association Industry Data

why association benchmarking surveys are suspectWe all want to find the benefit or set of benefits that grow membership, increase retention and improve member satisfaction. This elusive goal has us wondering where to turn. So with each new association benchmarking survey report released we eagerly scan the results. Why do members join, we ask. Each time the results are hardly surprising. On the list: professional development, networking, content, annual conference, chapter events, advocacy and more. This is not exactly a new brew with which to catapult our members and association into the future.

Why do so many benchmarking surveys tell us what we already know?

We are Providing the Wrong Answers

Ask a respondent to pick the best of 5 incorrect answers and they will pick the best of 5 incorrect answers. This is what I think we are doing when we design benchmarking surveys. When we ask why do you stay a member and then we provide a list of tangible benefits that the association provides respondents will dutifully sort through the list and rank their top choices even though these answers may not be correct for them. I think we are providing the wrong answers in these surveys because in qualitative member research members talk far more about outcomes than benefits. Unaided members will often highlight: belonging, friendships, support, time to contemplate the big issues, competence and being up-to-date with the latest industry issues, trends and threats. Even more importantly they talk in depth about the special somethings that their beloved association does that keep them coming back, engaged and wanting to contribute.

We Ask The Wrong Respondents

Surveying members is a lot different than surveying association professionals asked to guess why members join. Surveying non-members is very different than surveying an association’s most loyal members (the exact type of members we want more of). Respondents that take the time to fill out the survey in exchange for a gift card are very different than respondents that answer because they want to give back. The data is only valuable if the responses come from the members we most need to hear from. The trouble with many benchmarking surveys is in order to get the volume we ask for answers from everyone.

We Apply Generic Insights to Our Unique Association

Each piece of custom qualitative member research like unwrapping a one-of-a-kind treasure. No two associations are alike. Not even associations serving the same industry and this is because no member group is like another member group. Association benchmarking surveys take responses from very disparate and diverse populations, like from associations that serve hair stylists as well as associations that serve airline pilots and mash the results together which generalizes the data. Just because it is quantified for the association industry does not mean that it is correct or even right for our associaiton.

Surveys as a member research methodology are naturally and intentionally constrained. Because they are constrained, surveys do a fantastic job of reporting some types of data but they do a poor job of accurately reporting other types of data. Trying to answer questions about retention, satisfaction or experience? Trying to plan organization strategy, market or innovate? Benchmarking surveys can’t help you much, in fact surveys can’t help you much but a special qualitative member research method can.

Related:

  • Our members want to give us feedback
  • Did you know that member surveys can be risky?
  • You analyzed member data and conducted surveys, here’s what is next

Filed Under: Member Research Tagged With: data, decisions, feedback, member feedback, member research, qualitative, qualitative member research, quantitative member research, surveys

Follow

  • Facebook
  • LinkedIn
  • Twitter
  • YouTube

Sign up for member insights & association inspiration.

Work With Me

Recent Posts

Association Strategy
Association Marketing
Association Innovation
Association Leadership
Association Value
Member Experience

Member Engagement
Member Research
Member Insights
Conference
Membership

Member Research

Blog

About

Happy Clients

Speaking

Popular Presentations

Contact Us

More Resources

Copyright © 2022 Amanda Kaiser