There’s the old saying that one (something) is better than no (somethings). That adage is mostly true for associations too. One call to a member is better than no calls to members. Testing one new product or service is better than not testing any new products or services. One new member this week is better than no new members this week.
The saying breaks down however when we talk about soliciting insights from members. When a committee, the board or the staff gets to a point where they don’t know the correct answer, someone invariably says, “let’s conduct a quick survey”. While that sounds like a low cost and fast way for us to gauge member response it can actually do more damage than having no survey at all.
Our Methodologies are Out of Order
Big organizations always conduct qualitative research first and then quantitative (survey) research. Some just conduct research using a qualitative methodology (for most associations I highly recommend phone interviews). The reason why is without using qualitative insights to design the survey we may get all of the forced-choice answers wrong. Members will pick the best answer of the answers we provide which is not necessarily the right answer. But we always add a blank space for them to fill in their answer…
Members Don’t Fill in the Blank
Sometimes we provide free text blocks in surveys hoping or allowing members to write in their thoughts. They won’t. We expect surveys to be quick and easy. We fly through the pages clicking madly so stopping to write few paragraphs or even a sentence breaks our groove.
Surveys are Terrible at Context
The survey tells us that members are less satisfied than last year and it has something to do with professional development. What now? Is it the cost, the place, the speakers, the topic, or another of a hundred possible answers? Qualitative, with its rich context and detailed storytelling baked right in tells us what is wrong, why it is wrong, how it went wrong and even clues to use to make it right.
Surveys can give us the wrong answer or only a small part of the answer. But there is something about having concrete percents and statistically valid answers that helps us assume the results are correct. So we start developing strategies and tactics… strategies and tactics that might be very, very wrong.
There’s no need for associations to fly blind. We can get member insights, deep, rich, story-based member insights just like the biggest companies get but at a fraction of the price and time. One segment of qualitative member research is far better than no segments of qualitative member research.
Related: