Why do we want statistics so much? We feel better when we can say 75% of our members do this. 45% want this. 68% prefer this. We tend to think statistical measures are the truth. And many times they are.
But sometimes they are not.
So when we are making really important decisions we want to evaluate how true this data is. When making a strategic decision on the basis of qualitative data (a member survey, our own data or third party data) ask these questions:
- Who responded and are these the members we most need insight from?
- How did we ask the question? Are we interpreting the answer correctly?
- Are we missing important context before we make this decision?
- Did we ask respondents to predict what they would do? Or did we ask them about their challenge?
- What is our goal and does our research methodology support our goal?
- Are respondents picking the best answer or the correct answer?
Just because there is % symbol behind a number doesn’t make it accurate and true. I have to remind myself of this all the time. When making a big decision it is worth knowing what is behind the data before we trust the data.
Marketoonist, Tom Fishburne says now days we have so much access to data which allows us to present the data that makes our case so that we should be data skeptical. I agree.