During a recent conversation with a founder, the following question came up: "When surveying our customers, how do I ensure I'm not biasing anyone?"
It's a perfectly valid concern.
But it reminds me of the Buddhist parable about a man who asks trivial questions about a poisonous arrow lodged in his chest – like the genus of wood it was cut from – until he bleeds out.
The parable is apropos here because when it comes to surveying customers, the poisonous arrow is not the questions you ask but the people who answer them.
You can design a meticulously crafted survey, but the data will mislead you if you don’t weigh the results to reflect your customer base.
For example.
Imagine you own a bus company operating routes between New York City and Boston. You conduct a survey and discover that customers love your mobile app. However, upon closer analysis, you realize that 64% of the respondents fall within the 18-24 age group, while only 28% of your actual customer base fits into this demographic.
To rectify this selection bias, which might have been influenced by a discount code you offered, you weigh the results and find that the mobile app ranks second to price.
Frustrated, you decide to conduct a second survey, this time targeting a sample that represents your total addressable market – people who travel between NYC and Boston at least once every few years. The findings reveal that out of the 25 million trips taken annually, 85% are made by car and only 5% by bus.
This results opens up new growth opportunity: car drivers.
Thanks to your survey expertise, you replicate the amenities question, and you discover that many car drivers would consider taking the bus if your tray tables were large enough for laptops.
Avoid getting caught up in endless debates about minor survey details, such as using "Forbid" versus "Not allow" in a question. Instead, focus on addressing sample bias and recruiting shoppers from your total addressable market to uncover growth insights.
Have a question about sample size, sample composition, or recruiting a sample from your addressable market? Feel free to reach out.
