What my ill-advised COVID-19 question taught me about customer surveys
On April 24th, 2020, as COVID-19 was bringing the country to a halt, I posed a question to 483 people in an online survey:
“When do you think life will go back to normal?”
I provided a calendar for them to pick a date.
I rediscovered this data a few months ago, tucked away in some old survey files. The results – which you can view below – humorously illustrate our collective ineptitude at predicting the future.
Yet the question itself serves as a cautionary example for ecommerce brands.
Instead of extracting information about the respondents—such as their identity, recent purchases, or experiences with products—I targeted the nature of thinking itself.
Too much cognitive psychology. Not enough demography.
Let’s break my mistakes down.
Vague reference to “Normal”
”When do you think life will go back to normal?” What does “normal” mean here?
I have no idea.
But I do know that it violates a rule of thumb I like to call “Don’t ask about things that aren’t real.”
Last week, when I canceled my subscription to Typeform, I was asked, "How was your cancellation experience (1 -5 stars).”
What is a “cancellation experience?”
Speculation about the future
In this case it was the distant future, which is even worse.
A good question usually relates to a recent experience, whether it’s purchasing or using a product. Stick to the immediate past or present.
Non-falsifiable
The “when” in “When do you think life will go back to normal” is as vague as a corporate mission statement.
Was it when the majority got vaccinated? When schools reopened?
Though not every survey question needs a right or wrong answer, those that do tend to work better.
Questions like:
What’s your gender?
What’s your age?
Which product did you buy?
What do you like about it?
What other brands have you tried?