Calling BS on the message that “anyone can design a survey.”

One motivation behind $45 Survey Roasts is to call BS on the message that anyone can design a survey. It speaks to the lowest common denominator and completely glosses over the fact that figuring out what questions to ask is not something any one person can just “do.”

You’ll know what I’m talking about if you’ve ever sat down to design a survey and gotten stuck trying to write a single decent question.

How big is this omission?

Consider this.

In his 1940 book “The Pulse of Democracy,” George Gallup discusses how his interviewers at the American Institute of Public Opinion were required to report ambiguous questions and propose clearer alternatives. For instance the seemingly clear question “Do you think the United States will have to fight Japan within your lifetime?” turned out to confuse* most respondents. After extensive debate and testing, the question became “Do you think the United States will go to war against Japan sometime in the near future?”

The point?

Even the great Gallup wouldn’t dare design a survey on his own.

And he made sure that no single Interviewer wouldn’t either.

I don’t blame survey platforms (Typeform, Survey Monkey) for “democratizing” surveys and encouraging people to do-it-themselves. They’re opportunists, motivated by signs up and subscriptions, and I probably wouldn’t have a career without them.

It’s the methodology nerds I take issue with.

They sweep in and offer to help in exchange for expensive research, all while making it clear that they’re the pros and you should be kept far away from surveys.

It’s an overcorrection – and if you’re asking me an insult to Gallup, who was famously transparent and straightforward.

From where I sit, there’s an obvious middle ground between doing-it-yourself and hiring an expensive research agency.

It’s where $45 Survey Roast lives.

Designing the right questions takes time and money that you don’t have.

But the solution doesn’t need to be time consuming or expensive.

I received an email from a recent Roastee who wrote, “Holy sh*t. I’m 4 minutes in and this is gold.”

That’s precisely the response I aim for.

If you’ve ever got stuck designing a survey, desiring a moment of clarity, then consider booking a $45 Survey Roast. Just submit a request, and I’ll get back to you in 24 hours (M-F).

By the way: In a November, 1941 poll that featured the question “Do you think the United States will go to war against Japan sometime in the near future?” 52% of Americans said “Yes.”