Visualizing one year of taking every survey I encountered
Last year, I committed to taking every survey I encountered.
From March 2023 to 2024, I completed 199 surveys, averaging 0.35 per day.
I conducted this self-exposed experiment for two reasons.
The first is that while most surveys are as dull as furniture assembly manuals, they’re a decent source of inspiration.
The second is that surveys created by people who don’t take surveys are usually badly designed, just like housing projects designed by architects who don’t expect to live in them.
What I Learned
At the end of the experiment, I was able to articulate a problem I had noticed throughout but struggled to explain.
Surveys are primarily treated as a research tool, rather than as a design challenge.
By “design challenge” I’m not talking about survey aesthetics, but the extent to which a survey is designed with users in mind.
The worst part of the year-long experiment wasn’t the time commitment but answering confusing questions. Many felt like a button labeled “Mode” on your thermostat with no indication of what pressing it would do.
However, a few surveys were as intuitive as a thumb scroll.
"Smiley Terminals" in airports are a good example. You rate your experience by simply pressing a happy, sad, or neutral face. It’s effortless, even if you’re schlepping – or have a young child with you, as I did the dozen or so times I used them.
When I cancelled a Squarespace subscription, I was asked whether I was switching to another site builder (survey #88). I didn’t have to retrieve a distant memory or share an unformed opinion. I simply selected “No,” then shared that I was canceling an unused subscription.
A few weeks into the experiment, I started to appreciate surveys that accounted for users. They weren't just easier – they elicited more accurate and insightful feedback.
The best way to improve your surveys
Become a user.
If you don’t take as many surveys as you can – whether your own or others’ – you’ll be blind to poorly designed questions.
Think more like a UI designer and less like a pollster. Read Don Norman’s “The Design of Everyday Things” and this Paul Graham essay on research and design.
Don’t blame respondents for low response rates or bad feedback – that’s like blaming someone for pushing a door marked “pull.”
If you need help designing a survey, skip to the bottom and check out my Survey Roasts.
Otherwise, here’s the data-driven overview of my experiment.
Top 5 Businesses & Facilities Who Surveyed Me
YouTube: 26
Google Maps: 20
JFK Int. Airport: 7
LinkedIn: 6
WeWork: 6
Top 5 Categories
Travel and Transportation: 32
Technology and Apps: 25
Social and Community Services: 24
Media and Entertainment: 13
Food and Dining: 9
Three categories of “bad questions” worth mentioning
There are many more but these were the most common.
Nonsensical questions
In a survey from a SaaS platform I was investigating (survey #95), a popup said, “Passing on our service? We'd love to know how we can improve,” even though I hadn’t yet passed on their service.Vague response options
Google Maps asked me several times “What kind of trip are you on today,” but the options rarely captured my answer, so I just randomly clicked.Tiring questions
The New York Times once asked me, “What benefits were you looking to get from the page you were viewing?” and provided 14 response options (survey #54). It took me 27 seconds to read everything. I still didn’t know which option to select.
‘Steal-This’ Resource
You’ll need a free Data Wrapper account to download and edit this.
—> Data visualization showing my experiment over a calendar year.
Next Steps
If you’re considering creating a survey, but you’re unsure about your approach, then consider getting a Survey Roast.
Send me your survey draft, and for $145, I’ll make a 10-15 minute Loom video with copy-and-paste edits and suggestions.
I’d love to help.
Cheers,
Sam