Good Surveys Don’t Need Everyone

Image by Tara Winstead from Pexels.

Have you ever taken a survey that left you more confused than curious? Maybe it asked you ten slightly different versions of the same question, or worse, forced you to pick an answer that didn’t describe you at all. That’s not just annoying, it’s bad research design.

According to The Market Research Toolbox (Edward.F. McQuarrie.2015), a good survey starts long before the first question is asked. In fact it begins with an elucidation of the objectives of research, the decision-maker who has the authority to make certain decisions and could upset current marketing plans. Before you fire up your desktop to crank out the questions, you’d better know the situation it is covering. Otherwise nothing will help such a write to save a poorly planned study.

The Difference Between Good and Bad Surveys

A good survey is instinctual and relevant, because it's built around significant questions that really have a bearing on the business decision at hand. The questions are unbiased, the language is plain, and the scale appropriate to the type of information required. On the other hand, bad surveys try to be all things to all people, collecting whatever data they can, "just in case." This shotgun approach, McQuarrie warns, often creates noise instead of insight.

Think about it this way: A good survey doesn’t come at the same question, “What do you think of our brand?” ten times with just a bit of different phraseology each time. It takes a concept clear in order to ask one clear question. For example, what part matters most in how satisfied someone is with her life? (Is it gratitude, enjoyment of pleasures or mingling policy?),and successfully measures that idea.

Image by Tima Miroshnichenko from Pexels.

Why "The More Participants The Better" Isn't Always True

It's easy to assume that more participants will result in greater precision but not necessarily. As McQuarrie notes, quality matters more than quantity when it comes to sampling. If your sample isn't representative of your real audience, thousands of responses can't fix that.

Limiting the number of people who respond will help make sure that each response contributes meaningful and valid figures. The aim is not to survey everybody, but survey the right people.

For example, if you are testing a new sports merchandising campaign modeled for Generation Z fans, it's better to survey 100 fans who actually buy gear and love the sport than 1000 random users that never watched a game.

Image by Mikhail Nilov from Pexels.

Why Researchers Should Care

A badly designed survey can mislead a company just as easily as no data at all. Good research instructs real decisions, poor research burns money and reduces credibility. In McQuarrie's words,"It is not possible to make an intelligent selection from among research techniques without a clear and comprehensive formulation of the decision problem."

That's why researchers should care: each question, every response, and each sample stands as a decision that will either move a business forward or take it off in an entirely wrong direction.

"A good survey is not about asking more questions, instead it is asking the right one which will make real decisions.”