Keeping the Junk Out

One of the biggest aversions people seem to have to conducting DIY research is that respondents are not pre-screened and questions may be developed quickly and without much thought (the “Junk in, junk out” rule). With Survey Monkey, for instance, you may create a free quantitative survey within minutes, then send the link out to friends and family, having them forward the link on to their contacts. The researcher ends up with data for sure, but how useful the data is can be another story. So how does a researcher make sure they’re getting the data that matters?

In Gregory Ferenstein’s, “How to Make People Tell the Truth,” which appeared on FastCompany today, he interviews Professor Michael Traugott of the University of Michigan to come up with some interesting dos and don’ts of quantitative research that go beyond the basics. At the end of the article, he states:

Taugott recommends adding in questions that unearth the cause of an answer. Pollsters should ask which party a respondent is affiliated with; a manager might ask a worker how long they’ve been at their job. Without these additional variables, we’re left in the dark, unable to prove why the results turned out a certain way.

This “why” is incredibly important to understanding data, and it’s the main focus of any qualitative researcher. While quantitative research is useful in scaling data and generating larger generalizations, understanding and contextualizing that data is also incredibly important. Asking longer form, open-ended questions is the crux of qualitative, one-on-one research, and it allows for a deeper and clearer profile of the respondent at hand. By screening an anonymous respondent and then asking more detailed and thorough questions and probes, a researcher may have a better time verifying results of a quantitative survey, or even gauging reactions to statements or topics that wouldn’t be obtainable with quantitative research.

Paul Harvey knew all about getting “the rest of the story.”

While making sure quantitative results are accurate and unbiased is imperative to conducting good research, researchers should also think about how qualitative research fits into their objectives, allowing for respondents to answer at length and give feelings and experiences the detail that turns junk into justified insights.

2 Responses

1. Market Research 101 Qual v. Quant | GutCheck says:

   May 5, 2011 at 8:31 am

   [...] Keeping the Junk Out [...]

   Reply
2. Alex says:

   April 28, 2011 at 8:51 am

   Elizabeth,
   Great points. One thing that is also awesome about qual is in the pre-survey period determining which words to use. As Professor Ferenstein notes, a small word change can make a big difference in results. It makes sense to test different words and phrases qualitatively in order to understand how the audience perceives them before placing them in surveys and analyzing based upon the assumed meaning of the word or phrase. Thanks for posting!
   Alex

   Reply