From triathlete to professional moderator, practice makes better

Posted May 23, 2011 by Matt Warta | Permalink

I am an amateur triathlete…about as amateur as you can get.  I have never been even close to the professionals in any of events (swimming, cycling, or running), but I keep after it.  I leverage a lot of training tools from experts, and by training and exercising the different muscles for each event, I have gotten very comfortable with all facets of the triathlon and am proficient at all of them.  So much so that I am doing my first half ironman in a couple of weeks.
 
You are probably asking yourself (or should be asking yourself), what does this have to do with performing research?  The answer is you need to build up muscle memory in order to be successful at research just as you need to build up muscle memory around sport or anything else that is important to you.  
 
There are a couple of ways of building up muscle memory in this case.  The first is to get educated.  At GutCheck, we provide this education in a number of ways, including providing some thoughtful education on our site; by publishing discussion guide templates which our qualitative-expert-friends at iModerate developed for our customers; and we also have our own qual coach, Elizabeth T., who is more than happy to walk customers through the finer points of creating a solid discussion guide. (BTW – users love her.)

 If you are at a bigger company, you might have internal or external market research resources you can lean on to gain some great training, direction, and oversight.  The fact of the matter is, you’re not going to become a better athlete if you’re not looking for ways to improve, and you’re not going to become a better researcher if you aren’t open to asking about different methods or ways to improve your interviewing techniques.
Once you have received the appropriate amount of education, go for it.  Go exercise those research muscles you are developing.  Just like your first training run won’t be as good as your last run before the race, your first interview won’t be as good as your second, and your second won’t be as good as your third. Continuous learning, just like continuous training, will make your research (and your performance) the best it can be. And, as it tends to happen, the better you become at research, the more you’ll want to incorporate it into your projects.
 
What advice do you have for someone performing DIY research? How did you get comfortable as a moderator?

Posted in DIY Research, qualitative research, quantitative research | Bookmark Bar

Domino’s Pizza: Market Research Superfan

Posted May 13, 2011 by Adam | Permalink

In my experience, market research isn’t always viewed as the most hot and exciting industry, and sometimes we take a back seat to our more attractive sibling: Advertising. So while most folks value the importance of understanding consumers’ attitudes and emotions toward their product or brand, the actual research doesn’t have quite the same wow factor as a beautifully constructed piece of creative.

But beginning last year, market research finally got its moment in the sun!

Domino’s Pizza launched a big-budget ad campaign that was mostly memorable for saying their own product tasted like crap. But what got me excited was how they learned that they had been pushing inferior pizza for all these years — good old-fashioned qualitative research. The TV spots featured focus group participants telling Dominos executives that their crust tasted “rubbery” and that the pizza was “low-quality and forgettable.” They quoted other unhappy critics saying the pizza was like “cardboard” with “processed cheese.”

While the campaign (created by our friends at Crispin Porter up in Boulder, CO) was pretty unorthodox, Domino’s credited the honest approach with doubling their profits in the quarter after it came out.

But Domino’s promotion of qualitative research didn’t stop with the initial campaign. Late last year they came out with yet another commercial starring a focus group, this time featuring participants talking about the lack of real cheese before being surprised to find themselves on the real-life dairy farm of one of Domino’s suppliers.

And if you’re still not convinced of the trend, just two months ago Domino’s came out with an ad featuring the chef behind their new chicken recipe. In the ad, CEO Jim Doyle shows us the delivery box for the chicken that has market research built right in! “It’s not great until our customers tell us it’s great,” he tells us.

(Quick aside – what kind of strange methodology have they invented here? Have garbagemen been enlisted to collect the results one greasy response at a time? How representative is pizza box sampling? So many questions…)

While most products that make it to market are tested against a target audience, I believe Domino’s is unique in highlighting this bit of sausage making in the ads themselves (pun intended). Perhaps they just hit on an advertising gimmick to help them stand out, or maybe some rouge researcher has infiltrated and overtaken the marketing department.

But one thing is clear: Domino’s Pizza loves them some market research.

Posted in Advertising, Branding, focus groups, Just for fun, qualitative research, quantitative research | Bookmark Bar

Qualitative vs. Quantitative Research Studies: Market Research 101

Posted May 5, 2011 by Elizabeth Taddonio | Permalink

Quantitative research, in its most basic form, boils down to numbers: The idea is to ask a question of enough people to statistically estimate the general viewpoints of a large population. That is to say, it involves breaking a question down into a scale or into defined choices, then tallying up how a large sample of people have responded.

Qualitative research involves asking broader, more open-ended questions that cannot be quantified or broken into defined choices. In qualitative research, the interviewer is trying to elicit reactions and opinions from potential and/or actual users of a product or service in order to gain a more in-depth understanding of an issue – it can help researchers interpret and elaborate on data that might not be apparent from using just numbers or pre-defined answer choices. Because it involves more in-depth questioning, and asks respondents to provide their own answers and experiences, qualitative research usually involves a smaller sample over longer time.

(Photo “Conversation, NYC 1970″ via)

We see quantitative research a lot in the news – 55% of people say they approve of Candidate X, or 74% of people said that they felt better/the same as/ worse about themselves after reading Beauty Magazine. It’s a quick way to make a general point. But sometimes, these general statements have a lot of nuance. For instance, The New York Times reported yesterday that President Obama’s approval ratings have increased since the announcement of the death of Osama bin Laden:

Nearly half [of polled US Americans] said the nation should decrease troop levels in Afghanistan. But more than six in 10 also said the United States had not completed its mission in Afghanistan, suggesting that the public would oppose a rapid withdrawal of all American forces.

Enter here the need for qualitative research follow up – suggestions from the quantitative numbers need clarification found in one-on-one interviews. Would the public truly oppose a rapid withdrawal of American forces? What kinds of conditions would they want to be met in a withdrawal? What factors did they consider when judging the completion of the mission in Afghanistan?

The Times understood this, and later recontacted its quantitative respondents:

One Democrat polled, Richard Olbrich, 68, said in a follow-up interview that Bin Laden’s death was not sufficient reason to remove all American forces.

“The Taliban needs to be defeated,” said Mr. Olbrich, a lawyer from Madison, Wis. “I have no idea how long it will take to complete that mission. And we can’t leave until Afghanistan is back on its feet a little bit.”

As I have mentioned before, quantitative and qualitative methods have important places in market research. While popular platforms for quantitative research include SurveyMonkey.com and Zoomerang – right now the only way to conduct online qualitative research on your own is through GutCheck. This is why our goal is to make the qualitative research space as accessible and easy to use as the quantitative research tools currently available. Numbers can talk, but sometimes that voice needs to be singled out and asked “why”?

101 is an on-going series on how to effectively use qualitative methods in market research. Up next: Matching your project with the right method.

Posted in DIY Research, Market Research 101, qualitative research, quantitative research, Uncategorized | Bookmark Bar

Keeping the Junk Out

Posted April 27, 2011 by Elizabeth Taddonio | Permalink

One of the biggest aversions people seem to have to conducting DIY research is that respondents are not pre-screened and questions may be developed quickly and without much thought (the “Junk in, junk out” rule). With Survey Monkey, for instance, you may create a free quantitative survey within minutes, then send the link out to friends and family, having them forward the link on to their contacts. The researcher ends up with data for sure, but how useful the data is can be another story. So how does a researcher make sure they’re getting the data that matters?

In Gregory Ferenstein’s, “How to Make People Tell the Truth,” which appeared on FastCompany today, he interviews Professor Michael Traugott of the University of Michigan to come up with some interesting dos and don’ts of quantitative research that go beyond the basics. At the end of the article, he states:

Taugott recommends adding in questions that unearth the cause of an answer. Pollsters should ask which party a respondent is affiliated with; a manager might ask a worker how long they’ve been at their job. Without these additional variables, we’re left in the dark, unable to prove why the results turned out a certain way.

This “why” is incredibly important to understanding data, and it’s the main focus of any qualitative researcher. While quantitative research is useful in scaling data and generating larger generalizations, understanding and contextualizing that data is also incredibly important. Asking longer form, open-ended questions is the crux of qualitative, one-on-one research, and it allows for a deeper and clearer profile of the respondent at hand. By screening an anonymous respondent and then asking more detailed and thorough questions and probes, a researcher may have a better time verifying results of a quantitative survey, or even gauging reactions to statements or topics that wouldn’t be obtainable with quantitative research.

Paul Harvey knew all about getting “the rest of the story.”

While making sure quantitative results are accurate and unbiased is imperative to conducting good research, researchers should also think about how qualitative research fits into their objectives, allowing for respondents to answer at length and give feelings and experiences the detail that turns junk into justified insights.

Posted in DIY Research, qualitative research, quantitative research, Survey | Bookmark Bar