That one really bad focus group respondent…

Posted October 11, 2011 by Elizabeth Taddonio | Permalink

The October 1 airing of Saturday Night Live featured Melissa McCarthy as host, and man oh man were we ecstatic. What’s better? Half-way into the show, they featured a skit in which she played a focus group respondent and brand enthusiast with too much enthusiasm. It was perfect. It was beautiful. We laughed and laughed.

With GutCheck, you only have one respondent at a time, so the monster respondent is not really an issue you see on the platform (though some respondents are definitely more chatty and much funnier than others). However, we’d love to hear your stories. Have you ever had that one respondent take a focus group up to his or her own level?

Posted in focus groups, Just for fun, qualitative research | Bookmark Bar

Listening to Social Signals

Posted August 24, 2011 by Adam | Permalink

GutCheck CEO Matt Warta recently wrote an Op Ed for Direct Marketing News, urging researchers to double check the findings gleaned from social listening to avoid being taken in by noise. He writes:

Social listening provides a fire hose of data that is used to develop marketing hypotheses around campaigns, pricing, branding and messaging. But knowing how representative the feedback, mentions, reviews and likes are of your target market’s sentiments can be tricky. Is the buzz around your brand cacophony, or is there a signal in all that noise?

…

Whether it’s understanding a vocal minority or helping alleviate the concerns of your target market, digging deeper into the insights and patterns gained by social listening is important to maintaining a good consumer relationship and validating research hypotheses. Giving this kind of attention to a social media crisis shows consumers that a) you’re listening and b) they have a voice. However, discerning what’s noise and what’s losing your brand loyalists will mean the difference between success and expensive missteps.

Check out the rest of the article and let us know what you think — what value have you found in social listening?

Posted in Checkin' It Out, Market Research 101, Social Media | Bookmark Bar

Eavesdropping with Social Media: Opting in and Opting out

Posted August 22, 2011 by Elizabeth Taddonio | Permalink

Last week, the news that Linkedin would be opting users into social advertising was met with a lot of warnings: “UNCHECK THIS BOX!” and “Linkedin thinks it’s Facebook!” were just a few. And it’s true — Facebook really has set the tone for opting users in to social advertising and third party promotions. In the marketing realm, another question circling around social media is whether or not “social listening” is ethical.

In a paper published by Materials Research Society in the UK, the organization states, “In short, because information can be found it does not mean it should be used for research.” MRS made a list of ethical and legal arguments against the practice, the first on the list being “user expectations and terms of use.” They quote Facebook’s terms of use as follows:

“If you collect information from users, you will: obtain their consent, make it clear you (and not Facebook) are the one collecting their information, and post a privacy policy explaining what information you collect and how you will use it.”

This short hand overview of informed consent is set out to protect users, who have a certain expectation of privacy. In other words, they share their interests because they believe those interests are being shared with friends. Just because the information is out there, they don’t necessarily know how it might be used.

For this reason, many users are against having to use their legal names online. Specifically, danah boyd recently argued against the Google mandate to use real names online, calling it an abuse of power and a way of silencing those who might want to adopt pseudonyms for privacy. Bernie Hogan also articulates the idea of user expectations and online vs. offline behaviors, writing “You are not online when you are in front of a computer – you are online when your actions are being digitized and networked. Online is on-the-record. Offline is off-the-record.”

Aside from the idea of establishing a “safe” sharing space (with privacy) online, MRS quotes the Nuremberg code (as any academic researcher had to memorize and understand thoroughly in order to go through Institutional Review Boards), laying out the principle of voluntary participation. That is,

“…The person involved should have legal capacity to give consent; should be so situated as to be able to exercise free power of choice, without the intervention of any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion; and should have sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision.”

This is why doing academic research with children takes so much time — research participants have to go through many levels of consent in order to safely communicate with their subjects. Going through the IRB is full of red tape, but it’s there to protect participants and to prevent a slide down the slippery slope from “they pretty much understand” to “they don’t even know we’re watching.”

The other side of this coin, however, will show that users do volunteer information online, which is considered by many a public space. Digital MR writer Michalis Michael argues that the online space is a realm of performance, and that people use social media in order to be heard by the public. He writes:

“A garden can indeed be private; a public twitter account or a facebook fan-page is not. It is clear to the user, and if it isn’t it should be, that anybody has the right to read their comments and re-tweet (giving credit to the author) their tweets. As a matter of fact the writer in the majority of the cases, if not all, wants people to read their comments.”

Similarly, Ray Poynter argues that if researchers abide by “old” ethical standards, “…they will not be able to compete for business in most areas where market research is growing. This is because there will be no commercial benefits that will accrue to sticking to rules and ideas that nobody else does. To stick to out-dated rules simply provides a worse service for clients. Rules have costs, they only work when they also confer benefits.”

Dr. Annie Petit has also been following this topic and asking for research input on the subject. Through her account we found many arguments for and against setting standards, including a great post from Brian Tarran at Research Live, and also a link to CASRO’s call for comments on its draft for “guidelines that provide an ethical framework for research work performed within the unique formats, behavior systems, terminologies and varied privacy expectations of the social media space.”

So what do you think? Do users online deserve to know if they’re being listened to? Should they opt-in or opt-out of social listening? And what kinds of systematic changes would this mean for social networking sites and marketers alike?

Posted in Features, Market Research 101, Social Media | Bookmark Bar

Market Research 101: Focus Groups vs. One-On-Ones, or, the bar vs. the couch

Posted May 31, 2011 by Elizabeth Taddonio | Permalink

I spent a few days last week in Manhattan on business (this is still a funny idea to me considering I prefer to spell “business” as “bznz”), meeting some wonderful new people at different ad and marketing firms, talking about effective qualitative research and eating a lot of bagels and cupcakes.

I also got to spend time with some old friends from college — sharing Bánh mì (Denver, get me some Vietnamese Sandwiches to nosh on!) and talking about our career and life goals. Since I went to school in upstate New York (ITHACA IS GORGES!), dozens of my friends are in the city and surrounding boroughs, so I was able to meet up with a large group of people even though I was only in the area for a short amount of time.

The thing about meeting people in groups, though, is that there isn’t really that opportunity to talk about what you’ve been doing, your serious goals, your relationships, or all you’ve learned in the years since you’ve seen them. You end up all talking on the surface of topics, bringing up shared group jokes, smiling a lot, and then splitting into twos and threes for the real talk — the talk about anxiety, fear, what is truly making you happy, and your ideas for the future.

After spending an hour on my friend Kate’s couch, talking about our days, our careers, and what we find fulfilling, I realized that this dynamic is exactly the difference between conducting one-on-one qualitative research and focus groups. Focus groups or bulletin boards allow for that initial get together and catch up — everyone gets to talk if they want, everyone gets to add something to the conversation, and you leave feeling as though you got in some great experience with your audience.

One-on-ones, however, allow you to kind of sit across from one person in your audience, asking them things you might not bring up in a crowd, and, as would be expected, getting the kind of candid real talk you might not get around a larger group.

Edward Hopper, “Chop Suey” 1929.

There’s a saying that politics, religion, and money do not make good dinner conversation. Either that’s “a saying” or it’s something someone said to me once, I actually don’t know. But the same can happen in focus groups — sometimes people do not want to speak candidly about sensitive issues, or polarizing philosophies, around a big group of people. People end up dominating the space, they demean viewpoints, and not everyone’s voice is heard. In those cases, one-on-ones can be better suited for learning about individual beliefs and the WHYS behind them. Some other places where one-on-ones can be a better method for your project than a focus group?

1. With topics not suitable for a group, or where a respondent values anonymity

2. To avoid groupthink

3. To follow-up on quantitative results

4. When a topic needs personal rapport

5. For a pilot study to test questions/wording before a survey or larger focus group

When people ask me if GutCheck is a replacement for focus groups or bulletin boards, I honestly say no. There are plenty of awesome tools available out there for online focus groups or bulletin boards (we love the people at Crowdtap, for example). But there’s a time and place for meeting your friends at a weird ping pong bar, laughing and taking photos, and there’s also a time and place to sit on the couch and talk about feeling neurotic, lifestyle changes, and personal philosophies. A good researcher (and a good friend) knows when to make time for the right method, and ends up with the best results (and the longest lasting friendships).

101 is an on-going series on how to effectively use qualitative methods in market research. Up next: Who is a moderator?

Posted in focus groups, Market Research 101, qualitative research | Bookmark Bar

Domino’s Pizza: Market Research Superfan

Posted May 13, 2011 by Adam | Permalink

In my experience, market research isn’t always viewed as the most hot and exciting industry, and sometimes we take a back seat to our more attractive sibling: Advertising. So while most folks value the importance of understanding consumers’ attitudes and emotions toward their product or brand, the actual research doesn’t have quite the same wow factor as a beautifully constructed piece of creative.

But beginning last year, market research finally got its moment in the sun!

Domino’s Pizza launched a big-budget ad campaign that was mostly memorable for saying their own product tasted like crap. But what got me excited was how they learned that they had been pushing inferior pizza for all these years — good old-fashioned qualitative research. The TV spots featured focus group participants telling Dominos executives that their crust tasted “rubbery” and that the pizza was “low-quality and forgettable.” They quoted other unhappy critics saying the pizza was like “cardboard” with “processed cheese.”

While the campaign (created by our friends at Crispin Porter up in Boulder, CO) was pretty unorthodox, Domino’s credited the honest approach with doubling their profits in the quarter after it came out.

But Domino’s promotion of qualitative research didn’t stop with the initial campaign. Late last year they came out with yet another commercial starring a focus group, this time featuring participants talking about the lack of real cheese before being surprised to find themselves on the real-life dairy farm of one of Domino’s suppliers.

And if you’re still not convinced of the trend, just two months ago Domino’s came out with an ad featuring the chef behind their new chicken recipe. In the ad, CEO Jim Doyle shows us the delivery box for the chicken that has market research built right in! “It’s not great until our customers tell us it’s great,” he tells us.

(Quick aside – what kind of strange methodology have they invented here? Have garbagemen been enlisted to collect the results one greasy response at a time? How representative is pizza box sampling? So many questions…)

While most products that make it to market are tested against a target audience, I believe Domino’s is unique in highlighting this bit of sausage making in the ads themselves (pun intended). Perhaps they just hit on an advertising gimmick to help them stand out, or maybe some rouge researcher has infiltrated and overtaken the marketing department.

But one thing is clear: Domino’s Pizza loves them some market research.

Posted in Advertising, Branding, focus groups, Just for fun, qualitative research, quantitative research | Bookmark Bar