FAQs About Online Quantitative and Qualitative Research

We get a lot of questions about conducting quantitative and qualitative research online. For some companies and agencies, the thought of combining two market research methods without seeing any respondents in person is still a relatively new one. So we’re happy to help spread the agile gospel and answer some burning questions about how online market research actually works.

Q: What’s the nature of the interaction across respondents in qualitative studies, and how is it different from a small sample quantitative survey?

A: The nature of interaction between online qualitative respondents is really dependent upon the platform being used. One that encourages participants to interact and feed off each other as well as the moderator will probably prove to be the most effective, since that interactive discussion is what differentiates the study from a quantitative survey. Our own system closely resembles the posting walls of many social media sites, so users understand it and really enjoy building upon each other’s comments and reactions. We find results to be more thoughtful, honest, and conversational, thanks to the relative anonymity on the platform as well as the time to think without the pressure of a whole focus group waiting for your response.

Another critical aspect of online qualitative interaction that differs from a small quantitative study is the sample size. In quantitative research, a large sample will help offset any flukes and outliers in your data. But a smaller sample size is more appropriate for qualitative research—we’ve had our best success with samples between 20-30 people. This means we have enough respondents to uncover significant insights while ensuring that everyone has a chance to engage and be heard.

Q: What kind of stimuli can be used online?

A: When choosing your online service or partner, it’s important to consider what kind of stimuli you want your participants to interact with, because software capabilities vary from platform to platform. If you’re planning on using a JPG image, PDF, wireframe, website, video, product description, animatic, or any combination thereof, a competent online platform should be able to support it. However, it’s crucial to check with your research partner to ensure that all stimuli you have in mind fits their bandwidth requirements.

Q: How is sample size developed in online quantitative studies to ensure representativeness?

A: Every online research platform has a different method to their sample size madness. The team who runs the platform will usually have a set of best practices in place to determine how many respondents are recommended for your project’s needs. For example, in quantitative research, sample size must be large enough to offer statistical significance, while still focusing on a narrow audience. We’ve partnered with many panel companies across the globe and figured out the best ones for targeting purposes as they relate to country, region, and demographics, giving us a high level of confidence in our findings. We’ve also discovered that when looking at different countries, the comparison between them is not uniform, and thus country-specific findings cannot be weighted the same. Be sure that your full-service research firm is adequately aware of the nuances of online quantitative research, and works to adjust sample size and project scope accordingly.

Q: ­How specific can I get with my audience, i.e. can you access hard-to-reach audiences such as teens and kids?

A: Most online research platforms will be able to screen based on certain behaviors and characteristics as outlined in the project’s key questions: what products respondents buy, when, where, etc. The more capable a research partner is in narrowing down respondents to that specific audience you’re looking for, the more effective your research will be. Your research partner should work with you on getting as much information into those screening questions as possible in a clear, concise manner. We believe that once you give us that target audience, it’s our job to tailor the study to your needs, and working with cross-cultural panelists to cover various markets and industries is extremely helpful in targeting those people for you.

Of course, there are limitations as well as advantages to the audiences you can reach through online market research. For example, laborers such as roofers and HVAC specialists will be hard to target simply due to the nature of their occupation: they’re out and about most days, though they may be online in the evening. However, online platforms can often reach kids and teens with the help of their parents’ consent and without adding additional fielding time. Specific activities are usually incorporated to help verify it’s the teen and not the parents completing the study.

Q: How does online qualitative research uncover deep insights?

A: There’s no singular thing about online qual that ensures a deeper level of insight. Rather, the many aspects of how full-service firms conduct online market research allow them to deliver on those insights that matter most.

First, the platforms utilized for online qualitative research offer a number of benefits. Being able to talk to the same group of consumers over several days—unlike traditional focus groups—allows moderators to stage the flow of content strategically and digestibly. For example, the first day of fielding can begin with a broad exploratory discussion, and the next few days can gradually expose stimuli for specific feedback. We can even adjust the discussion guide questions from day to day as we hear what respondents are thinking. In traditional research, it can be awkward or difficult to go back to the same consumer a day later to probe on key comments, but uncovering such insights as you go is completely natural to the process of online qualitative research.

Additionally, online qual offers the same interactive methods that in-person interviews do, but with added time to reflect on the exercises and articulate responses. An effective online exercise might include image uploads for both emotional response and in-home context. In face-to-face research, it’s not uncommon to recruit, say, a self-proclaimed organic consumer who then opens her cabinets to reveal nothing but Mountain Dew and Cheetos. Online, respondents can upload a photo of their pantry and describe themselves in a safe, anonymous setting that allows them to be honest about their purchase drivers and eating habits.

Finally, online qualitative research would be lost without expert analysis. A full-service research approach should offer trained moderators and analysts who help design the discussion guides in order to keep the research aligned with your objectives throughout the study. Their ability to observe patterns within respondents and the group as a whole, as well as assess the implications of the resulting consumer insights, are what sets full-service online research apart from DIY options.

And these are just a handful of the questions answered in our project development processes and webinars! To learn more about how online quantitative and qualitative research fits into a successful agile methodology, watch this short video clip about how a product marketing team at Google builds, tests, measures, and repeats.