Research Best Practice: Concept Testing and Optimization

One of the most common types of research our clients conduct is concept testing and optimization. As you may know, this is the process of obtaining consumer responses to a product idea and using it to improve the idea prior to the introduction of that product into the marketplace. A concept can also be a package design, an advertisement, or any form of marketing communication designed to alter consumer attitudes toward existing products.

Employing best practices in concept testing will ensure that you are getting accurate and insightful input. We’ve completed thousands of projects in fast-moving consumer goods and food service industries, and much of that work has been in this subject area.  Here are the best practices that we have found to be most useful:

Use both quantitative AND qualitative techniques for concept testing market research

Concept testing market research can be either quantitative or qualitative. But in our experience, the most effective approach is one that combines both because using only one can provide incomplete or skewed results. Moreover, the combination of the two yields a 1 + 1 = 3 effect when used in a holistic, iterative methodology (more on that below). Here’s why:

1. Concepts that clients think are great fail because they were developed without deeply knowing the “who” or the “why” and therefore miss delivering actual value to the consumer; or
2. Truly great concepts die on the vine because they weren’t described in the text or imagery that resonated best with the customer. We see this all the time working with clients after failing BASES or some other form of pre-market testing. Combining qual with quant is a great way to gain an understanding of the who, why, messaging, and imagery necessary for creating a truly great product, ad, or communication. With the advent of on-demand tools, moreover, incorporating qualitative techniques (e.g., on-demand communities, one-on-one chats, online focus groups) is now much more practical, timely, and affordable.

Talk to enough of the right audience

This is key to successful concept testing:  working with the consumers who are most representative of the consumers you are trying to reach. So, draw from a large panel (in the millions) to obtain enough sample from the specific target segment. Also, when screening the panel, while the respondent may be in the right demographic, they may not be the right research subject. Make sure they are open to and interested in the concept, so you don’t get panel members who don’t match your psychographic target. This will drive your overall incidence down, but if you’re drawing from a large panel, this won’t be a problem unless you get below a 10% combined incidence. Lastly, drawing from a large panel is also important because the best concepts are the product (no pun intended) of multiple, iterative reviews with fresh samples of your target segment.

Focus on the big questions

While it may be tempting to focus on some nuanced questions about each concept (how do you feel about that particular word, etc.), focusing on the big questions will yield the highest-impact results: how appealing is this, how clear, how believable, how unique, and how likely is it to be purchased.

Avoid bias and distortion

Asking little questions introduces bias and distortion (how do you feel about this particular benefit articulated in the concept? this elucidates the company’s bias about what THEY think is important or controversial in the concept. The best way to drill into various elements of the concept being tested is to be objective and consistent. To minimize order bias, we set up different groups and show concepts in a different order in each. We treat each concept the same by asking the same set of optimized questions about each in the same order. For more on this topic, check out our post on avoiding confirmation bias.

Employ iterative and monadic testing

We help clients create optimized concepts over the course of several days by allowing them to quickly test and refine concepts during the research in an agile way. It’s one of the ways we can deliver a more highly customized research environment for our concept testing clients. Monadic testing, where each person sees only one concept versus a series of concepts, is helpful when clients have very closely related concepts and are trying to choose between them because it more closely mimics the real life respondent experience. For example, you wouldn’t be choosing between Version A and Version B in real life on the shelf; you would be choosing between two different products.

Use clear concepts and consider the audience for the best results

Clients should come with a clearly written concept articulating distinct benefits and reasons to believe. They should know their audience. We always refer to the old adage, “give me more time and I’ll write you a shorter letter.” This applies in concept writing—less is more.  Descriptive images and, where applicable, videos are helpful, especially if you want feedback on them.

Limit group size to 20-30 

We recommend 20-30 per group since this allows for engagement between respondents (they comment on each other’s posts) and engagement with the moderator. Respondents feel they have a voice with this size, and are part of a group. It also provides enough data that we can notice trends and patterns and analyze effectively.

Share the brand name.

Unless the concept being tested is a new brand or sub-brand, we recommend sharing the brand. It’s an important frame of reference and often a powerful endorser of a new concept.

 Use a fresh audience

 With each new concept it’s important to go back to the drawing board with a fresh audience and perspective each time. So, recruit a new audience for each study. It’s important to treat revised concepts just the same as new ones, because from the respondent’s point of view, it’s brand new!

To learn about some more best practices for concept testing market research, take a look at the eGuide below.