Tomorrow’s Market Research: Experts Talk AI, Automation, and Acquiring New Skills

To solve business problems today and into the future, market researchers now have to be adept at pairing the traditional training they’ve received with new technologies and skills to extract as much meaning and value as possible from data.

GutCheck recently hosted a roundtable with three experts who weighed in on the changing expectations and challenges of market research and why it all matters. The participants included Larry Friedman, Senior Advisor at Larry Friedman Market Research Advisory Services; and GutCheck’s own Chief Product Officer Keith Johnson and Chief Research Officer Renee Smith.

This discussion has been edited and condensed for length.

Why is innovation so important right now, and why is change such an important topic in market research?

Larry Friedman: What’s driving change for almost every company is digital transformation. Today everything has to be better, faster, and cheaper, but the standard joke has always been that two out of three is all you could ever get. Clients are clearly saying that’s not good enough. With digital transformation, there is no shortage of data anymore. And the challenge for market research has changed from how to best collect data, to how to use the data “out there” to create value.

Renee Smith: On top of that, consumer preferences are changing, and clients have more direct-to-consumer competition. A lot of factors are forcing companies to innovate more rapidly and release new and innovative products more frequently. “Faster” and “cost-effective” are still important, but clients need a deeper, more complete understanding of the consumers that buy their products. Becoming audience experts is equally important to speed and cost-effectiveness.

AI is important today as well. How will AI impact the research industry?

Keith Johnson: Machine learning helps us churn through the data and understand what’s really important. It’s a fantastic opportunity to help fuel growth in the market research industry because the answers we provide our clients become much more actionable. We can directly address the business question or need through the use of AI.

There are a couple of areas where we’re likely to see AI being used. An obvious one is the rise of smart assistants and voice interaction with machines. It opens up natural language processing, spoken surveys, and translation of surveys to other markets. What we couldn’t previously do rapidly and at scale, we can now do much more quickly.

Another interesting area is facial recognition. In many ways we’ve been limited to text responses unless we were running a focus group or doing a one-on-one interview. But now we can utilize facial recognition technology to understand the emotion behind the response, which gives us a much deeper context of meaning behind consumer answers.

One area we haven’t spent a lot of time talking about is blockchain technology. There’s a tremendous opportunity to harvest the technology behind blockchain and use it to reduce fraud within the industry, improve the speed of response and data quality, and also improve sampling logistics.

We also need to consider automation. Where has automation made a significant impact on the industry in recent years, and what’s next for it?

RS: I divide the impact of automation into a couple of different areas. One is the automated types of analyses that we couldn’t do before, like with image coding and text analytics. The second is what I broadly classify as workflow automation—for example, research apps like the GutCheck platform where we’ve taken a type of methodology and automated it end to end. It’s been important for some of the speed and cost-effectiveness clients are looking for.

LF: Another promise of automation is that it can take away some of the drudgery and labor in market research and allow people to spend more of their time on what it all means. Unfortunately, I’ve seen some cases where people use it as an excuse not to think at all—they just let the automation do the analysis and that’s it. But it’s really best when we work together with the automation instead of just sitting back, flipping the switch, and letting the machine go.  This is why some people have started using the phrase “augmented intelligence.”

KJ: Expanding on what Renee and Larry said, successful market research companies can blend these evolving technologies with human insight. For example, we may need fewer people doing data analysis if we can have a lot of level-one analysis done by machines and algorithms first, to pre-aggregate information.

RS: I would add that one of the reasons some direct-to-consumer brands are winning is because they have a complete understanding of the audience they’re going after; they understand the segment market and consumer in a holistic way. As digital pushes us in that direction, it’s only going to get more important to understand the person holistically with human brains, not just machines.

So we’ve talked about technology. How do market researchers’ skills need to be transformed, and how will their jobs change?

LF: What needs to be transformed is a better skill around developing hypotheses. It’s a mindset shift as much as a technical one. One of the major challenges with huge data sets is how to get your arms around them. It’s not just a matter of throwing all data into a hopper and hoping the answers magically come out. So that you’re not chasing down rabbit holes, you need to have a good sense of how you think the world works and what you think those relationships should be and then test those. It’s taking design skill and transforming it into a scientific approach.

RS: As a researcher, I like to think about how survey and non-survey data can work together to give us new ways to solve problems. The best art curators think of themselves as junction makers. And researchers need to think about what kinds of junctions they can make between different types of data, between new technologies, or between two hypotheses.

LF: Speaking of surveys, one frustration in the survey industry is how slow the transition to mobile surveying has been. We’re so used to including everything in a survey that could possibly be important that it makes surveys twenty to thirty minutes or longer, and you can’t do that on a mobile device. Combining certain premises out of the survey with others out of data sets enables you to cut that survey down to where mobile becomes much more valuable than it’s been.

Also, market researchers still give primacy to surveys or focus groups. But we can flip that around. Analysis of data sources outside of surveys, like social media, online behavior, location data, and others, is really primary now. One of the roles of survey research going forward is to answer specific questions that come out of big data analyses in order to quickly get at some of the whys or reasons for seeing what we’re seeing.

KJ: We’ve had a couple of very large clients who have combined market research with insights and analytics, which used to be siloed. There’s an opportunity for us, as these roles evolve, to help reduce friction in the organization so we can move faster.

To listen to the full discussion on these and other topics, watch the webinar below.