AI is increasingly being used in market research. Among other applications, the concept of synthetic respondents is becoming more widespread. But can AI really simulate real people in surveys, and can it also provide reliable insights?

Generative AI is making waves in marketing, especially in market research. From analyzing social media sentiment to coding survey responses, AI has the potential to transform how we understand consumers. But can it truly simulate human insights? And how reliable are these machine-generated responses?

One emerging concept is using "synthetic respondents," where AI answers survey questions instead of real people. While intriguing, this raises concerns: Are the insights accurate enough to base business decisions on? Critically, the risks of bias and unreliable data could negatively impact business strategies.

This research delves into whether AI-generated responses can truly replicate human feedback. The potential is huge, but marketers must be cautious. Understanding the strengths and limitations of generative AI will be critical in deciding if it’s the right tool for deeper consumer insights—or just a shortcut to flawed data.

To explore whether AI can simulate human insights, a team of researchers from NIM designed three surveys on diverse topics and compared the responses of 500 real U.S. consumers to those generated by 500 AI-based respondents. These AI respondents were created using OpenAI's GPT-4, set to standard parameters.

We focused on U.S.-based respondents because GPT-4 was trained on large volumes of American data, increasing the likelihood that AI responses would closely mirror those of real people. The surveys covered opinions on soft drinks, sportswear brands, and U.S. political views.

For each survey, we gathered demographic information from the real respondents, then created an AI "digital twin" for each. These digital personas replicated key characteristics—like ethnicity, gender, location, and profession—across 10 demographic variables. For example, if a real respondent was a white male from Michigan, the AI was instructed to match that profile when generating responses.

The AI answered all survey questions as its assigned persona, and to ensure consistency, it was given prior answers to simulate human recall, just as a person would reference their previous thoughts when answering follow-up questions.

• Efficiency with limits: AI offers significant advantages in streamlining market research processes, reducing costs, and providing quick, broad insights, but it lacks the depth and nuance of real consumer feedback.
• Mainstream bias: AI tends to favor well-known brands and mainstream opinions, missing the perspectives of early adopters or niche markets, which can lead to a narrow and potentially misleading view of consumer behavior.
• A supplemental, not a replacement: While AI can provide helpful directional insights, it lacks the precision and diversity needed for actionable consumer insights, making it more suitable as a supplementary tool rather than a replacement for human respondents.