This new AI technique creates ‘digital twin’ consumers, and it could kill the traditional survey industry
A new one research paper quietly published last week outlines a groundbreaking method that will allow large language models (LLMs) to simulate the behavior of human consumers with astonishing accuracy, a development that could reshape the multi-billion dollar industry. market research industry. The technique promises to create armies of synthetic consumers who can provide not only realistic product reviews, but also the qualitative reasoning behind them, at a scale and speed currently unattainable.
Companies have been trying to use AI for market research for years, but have been hampered by a fundamental flaw: when asked to provide a numerical rating on a scale of 1 to 5, LLMs produce unrealistic and poorly distributed answers. A new paper,”LLMs reproduce human purchase intention via semantic similarity elicitation of Likert ratings,” submitted to the pre-print server arXiv on October 9, proposes an elegant solution that avoids this problem completely.
The international team of researchers, led by Benjamin F. Maier, developed what they call a method semantic similarity assessment (SSR). Instead of asking an LLM for a number, SSR asks the model for a rich, textual opinion about a product. This text is then converted into a numerical vector – an ’embedding’ – and its similarity is measured against a set of predefined reference statements. For example, a response of “I would absolutely buy this, it’s exactly what I’m looking for” would be semantically closer to the reference statement for a “5” rating than to the statement for a “1.”
The results are striking. Tested against a massive real-world data set from a leading personal care company – consisting of 57 product studies and 9,300 human responses – the SSR method achieved 90% human test-retest reliability. Crucially, the distribution of AI-generated ratings was statistically almost indistinguishable from that of the human panel. The authors state, “This framework enables scalable simulations of consumer research while preserving traditional research metrics and interpretability.”
A timely solution as AI threatens survey integrity
This development comes at a critical time, as the integrity of traditional online survey panels is increasingly threatened by AI. A 2024 analysis of the Stanford Graduate School of Business highlighted a growing problem of human pollsters using chatbots to generate their responses. These AI-generated responses were found to be “suspiciously nice,” overly verbose, and lacking the “snark” and authenticity of real human feedback, leading to what researchers called a “homogenization” of data that could mask serious problems like discrimination or product defects.
Maier’s research offers a very different approach: instead of fighting to cleanse contaminated data, it creates a controlled environment for generating high-fidelity synthetic data from the ground up.
“What we are seeing is a shift from defense to attack,” said an analyst not involved in the study. “The Stanford paper showed the chaos of uncontrolled AI polluting human data sets. This new paper shows the order and utility of controlled AI creating its own data sets. For a Chief Data Officer, this is the difference between cleaning a dirty well and tapping a new well.”
From text to intention: the technical leap behind the synthetic consumer
The technical validity of the new method depends on the quality of text embedding, a concept explored in a 2022 paper in EPJ Data Science. That research called for a rigorous “construct validity” framework to ensure that text embeddings—the numerical representations of text—really “measure what they are supposed to do.”
The success of the SSR method suggests that its embedding effectively captures the nuances of purchase intention. To deploy this new technique at scale, companies must be able to trust that the underlying models not only generate plausible text, but link that text to scores in a robust and meaningful way.
The approach also represents a significant leap from previous research, which largely focused on using text embeddings to analyze and predict ratings of existing online reviews. A 2022 studyfor example, evaluated the performance of models like BERT and word2vec in predicting review scores on retail sites, finding that newer models like BERT performed better for general use. The new research goes beyond analyzing existing data and generates new, predictive insights before a product even hits the market.
The beginning of the digital focus group
For technical decision makers, the implications are profound. The ability to create a “digital twin” of a given consumer segment in a matter of hours and test product concepts, ad copy or packaging variations could dramatically accelerate innovation cycles.
As the article notes, these synthetic respondents also provide “rich qualitative feedback that explains their ratings,” providing a wealth of data for product development that is both scalable and interpretable. While the era of human-only focus groups is far from over, this study provides the most compelling evidence yet that their synthetic counterparts are ready for business.
But the business case goes further than speed and scale. Consider the economics: A traditional survey panel for a national product launch can cost tens of thousands of dollars and take weeks to complete. An SSR-based simulation could deliver similar insights in a fraction of the time, at a fraction of the cost, and with the ability to iterate directly based on findings. For companies in the rapidly changing consumer goods categories – where the window between concept and shelf can determine market leadership – this speed advantage could be decisive.
There are of course caveats. The method was validated on personal care products; performance on complex B2B purchasing decisions, luxury goods or culturally specific products remains unproven. And while the paper shows that SSR can replicate aggregated human behavior, it does not claim that it can predict individual consumer choices. The technology works at the population level, not at the person level – a distinction that is of great importance for applications such as personalized marketing.
But even with these limitations, the study is a watershed moment. While the era of human-only focus groups is far from over, this article provides the most compelling evidence yet that their synthetic counterparts are ready for business. The question is no longer whether AI can simulate consumer sentiment, but whether companies can move fast enough to take advantage before their competitors do.
