Why Your AI Focus Group Keeps Saying Three - Agentic AI - The Digital Transformation Playbook

Topics in this episode

Agentic AI generative AI Artificial intelligence Synthetic Customers

Episode notes

You spend years building a product, polish the packaging, nail the pitch… then you hit the terrifying question: is anyone actually going to buy it? We dig into a 2025 research result from PyMC Labs and Colgate-Palmolive that aims straight at that fear with AI market research, synthetic consumers, and large language models that can simulate purchase intent at scale. TL;DR / At A Glance the core problem with direct Likert ratings and why LLMs collapse to neutral threes how semantic similarity rating converts free-text responses into numerical scores using embeddings and cosine similarity why follow-up AI grading helps but still trails the embedding-based approach what 57 real product surveys and 9,300 human responses reveal about accuracy and distribution matching how persona prompting reproduces real demographic patterns across age and income constraints why zero-shot LLM methods can beat supervised machine learning models trained on the same domain The shocker is that the first attempt fails badly. When you make models like GPT-4 or Gemini answer a classic Likert scale with a single number, they hedge and pile up on neutral “3” ratings.