Marketing Analytics with Fexingo · 2026-09-09 · 11 min
Key moments - from our scoring
Substance score
71 / 100
Five dimensions, 20 points each
Fexingo's Lucas and co-host Luna dissect the structural flaws in last-click attribution models, which systematically overvalue bottom-funnel channels and create feedback loops that starve awareness campaigns of budget despite their genuine impact on demand. Using a real case study of a DTC apparel brand claiming 3.5x ROAS but actually delivering only 1.8x incremental lift, they demonstrate how attribution dashboards mask the truth: roughly half the attributed sales would have happened anyway. The pair advocate for geo-lift tests and randomized holdout groups - now accessible through Meta and Google's built-in platforms - as the path to causal measurement. They cover statistical rigor (sample sizing, p-values, confidence intervals), experimental design windows (4 weeks for FMCG, 6-8 for high-ticket items), and the importance of triangulating quantitative conversion data with qualitative brand recall surveys. The methodology scales from $100k tests upward and transforms the CFO conversation from defensive justification to strategic partnership grounded in causality, not correlation.
Last-click attribution assigns all credit to the final touchpoint before conversion, ignoring the awareness and consideration work done by earlier channels. A customer exposed to a brand-building video three weeks prior appears in the system as having been converted by a retargeting banner seen an hour before purchase, making the expensive awareness spend invisible and creating a feedback loop where marketers cut those channels despite their genuine impact.
You must define your required sample size and expected lift percentage before launching (e.g., expecting 2% lift requires sufficient impressions to reach 95% confidence interval with low variance). After the test, check the p-value; if it's above 0.05, the result is not statistically significant and you should wait for more data or redesign rather than acting on potential noise.
The brand claimed 3.5x ROAS based on last-click attribution, but geo-lift testing across six mid-sized cities revealed actual incremental lift of only 1.8x, meaning approximately 50% of attributed sales would have converted anyway without the ad spend, and the campaign was also driving significant branded organic search volume not captured in attribution.
Four weeks is typical for fast-moving consumer goods to capture standard purchase cycles; high-ticket items like furniture or electronics require 6-8 weeks to measure the full consideration period. Batching tests over discrete windows is smarter than continuous testing, which can lead to fatigue and diminishing learning returns.
Combine sales volume with average order value to create an incremental revenue per impression metric, which accounts for both quantity and quality of conversions - preventing the trap of chasing cheap low-margin clicks while ignoring expensive clicks that bring high-value repeat buyers.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode delivers substantive, specific critiques of last-click attribution and articulates the mechanics of geo-lift testing clearly. The concrete example (apparel brand with 3.5x ROAS masking 1.8x true lift) and the branded search insight demonstrate non-obvious claims. However, some sections verge on repetition (the feedback loop point recurs multiple times) and latter portions become slightly more procedural than novel.
If you run a twenty-second video ad that builds genuine brand recognition, that customer might not convert for three weeks, but when they finally do, your dashboard gives all the credit to the retargeting banner they saw an hour later.
They found the actual incremental lift was only one point eight, meaning nearly half of those attributed sales would have happened anyway without the ad spend.
The core argument - that incremental lift testing is superior to attribution - is sound but not especially contrarian in marketing analytics circles; practitioners and researchers have advocated this for years. The framing around the feedback loop of budget cuts is somewhat novel, and the emphasis on mixing quantitative and qualitative data adds dimensionality. However, the episode largely reinforces established incrementality methodology rather than proposing fresh theoretical ground.
The real solution isn't to guess which channels matter, it's to measure incrementality directly through controlled experiments rather than relying on correlation in aggregated data.
Quantitative tells you what happened, qualitative helps you understand why the holdout behaved differently.
Lucas is presented as an analyst or researcher working on incrementality at a firm (Fexingo), but the transcript reveals minimal biographical detail about his specific operating history, scale of campaigns managed, or organizational seniority. He speaks knowledgeably but sounds more like a methodologist than a practitioner who has actually built and scaled direct-to-consumer businesses or run major media operations. Luna appears to be the host/interviewer rather than a co-guest.
We see this play out constantly with mid-market consumer brands
They were spending roughly five million dollars annually on broad social video placements
The episode is rich with specific numbers and concrete examples: the $5M apparel brand case with 3.5x vs. 1.8x ROAS, 20% drop in branded searches in holdout, 4-week test windows for FMCG and 6-8 weeks for furniture, 95% confidence intervals, p-value thresholds (0.05), and sample-size calculations. The geo-lift methodology is explained with operational detail (city-level holdout groups, randomization, metrics like incremental revenue per impression). Few claims float without supporting numbers.
They were spending roughly five million dollars annually on broad social video placements, claiming a three point five return on ad spend based on last-click models.
When they ran a geo-lift test across six mid-sized cities, they found the actual incremental lift was only one point eight
Luna asks intelligent follow-up questions that probe edge cases and risks (pool drying up, holdout defection to competitors, sample size validity, seasonality), demonstrating genuine critical thinking. However, most responses from Lucas go unanswered or are met with agreeing reformulations rather than productive pushback or skeptical challenge. The interview lacks moments of genuine tension or disagreement; Luna's questions are supportive rather than adversarial. A brief aside on show support breaks conversational flow and feels like an ad insertion.
But if you pull back on those awareness spends, don't you eventually run out of new people to retarget? The pool has to dry up somewhere.
But isn't there a risk that the holdout group just gets bored and buys from a competitor?
Computed from the transcript - who did the talking, and the words that came up most.
Most marketing teams treat last-click attribution as gospel, but it systematically undervalues upper-funnel work. We look at how a mid-market consumer brand shifted to incrementality testing to find out what actually drives sales versus what just gets credit. The data reveals that nearly forty percent of their top-of-funnel spend was invisible to standard dashboards until they ran geo-lift experiments. This episode breaks down the mechanics of holdout groups, why statistical significance matters more than daily averages, and how to structure a test that doesn't bleed budget. It is a practical guide for anyone tired of defending creative work that never appears in the final conversion column. #MarketingAnalytics #IncrementalityTesting #LiftStudies #LastClickBias #HoldoutGroups #MediaMixModeling #UpperFunnel #ConversionAttribution #StatisticalSignificance #GeoLiftExperiments #BrandAwareness #PerformanceMarketing #DataDrivenDecisions #FexingoBusiness #BusinessPodcast #MarketingStrategy #ROIMeasurement #DigitalAdvertising Keep every episode free: buymeacoffee.com/fexingo
Transcribed and scored by The B2B Podcast Index.
Lucas: The problem with last-click attribution isn't that it's completely wrong, it's that it's aggressively misleading for anything that happens before the final button press. If you run a twenty-second video ad that builds genuine brand recognition, that customer might not convert for three weeks, but when they finally do, your dashboard gives all the credit to the retargeting banner they saw an hour later. Luna: So you're essentially saying the system rewards the closer and punishes the creator, which makes upper-funnel investment look like a waste of money every single time.
Lucas: Exactly. And this creates a feedback loop where marketers keep cutting broad awareness campaigns because the numbers look bad, which then shrinks the top of the funnel, making the remaining bottom-funnel traffic even more expensive and competitive. We see this play out constantly with mid-market consumer brands who think they are optimizing for efficiency when they are actually starving their growth engine. Luna: But if you pull back on those awareness spends, don't you eventually run out of new people to retarget?
The pool has to dry up somewhere. Lucas: It does, and usually that's when the CPA spikes so hard that leadership panics and pulls the entire digital budget. The real solution isn't to guess which channels matter, it's to measure incrementality directly through controlled experiments rather than relying on correlation in aggregated data. Luna: That sounds expensive and complicated to set up properly without messing up the rest of your media buy.
Lucas: It used to be, but the infrastructure has gotten surprisingly accessible. You can now run what we call geo-lift tests or randomized holdout groups directly within major social platforms, isolating a specific population to see what would have happened anyway versus what happened because of the ad exposure. Luna: So instead of looking at total sales, you're comparing two identical markets where one group is completely blind to the campaign? Lucas: Precisely.
You take a region or a user segment, withhold the ads from half of them, and then measure the delta in conversion rates between the exposed group and the holdout. That delta is your true incremental lift, stripped of organic demand, seasonality, and baseline purchase behavior. Luna: And that tells you exactly how much revenue the ad actually created versus how much it just intercepted existing intent. Lucas: Right.
Let me give you a concrete example from a direct to consumer apparel brand we looked at recently. They were spending roughly five million dollars annually on broad social video placements, claiming a three point five return on ad spend based on last-click models. Luna: Three point five is pretty healthy by most standards, so why would they need to test that? Lucas: Because their CFO kept asking why customer acquisition costs were creeping up quarter over quarter despite the high ROAS.
When they ran a geo-lift test across six mid-sized cities, they found the actual incremental lift was only one point eight, meaning nearly half of those attributed sales would have happened anyway without the ad spend. Luna: That is a massive difference. So they were effectively burning fifty percent of their budget on capturing people who were already going to buy? Lucas: Yes, and worse, they were missing the fact that the same campaign was driving significant brand search volume that wasn't showing up in the app due to privacy restrictions.
The holdout group showed a twenty percent drop in branded organic searches, proving the ads were working, just not in the way the dashboard reported. Luna: I love that. It proves the channel works, but it exposes the measurement failure. How do you ensure the test is statistically valid though?
A small sample size could easily skew those results. Lucas: You have to calculate your required sample size before you launch. If you expect a two percent lift, you need enough impressions to reach statistical significance, usually defined as a ninety-five percent confidence interval with low variance. Most teams fail here because they stop the test as soon as it looks profitable, which is a classic gambler's fallacy.
Luna: They chase the early win and miss the long-term signal. But isn't there a risk that the holdout group just gets bored and buys from a competitor? Lucas: That is a real risk, especially in highly competitive categories. If the holdout converts less, it might be because they defected, not because the ad didn't persuade them.
That's why you also track engagement metrics and brand recall surveys alongside pure sales data to get a fuller picture. Luna: So you're mixing quantitative sales data with qualitative sentiment to triangulate the truth? Lucas: Exactly. Quantitative tells you what happened, qualitative helps you understand why the holdout behaved differently.
It turns a binary conversion metric into a nuanced view of customer journey dynamics. Luna: This feels like the kind of rigorous approach that separates mature marketing organizations from ones that are just guessing. Lucas: It really is. And the good news is you don't need a supercomputer to run these anymore.
Platforms like Meta and Google have built-in lift tools that automate the randomization and reporting, lowering the barrier to entry significantly. Luna: That makes it feasible for smaller budgets too, not just enterprise giants with millions in media spend. Lucas: Absolutely. Even a hundred thousand dollar test can yield actionable insights if designed correctly.
The key is treating it as an investment in knowledge rather than a cost of doing business. Luna: I'm curious about the timing aspect. Do you run these tests continuously or is it better to batch them? Lucas: Batching is usually smarter.
Run a four-week test, analyze the results, adjust your media mix, then run another test on a different channel or creative variant. Continuous testing can lead to fatigue and diminishing returns on the learning side. Luna: Four weeks seems like a reasonable window to capture typical purchase cycles for most consumer goods. Lucas: For fast-moving consumer goods, yes.
For high-ticket items like furniture or electronics, you might need six to eight weeks to capture the full consideration period. Context is everything when designing the experiment. Luna: It sounds like a lot of work to set up initially, but the payoff in clarity is huge compared to staring at a confusing dashboard. Lucas: Agreed.
Once you have that ground truth, you can allocate budget with confidence knowing exactly which channels drive net new demand versus which ones just harvest existing interest. Luna: That kind of certainty must make life much easier for finance teams who are always questioning marketing spend. Lucas: It transforms the conversation from defensive justification to strategic partnership. You bring data that speaks the language of causality, not just correlation.
Luna: Before we dive deeper into how to structure those initial tests, I want to pause for a moment. Lucas: Of course. What's on your mind regarding the show support angle? Luna: If these deep dives into measurement and analytics have helped you rethink how you evaluate your own campaigns, consider supporting the network directly.
Lucas: Listener contributions allow us to keep producing independent, ad-free research and analysis without corporate influence. You can join us at buy me a coffee dot com slash fexingo. Luna: It truly keeps the lights on and lets us dig into these complex topics freely. Thanks for considering it.
Lucas: Let's get back to the practical side of execution. When you start your first geo-lift test, you need to define your primary metric upfront. Luna: Is that just sales volume, or should you include something like average order value to account for quality differences? Lucas: Both are important.
Sales volume shows scale, but average order value can reveal if your ads are attracting premium customers versus bargain hunters. Ideally, you combine them into a single incremental revenue per impression metric. Luna: That gives you a clear efficiency number that accounts for both quantity and quality of the conversions. Lucas: Exactly.
It prevents the trap of chasing cheap clicks that result in low-margin sales while ignoring expensive clicks that bring in loyal, high-value buyers. Luna: It shifts the focus from vanity metrics to actual business health. Lucas: Which is ultimately what every boardroom cares about. Profitability, not just activity.
Luna: Do you recommend starting with broad awareness campaigns or performance-driven ones for the first incrementality test? Lucas: Start with performance. The lift is usually higher and more immediate, giving you quick wins and building internal credibility for the methodology before you tackle harder upper-fuel questions. Luna: Practical advice.
Prove the model works with the easiest data first. Lucas: Then once the team trusts the process, you can expand into brand lift studies and longer-horizon retention analysis. Luna: It feels like a logical progression from tactical optimization to strategic insight. Lucas: It is.
And it frees you from the tyranny of the last click, allowing you to see the whole machine working. Luna: What's the biggest mistake you see companies make when interpreting the lift data after the test ends? Lucas: Ignoring the confidence interval. They see a positive number and assume it's real, even if the margin of error is wide.
Statistical significance is non-negotiable for decision-making. Luna: A positive result that could be noise is just as dangerous as a negative result that could be signal. Lucas: Correct. Always check the p-value.
If it's above zero point zero five, you don't have a winner yet, you have a question mark. Luna: So you wait, gather more data, or redesign the test entirely? Lucas: Usually you wait, unless the sample size is clearly insufficient. Patience in analytics is often rewarded with precision.
Luna: It requires a different mindset than the instant gratification of daily reporting dashboards. Lucas: It does. But the clarity you gain is worth the extra time spent waiting for the numbers to settle. Luna: How do you handle seasonal fluctuations when running these tests?
If a holiday hits during the test window, doesn't that skew the holdout comparison? Lucas: Good point. Seasonality affects both groups equally if the randomization is sound, so the delta remains valid. However, extreme events can amplify variance, so you might need a larger sample size during peak seasons to maintain confidence.
Luna: So the math holds up, but the logistics require bigger buckets of data during busy periods. Lucas: Precisely. It's a minor adjustment, but one that ensures your conclusions aren't distorted by external noise. Luna: It sounds like incrementality testing is becoming the gold standard for serious marketing measurement.
Lucas: It is. As cookies disappear and privacy laws tighten, causal inference becomes the only reliable path left. Luna: A future where we actually know what our money buys, rather than just hoping it does. Lucas: Now that is a future worth investing in.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
The 3-Pillar Framework to Scale Any E-Commerce Brand (Feat. Abir Syed)Brain Driven Brands · on Incrementality testing90 / 100
A True Agentic Orchestration Platform for Hotel Operations | with Tim MajorGAIN Momentum · on confidence intervals87 / 100
How CMOs Are Using Data Clean Rooms for Privacy-First TargetingThe CMO Podcast with Fexingo · on Incrementality testing85 / 100
Retail Media In Your Ears: Inside Dollar General's 21,000-Store Audio NetworkFuture Commerce · on Incrementality testing83 / 100
230: Zero-click marketing broke the measurement layer, so what should ops teams do now, with Amanda NatividadHumans of Martech · on Incrementality testing83 / 100
#124 Why Authentic Content Beats the Algorithm | Yoray HalevyAlways Be Testing · on Last-click attribution83 / 100
How Brand Lift Studies Reveal True Marketing ImpactHow Offline Sales Drive Online Marketing ROIHow Contextual Targeting Replaces CookiesHow First-Party Data Reshapes AttributionThe Hidden Cost of Free Samples in Marketing