the analytics vault
Resources: 136 (72 new this week)
a Hipster Data Club product Hipster Data Club

When A/B tests just won't work: BBC's clever approach to measuring marketing impact

Original: Using Causal Inference for Measuring Marketing Impact: How BBC Studios Utilises Geo Holdouts and…

Frank Hopkins
May 29, 2025
7 min read
Case Study
Advanced
Using Causal Inference for Measuring Marketing Impact: How BBC Studios Utilises Geo Holdouts and…

Summary

This article discusses how BBC Studios utilizes geo holdouts and causal inference techniques like Propensity Score Analysis (PSA) to accurately measure the marketing impact of their campaigns. It provides a detailed overview of the methodologies and challenges involved in conducting rigorous lift tests to establish causality.

Who This Is For

Marketing Analysts
Data Scientists
Performance Marketers

Key Takeaways

  • Learn how to use geo holdouts when traditional A/B testing isn't feasible for out-of-home and multi-channel campaigns
  • Understand how Bayesian Synthetic Control methods create more robust counterfactuals than traditional approaches
  • Discover practical techniques for selecting control regions based on demographics and historical similarity
  • See how CausalPy can be implemented to measure incremental marketing impact with confidence intervals

Tools & Technologies

CausalPy Bayesian Synthetic Control Methods Python Statistical modeling frameworks

Topics Covered

causal-inference geo-holdouts lift-tests psas quasi-experimental

Ready to dive deeper?

Read Full Article on medium.com