Building an experimentation system that actually works
Original: The Data Informed Experimentation System
Manuel Cellier
May 29, 2025
12 min read
Framework
Intermediate
Summary
Build your experimentation system to get new insights at a fast pace. Kickstart the compounding effect that will generate long term business outcomes and customer value.
Who This Is For
Product Managers
Product Analysts
Growth Marketers
Key Takeaways
- Learn how to structure a lean experimentation team with the right mix of PM, analyst, designer, and engineer
- Use Spotify's Thoughtful Execution Tree framework to focus on problems first, then solutions
- Set up systems to launch experiments quickly and compound learnings over time
- Understand that only 10-20% of experiments succeed, so volume and learning velocity matter most
- Build toolboxes for rapid implementation and robust measurement of experiment results
Tools & Technologies
A/B testing platforms Analytics tracking tools Spotify's Thoughtful Execution Tree framework Statistical analysis tools for causal inference
Topics Covered
experimentation a-b-testing causal-inference
Ready to dive deeper?
Read Full Article on growthscience.substack.com
