Learning path

Read test results without fooling yourself

Know when a result is real and what to decide.

Who it's for

Anyone who reads or reports A/B test results.

What you'll be able to do

• Explain significance, power and confidence intervals
• Spot peeking, sample ratio mismatch and novelty effects
• Protect revenue with guardrail metrics
• Measure incrementality with holdouts

The steps

1. A/B Testing Statistics for Marketers: The Models, When to Use Each, and How AI Can Help
2. Podcast
3. 10 A/B Testing Mistakes That Fake a Result (and How to Catch Them)
4. Guardrail Metrics in A/B Testing: How to Choose Them and When to Stop a Test
5. Measuring Personalisation ROI: Holdouts, Incrementality and Honest Attribution