I hope you had a good weekend. It was a long one here, so that’s why this newsletter is a bit late. But the setup is the same: a curated overview of interesting reads, events and jobs for the experimental mind (that’s you).
Everything is handpicked by me, so you don’t have to. Enjoy!
🔎 What I’ve been reading
Picking Metrics 101: do’s & don’ts — blog.statsig.com
Picking the right metrics for an experiment is not easy. The Statsig team is giving some good advice, including clear do’s and don’ts.
Paper: A/B Testing Intuition Busters
Ronny Kohavi, Alex Deng and Lukas Vermeer have written a paper on Common Misunderstandings in Online Controlled Experiments: A/B Testing Intuition Busters. They are still open for review and comments.
A/B Testing: Ruling Out To Conclude — conductrics.com
Matt Gershoff explains two important concepts from A/B testing.
There are two things we are trying to rule out when we run AB Tests: confounding & sampling variability/error.
A Survey of Causal Inference Applications at Netflix
Netflix relies heavily on A/B testing to measure causal effects. In this blog post we get a peek into the recent held internal Causal Inference and Experimentation Summit.
The weeklong conference brought speakers from across the content, product, and member experience teams to learn about methodological developments and applications in estimating causal effects. We covered a wide range of topics including difference-in-difference estimation, double machine learning, Bayesian AB testing, and causal inference in recommender systems among many others.
🚀 Job opportunities
All featured roles:
- Conversie Optimalisatie Specialist at Kaartje2go (Zwolle, NL)
- Conversie Optimalisatie Lead at Kaartje2go (Zwolle, NL)
- CRO Specialist at Randstad Global (Amsterdam, NL)
- Digital Analyst at RTL (Hilversum, NL)
- Webanalist at Kaartje2go (Zwolle, NL)
Also checkout the job board for more open roles in the field of experimentation, CRO and analytics.
📅 Upcoming events
Experimentation Culture Awards 2022 — experimentationcultureawards.com
I’ll be one of the jury members at the 2022 Experimentation Culture Awards. Join the award session and listen to inspiration stories.
All upcoming events in the next months:
- ⏩ 8-9 June: Advances with Field Experiments (Chicago)
- ⏩ 9 June: DiDo#39 Ga voor (CRO) goud! (Utrecht)
- ⏩ 9-10 June: NeuroPsychoEconomics (Lille)
- 15 June: Growth Marketing SUMMIT (Frankfurt am Main)
- 16-18 June: Marketing Science Conference (virtual)
- 25 June: MeasureCamp (Paris)
- 7 July: Experimentation Culture Awards (virtual)
- 4-6 Aug: Digital Elite Camp (Estonia)
- 🆕 22 Aug: Accelerating Innovation with A/B Testing (virtual)
Second batch of tickets will release soon:
- 🎫 18-20 Nov: Conversion Hotel (Texel, Netherlands)
See the full overview of events to find other events and conferences for you and your team.
💬 Quote of the week
“The old is dead, and I don’t know what the new is. The only way to find the new is to start different things and see if there’s something that can come out of experimentation.” — David Lynch
😉 Fun of the week
The latest Marketoonist on The Innovation Funnel:
… this Darwinian process of bringing ideas to life doesn’t necessarily lead to survival of the fittest ideas. If we’re not careful, the innovation funnel leads to survival of the safest ideas.
☕ Thanks for reading
If you’re enjoying the Experimental Mind newsletter and want to say thank you? Buy me a coffee (or a beer). Or share this newsletter with a friend or colleague.
Have a great week — and keep experimenting.
Kevin
