Aller au contenu
Global sound
Data Science in the Wild

Data Science in the Wild

Etsy — Using Post-Treatment Data to Sharpen A/B Tests

· 21 min · Saison 2 Épisode 2

Notes d’épisode

Etsy and UC Berkeley stack 23 screened in-experiment covariates on top of CUPAC, cutting A/B-test variance a further ~2.5–14% across 29 experiments.
Key takeaways:

• Two stages. Stage 1 is Etsy's unchanged CUPAC (one LightGBM model, 117 pre-experiment covariates). Stage 2 is a linear adjustment on 23 in-experiment covariates, fit to CUPAC's residuals.
• Mean equivalence, not independence. A post-treatment covariate is usable if its mean matches across arms. That's testable; if it fails by Δ, the bias is −γᵀΔ.
• What Etsy ran. Drop too-sparse count variables, Mann–Whitney U per experiment…

Site web: rss.com

En direct, dernière heure : 7 514 titres sur 3 129 stations dans 128 pays

Découvrez le monde par la radio

Plus de façons de découvrir (9)

Vous avez entendu quelque chose de bien ? Enregistrez-le, ou ouvrez-le directement dans Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
XFresh FMContinuer l’écoute