Notes d’épisode
Etsy and UC Berkeley stack 23 screened in-experiment covariates on top of CUPAC, cutting A/B-test variance a further ~2.5–14% across 29 experiments.
Key takeaways:
• Two stages. Stage 1 is Etsy's unchanged CUPAC (one LightGBM model, 117 pre-experiment covariates). Stage 2 is a linear adjustment on 23 in-experiment covariates, fit to CUPAC's residuals.
• Mean equivalence, not independence. A post-treatment covariate is usable if its mean matches across arms. That's testable; if it fails by Δ, the bias is −γᵀΔ.
• What Etsy ran. Drop too-sparse count variables, Mann–Whitney U per experiment…
Site web: rss.com

