MOST WINNING A/B TEST RESULTS ARE ILLUSORY [PDF]

March 3, 2014

a/b testing / design / marketing

I’ll introduce three simple concepts which come as second nature to statisticians but have been
forgotten by many web A/B testing specialists. The names of these concepts are ‘statistical power’, ‘multiple
testing’ and ‘regression to the mean’. Armed with them, you will be able to cut through the misinformation and
confusion that plague this industry.