What should you do when your sample doesn’t match the known population composition on key variables like prior experience? One approach is to weight your data to rebalance the sample. In a previous article, we discussed how to weight means (such as from rating scales) when there are differences between group proportions in a sample […]| MeasuringU
In A/B testing sequential tests are gradually becoming the norm due to the increased efficiency and flexibility that they grant practitioners. In most practical scenarios sequential tests offer a balance of risks and rewards superior to that of an equivalent fixed sample test. Sequential monitoring achieves this superiority by trading statistical power for the ability to stop earlier on average under any true value of the primary metric.| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
I got a question today about our AGILE A/B testing calculator and the statistics behind it and realized that I’m yet to write a dedicated post explaining the efficiency gains from using the method in more detail. This despite the fact that these speed gains are clearly communicated and verified through simulation results presented in our AGILE statistical method white paper [1].| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
Sample size estimation is an important part of study planning. If the sample size is too small, the study will be underpowered, meaning it will be incapable of detecting sufficiently small differences as statistically significant. If the sample size is too large, the study will be inefficient and cost more than necessary.| MeasuringU
