The Importance of Statistical Power in Online A/B Testing | Analytics-Toolkit.com
What is Statistical Power?| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
Run split tests faster, more efficiently and with better accuracy! The A/B testing significance calculator provides an advanced statistical approach to A/B and Multivariate testing in Conversion Rate Optimization, landing page optimization, e-mail template optimization, mobile app optimization and more. With AGILE A/B testing you get control over statistical significance and power while doing interim analysis and requiring less users to complete tests, on average. This A/B test calculator als...| www.analytics-toolkit.com
What is Statistical Power?| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
I got a question today about our AGILE A/B testing calculator and the statistics behind it and realized that I’m yet to write a dedicated post explaining the efficiency gains from using the method in more detail. This despite the fact that these speed gains are clearly communicated and verified through simulation results presented in our AGILE statistical method white paper [1].| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
What is the goal of A/B testing? How long should I run a test for? Is it better to run many quick tests, or one long one? How do I know when is a good time to stop testing? How do I choose the significance threshold for a test? Is there something special about 95%? Does it make sense to run tests at 50% significance? How about 5%? What is the cost of adding more variants to test?| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
After many months of statistical research and development we are happy to announce two major releases that we believe have the potential to reshape statistical practice in the area of A/B testing by substantially increasing the accuracy, efficiency and ultimately return on investment of all kinds of A/B testing efforts in online marketing: a free white paper and a statistical calculator for A/B testing practitioners. In this post we’ll cover briefly the need for a new method, some highligh...| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
This is a comprehensive guide to the different types of costs and benefits, risks and rewards related to A/B testing. Understanding them in detail should be valuable to A/B testers and businesses considering whether to engage in A/B testing or not, what to A/B test and what not to test, etc. As far as I am aware, this is the first attempt to systematically review all the different factors contributing to the return on investment from the process of A/B testing. Here I will cover A/B testing m...| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
The question of whether one should run A/B tests (a.k.a online controlled experiments) using one-tailed versus two-tailed tests of significance was something I didn’t even consider important, as I thought the answer (one-tailed) was so self-evident that no discussion was necessary.| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
The p-value is the most commonly used statistic in scientific papers and applied statistical analyses. Learn what its definition is, how to interpret it and how to calculate statistical significance if you are performing statistical tests of hypotheses. The utility, interpretation, and common misinterpretations of observed p-values and significance levels are illustrated with examples.| GIGAcalculator Articles
Learn the meaning of p-value in the context of A/B testing, a.k.a. online controlled experiments and conversion rate optimization. Detailed definition of p-value, related reading, examples. Glossary of split testing terms.| www.analytics-toolkit.com
The one-stop-shop for statistical planning and analysis of online A/B tests. Analytics Toolkit's advanced A/B test statistical calculator enable your A/B testing program to reach new levels of statistical rigor and efficiency. Plan and analyze A/B tests with ease and get results you can trust.| www.analytics-toolkit.com
Navigating the maze of A/B testing statistics can be challenging. This is especially true for those new to statistics and probability. One reason is the obscure terminology popping up in every other sentence. Another is that the writings can be vague, conflicting, incomplete, or simply wrong, depending on the source. Articles sprinkled with advanced math, calculus equations, and poorly-labeled graphs represent a major hurdle for newcomers.| Blog for Web Analytics, Statistics and Data-Driven Internet Marketing | Analy...
