Ab Test Sample Size Formula. Our AB test sample size calculator is powered by the formula behind our new Stats Engine which uses a two-tailed sequential likelihood ratio test with false discovery rate controls to calculate statistical significance. The null hypothesis being that there is no difference between the two landing pages.
Relationship between the levels of α-risk and β-risk. Putting all of that together the equation to determine what minimum sample size is required to show an accurate statistically significant improvement of CTR for the tested page with the orange button is. α is significance level Typically α 005 β is desired power Typically β 08 and are critical values for given parameters α and β.
We use here pooled estimator for variance assuming that.
N 1641435 0034 2 n 4791. The fastest way to help you identify your winners is to run a 1-tailed test. Typically for α005 equals 196 and for β08 is equal 084. Using alpha 005 and beta 020 applying the formula the required sample size is 1892 sessions per group.
