Bayesian Ab Testing Sample Size. See How Not To Run An AB Test for more context on the peeking problem and Simple Sequential AB Testing for a frequentist solution to the problem. To start lets go back to what a prior actually is in a Bayesian context.
Even though ab testing statistics might seem objective there are actually a number of opinions about the best way to interpret them. The bigger the sample size the more the Bayesian results align with the frequentist results. Bayesian AB testing with theory and code.
If they conduct an AB test with a 95 confidence and a 10 MDE then the required sample will be something like 263 visitors per variation.
By Antti Rasinen - the logical conclusion of an unfinished series of articles series Exact Bayesian Inference for AB testing. B rather than p-values and distributions over the parameter estimates rather than point estimates. The Power of Bayesian AB Testing Bayesian AB Test Duration. This simplifies the calculations.
