A/B testing question - If set to 50/50 at what sample size is that actual


i am learning to AB test. I set up 2 variants and set 50% to each in terms of weight. The first 2 hits that came went to the the first variant - why is that? I thought it would take turns since its 50% each?


Hi Valentine, good question!

This article sums up how we do our page variant selection for split tests:


To sum up, when you assign weights to each variant, you’re assigning the chance or probability that that variant will be assigned to a particular visitor. Every time a new visitor arrives at your page, we randomly select a variant based on that weighting. But it’s the same thing as flipping a coin or rolling dice. With a coin toss, every time you flip the coin, there’s a 50/50 chance of it coming up heads (or tails). But you can still get two or three heads in a row before tails comes up.

Does that help explain it?