Variant and A/B Testing

  • 2 October 2010
  • 5 replies
  • 7 views

A/B Testing I’m not getting the second test page to display even after publishing it, any suggestions? Even after I promoted the second variation to the champion (and republishing it) its, still displaying the old page?


5 replies

Hi Joshua,
Is the variant that you’re seeing a part of the test still (is it a challenger receiving a percentage of the traffic?) As long as the variant you were first cookied with is receiving any traffic at all in the test, you will always see that variant regardless of any demotions/promotions of other variants.

If the variant you’re seeing is not supposed to be receiving any traffic, then it sounds like something else is wrong. In that case, can you send us an email at support@unbounce.com with the page URL and the variant that you are still seeing?

Cheers,
Carter

Let me see if I can explain this, I went in and added a variant to a published ad, so this created an A/B test. After re-publishing the file I dont ever see the second variant when looking at the ad. Nor do I see traffic being recieved for the second variant created.

Hi Joshua,
Did you give the second variant a traffic weight before you published it? If so, how is the traffic split between your A and B variants, and how much traffic have you had sense you re-published?

The other thing to note is that refreshing will not serve up a random variant each time. The first time you (or your visitors) see your page, they are given a variant, randomly chosen based on the traffic weights that you’ve assigned. After that, they will always see that one variant as long as it exists in the test. If you discard it ever, then the visitor will again be served up a new random variant. Does that make sense?

If you want to test out both variants, try deleting your browser cookies in between hitting refresh.

Cheers,
Carter

I gotcha, so I will always see the same variant. You would think you would want that to change incase the I wasnt interested in the first variant and came back and the second approach sold me. I guess if your testing which variant would be ideal you would want that to change based on the refresh not based on the unique user. Thanks

Glad that helped clear things up a bit. And yes, in order to run a controlled experiment, you wouldn’t want a user to be able to cycle through all the variations of your page, plus it’s not often you even get a second chance with users after they’ve bounced in the first place!

Reply