Views validity and amount of views to get conversions


#1

Hello everyone. I have this issue with my A/B testing. So I have created 5 options with different company names and apart from that everything is the same. So far I got 80 views and I have two problems - first of all I put the weight 20% to each of them so I get the same amount of views. However I have very unstable amount of views for each page - 12, 24, 15, 7 for each page. I know it’s random but it seems like it’s diverse too much. Or it’s not random and it means that some of my pages have higher chance of success? Yes I read the manual but these results seems to be strange.

Seconds of all do I need more traffic to my page to have the validity and conversions? So far I got 0 conversions from over 80 visitors, maybe has to do something with having a shop online but on the landing page I’m just offering to subscribe because I don’t have an actual shop yet?

Or do I need just way many more visitors to get any conversions at all?
(in which case it’s gonna cost a lot for marketing PPC campaigns and yes I already read the article about optimizing my PPC campaign and changed it to a specific keywords campaigns)

Thank you for any advice!


#2

5 options for A/B testing is far too much. Test only two ; then test the best of those with another one and so on to the end of your 5 existing ones.


#3

Hi Josef, 

First, welcome to the Unbounce community. 

Straight to your questions:

  1. The distribution per variation is indeed random however the sample size you have is relatively small.
    The system needs time to balance out the views per variation.
    Once you reach more views, the break down per variation would be more or less the same. Keeping in mind they would never be the exact same number

  2. Yes, you do need more visitors before drawing any conclusions from your landing page. 

Flora is right that 5 variations with limited traffic might be a bit too much. 

Start with 1 original and 1 variation for a total of 2 pages. 

A general rule of thumb, that might change depending on your targeting and traffic quality, is to have at least a few hundred visits per variation before digging into the data. Again that is highly dependent on your particular circumstances but a good rule to keep in mind. 

A few more tips:

  • Don’t forget to exclude your IP address (home and office) from the statistics by going to Settings > IP Filters. (Also, exclude anyone else who might be working on the page)

  • Make sure your PPC campaigns are really well targetted and that your landing page offers something of value to potential leads. 

  • Per your original post… Changing just the company name on each variation is relatively small change and thus would need a lot of traffic to detect a statistically significant change if there is one to detect!

Best,
Hristian


#4

Hi Josef. Everything Flora and Hristian suggested is spot on. I just wanted to add, that when it comes to A/B tests, you want to make sure you’re at a point where your results are “statistically significant.” This calculator is very helpful in determining how long you’ll need to run your tests: https://vwo.com/ab-split-test-duration/


#5

Thanks all for your answers! That’s really helpful and helps me understand this better.

To Nicholas: It says I need 650000 more days so I definitely need to increase the traffic.

I have already reduced the variants to 2 and will try to drive more traffic to the pages, thanks!