Variant Testing for Mobile versus Desktop


#1

I watch conversions based on device with all of our ads, but I wondered if there was a way to gleam “confidence” between more than just variants in Unbounce. I was wondering if there might be a way to get the confidence level that is already available between variants translated into a confidence between devices as well?

Most ad services as well as Google Analytics provide some insight into devices, but they don’t easily report separate variants and more importantly the confidence level of when to “champion” a variant.

So basically, it would be great to be able to check Unbounce for a quick/high-level snapshot report that says something like “Mobile performs 40% better than Desktop with a 90% confidence.” That way I can focus more on the better performing devices and turn on/off certain ad deliveries/traffic sources. Knowing the conversion rates AND confidence of mobile over desktop along side each variant would help me optimize both the pages AND ads.

A simple solution to this might be to elaborate the conversion stats on the dashboard/overview page. If each variant reported the desktop AND mobile conversion rates separately instead of averaged/combined, then I would be able to see either mobile versus desktop along side which variant is performing the best. With this knowledge I could learn that variant A’s mobile version is doing better than variant B’s. And overall, mobile is beating desktop… and so on.

Is there a solution like this already in Unbounce or is this a future feature? I know it would be valuable for us. Thank you!


#2

Hi @Rudydo,

As a general rule, you should always calculate your sample size outside of Unbounce.

Google Analytics can provide you with all the data you need to calculate confidence levels per device, browser, OS, etc.

However, confidence level alone should not be used as a stopping rule for your A/B tests.

Best,
Hristian