Testing a popup against...no pop up

S

I want to test the effect of a pop up as a device meaning vs no popup at all.
Any ideas how to do this? Can a variant be no pop?

11 replies

Brian_Burns
Rank

Hey @sgringer,

That’s an interesting question. This is a landing page A/B test rather than a popup A/B test.

To achieve this, create two variants of the landing page and only add the popup embed script to one of the variants. Then the popup will only show to visitors who hit the variant that has the embed script and you can compare stats. Note, if you’ve got a conversion goal on your popup you’ll want to include the popup’s stats in your final analysis.

Let me know if this answers your question or if there’s anything else I can help with.

S

That was how I was going to do it. Just wanted to try and test multiple variants of the pop up concurrently. Maybe a feature request?

Brian_Burns
Rank

Hey @sgringer,

That will still be possible. You can set up multiple variants of the popup and they will show on the page variant with the embed script. This would be the setup I’d envision:

Host page: 2 variants, A: 50% traffic, B: 50% traffic.
Popup: 2 variants, A: 50% traffic, B: 50% traffic.
The result would be that 50% of visitors wouldn’t see a popup while 25% would see variant A of the popup and 25% variant B.

As for the feature request, I’m unsure how this would work. To compare a page with a popup vs one without as part of the popup’s A/B testing would require using stats from the host page. Since popups and sticky bars can be used on pages outside of Unbounce landing pages this isn’t something we could do. Am I understanding your request correctly?

S

You are correct and I had that same a-ha moment after I posted.

Thanks!!

A
Badge

I still think it makes sense for Unbounce to consider adding as a feature!

We have an overlay that can trigger on a number of pages on the Wordpress side of our site, so the workaround presented here isn’t an option. We tried to randomly assign a cookie to 50% of traffic and trigger overlay based on that, but that didn’t work.

@Brian_Burns, outside of this feature (maybe one day) being built into the product, any suggestions for how we can accomplish this now?

Brian_Burns
Rank

Hi @Andy2,

I understand where you’re coming from. Is the feature you’re looking for just an option to “show to X percentage of visitors”? That would be an easier feature to add but it doesn’t allow you to perform a true A/B test, which was the original question here.

You can accomplish that behaviour using the script below. Place it on the host pages. Note that if you’ve more than one popup or sticky bar on the page the script will need to be changed. Change the “percentageOfVisitorsToShowTo” variable to the value you wish to use.

<script>
 window._ubeConfig = window._ubeConfig || {};
 window._ubeConfig.shouldShowOverlay = function() {
   var percentageOfVisitorsToShowTo = 50;
   
   var random = Math.floor(Math.random() * 11);
   if (random < (percentageOfVisitorsToShowTo / 10)) 
     return true;
   return false;
 }
</script>

Please let me know if this works for you or if you’ve any other questions.

A
Badge

Hey @Brian_Burns thanks very much!! I can definitely give that a try.

Of course, in order to test running the overlay just some of the time, the last piece of the puzzle is to measure results for each audience. In other words, if the overlay is eligible to show 50% of the time, how can I see results for each segment of traffic? The question we’re trying to understand is, how much (if at all) does the overlay help?

Alex_Scovell

I think for this you have to control it from the LPs - with any popups set to show 100%.

So to show two pop variants have three LP variants:

LPA - no pop
LPB - pop A
LPC - pop B

Results is LP impressions vs (LP conversions + pop conversions)

It’s very possible i’ve missed a point here.

A
Badge

Thanks @Alex_Scovell. That was the solution for the original question from @sgringer, but I’m looking to run pop-ups across a number of pages on the Wordpress side of our site. So, unfortunately, not possible to run a test like that.

Alex_Scovell

hmm - i’ll leave that one for someone more technical!

I guess you need to pick up whether it’s being shown or not on the page each time and then pass that info to a tracking system (whilst also forwarded on with the user to be included against conversion tracking also).

I think there will be nothing helpful in the script of the loaded page - i don’t know if it may be possible to pick something up from the script being loaded - once you have the info (i.e is this in the 50% being shown the pop) getting it to the tracking systems is doable.

Sorry - not my most helpful post 😦

Alex_Scovell

if you use something like hasoffers, you could create a clone of of the site and route 50% to site, 50% to clone and then has offers is very cable of tracking results.

You would need to use a tracking link - on paid media very doable, on natural very not doable!

i think there must be a better solution (unless you already use something like hasoffers).

Reply


Terms & ConditionsCookie settings