About the AB Testing Beta category


#1

Beta AB Testing, Scheduling & City Targeting for Sticky Bars and Popups


Hey everyone! :wave:

You all reached out looking for early access to AB testing. Well I’m happy to say it’s finally here!

You should now have AB Testing available in your accounts for sticky bars and popups. It follows the same work flow as for landing pages but there are a couple of things to note that are outlined here. The most important of these is that only content can be AB tested, not url targeting, frequency or advanced targeting options.

As well as AB testing, anyone with a premium account has also been given early access to scheduling and city targeting. Scheduling gives you the ability to time box your campaign while city targeting allows for greater granularity in location targeting.

We’d loved to hear all about your experience using these new features in the comments below. :point_down:


#2

WOOOOOOOOOOOOOOOOOOOOO!!! :tada: :tada: :tada: :tada: :tada: :tada:


#3

This is awesome. The new scheduling feature is even more awesome! I look forward to finally start implmenting some of my a/b tests that I haev planned for convertables!


#4

This is incredibly cool. :star_struck:

Any chance there are any future plans to implement A/B testing of triggers/frequency/etc.? It’d be really cool if we could run a test, for example, against the triggers that have variable units (10 second delay vs 20 second delay, or 25% down the page vs 50% down the page, etc.).


#5

Hey @leah.ann,
That is definitely something that has been and is still being considered. There are a number of other improvements currently in progress to enhance AB testing for landing pages, sticky bars and popups that will be coming soon. These improvements are the current priority but based on the feedback we see from this release adding AB testing of triggers/frequency/etc. may increase in priority.

Hope this helps for now,
Brian


#6

Thank you for the response and sorry for the delay in mine! I’ve been in holiday mode. :slight_smile:

Totally understand that something like this might not be top priority with all the other cool things you guys are working on. I’m just happy to hear that this is a feature being considered and something that we can potentially look forward to as a future release!

I have another question for you:

If I’m integrating Popups with Google Analytics, do the events fire as interaction or non-interaction events, or are they considered interaction in some triggers (e.g. element click trigger) and non-interaction in other triggers (e.g. time delay trigger)?

What I’m ultimately wondering is if I can accurately look at my bounce rate data in Analytics when comparing popup variants.

For example, let’s say I have two variants of a popup with my trigger being a 10 second delay. I would hope that the “view” event in this instance gets sent to Analytics as a non-interaction event. if a visitor lands on my page and makes no other interaction with the page, sees the popup after 10 seconds, and immediately leaves, I would think that should be considered a bounce. This way, I could test to see if one of my popup variants is potentially more likely to immediately deter visitors than the other, or if one is potentially more likely to encourage them to interact further with my landing page, without the popup itself counting as an interaction (as that would artificially deflate my bounce rate).

However, if instead I have a popup trigger that is based on the click of an element, I would think that should be considered an interaction event.

Hope I’m articulating that well enough!


#7

Hey @leah.ann,

You bring up an interesting point. The current behaviour is for a “view” event to be fired once a visitor sees the popup, regardless of the trigger, and to fire a “conversion” event when a visitor converts on the popup.

The Google Analytics integration was a first iteration that only deals with events on the actual popup rather than any interaction on the host page. We’re happy to hear this kind of feedback and it’s something I’ll bring to the team.

While I’m not the most familiar with Google Analytics, I believe there is the ability to combine events into segments, e.g. watch for a popup view event and then a further interaction. Would that possibly address this?

Hope this helps for now and thanks again for the feedback.

Brian


#8

It might be even simpler than that (or more complicated, depending on how all of this is implemented on your end haha). There’s an object that you can attach right to the event when it fires to send it either as an interaction or non-interaction. There’d have to be some kind of logic that checks for the type of popup trigger though. Something like:

if ( popupTriggered == true && triggerType == elementClick ) {
  ga('send', 'event', 'Unbounce Popups & Sticky Bars', 'view', 'ID', {
    nonInteraction: false    // This sends the event as an interaction event, so it will affect bounce rate
  });
} else if ( popupTriggered == true && triggerType == secondDelay ) {
  ga('send', 'event', 'Unbounce Popups & Sticky Bars', 'view', 'ID', {
    nonInteraction: true    // This sends the event as a non-interaction event, so it will not affect bounce rate
  });
}

It would essentially be use-case logic that either sends the “view” event as interaction or non-interaction depending on the type of trigger that is being implemented. Since some triggers would be considered interactions and others wouldn’t be, I think that might be a better solution than to segment the event. You wouldn’t necessarily want to check for a second interaction after an element click trigger, for example, as the element click itself is an interaction.

What possibly complicates things is whether or not you can check for the type of trigger from within the popup script itself (or wherever it is that you’re firing the popup events). If you can do that, it should be a fairly easy implementation. If you can’t, and that’s only accessible from within the host page scripts, that makes things a bit more complicated.


#9

Ah, yes. That is totally doable on our end. Having looked into this in more detail I see we’re currently sending all events with “nonInteraction” set to true. It wouldn’t be a major change for us to handle checking for the trigger type. I’ll create a ticket for us to look into this further.

Thank you for the incredibly detailed response and I’ll do my best to keep you posted on this!

Regards,
Brian.


#10

Awesome, that’s helpful to know and makes sense since all but one of the triggers don’t necessitate an interaction.

Thanks!


#11

Chiming in very late here, but wanted to share my experience on setting up an A/B test. The new engine seems more robust than the old one, and it’s very straight forward!

And that I can assign labels to the events is super useful. My only gripe is that the convertables are still not using the custom domain, so I would have to define segments by events instead.

A typical segment scenario in GA looks like this


I then get a view where I can compare sessions and conversions (based on my goal) in GA.

It gets slightly more complicated if I want to track how many has clicked the CTA on a certain popup and then converted on a second page.


#12

Hey @leah.ann,

Just wanted to update that the change you requested has now been made. Click trigger and conversion events are sent as interaction events while all other triggers are sent as non-interaction events.


#13

Oh wow, I totally wasn’t even expecting anything to come of this, but I’m glad to hear that the change has been made! This will make bounce rate analysis of popup A/B tests that much more reliable, especially with the click trigger. Thanks for the update! :raised_hands: