A/B testing your landing page videos

Hi Margot!

Thanks for sharing the article and your learnings with us.  Just wanted to chime in and share a mistake I have made in the past for those that use Wistia and want to A/B test their videos.  
Make sure to create a new video and be in control of the variable you are testing.  We were testing different video dimensions without realizing that the video was embedded elsewhere and in other uncontrolled environments…needless to say the results were lost in the averages.
Anecdote aside, the traffic we send/receive to our landing pages can cause big differences in play rate, engagement rate and ultimately, the conversion that you are offering.  A lot of the takeaways can get lost by not segmenting the niches and judging the success accordingly.  
Also do you have any insights on effective thumbnail testing for higher play rates?
Cheers,
Joe

Hey guys!

I’m also very interested in hearing some takeaways in re to video testing. Most interestingly the difference between animated videos vs. _human _videos. I would assume animations may be most appropriate when targeting those at the earlier stages of the buying decision process (via informational capsules).

@Joe , any thoughts? 

Awesome - thanks for pointing that out for me.

It’s hard to beat the face of a good ol’ human…

Hi Joe,

Thanks for the quick & detailed response! Makes total sense and aligns with my theory of viewer experience in reference to the established goal 😉

Your answer is:

Hey Margot (and errbody),

Thought I’d chime in with a video-on-landing-page test we’ve got in the works here at Unbounce.

In a recent campaign Ð where video acts as the heart-n’-soul of the content Ð we were wireframing the landing page design and asked the age ol’ important question…Òwhy are we designing it like this?Ó.

We’d planned on designing the page to feature a large full-screen type video that took up the first page section entirely (like we normally do when video’s involved), but then we thought, Òis full page-width really the best way to prompt a higher play rate? How about higher engagement?Ó (ie. the amount of people who watch the video all the way through).

So we set up an A/B test with the following hypothesis/question:

• Will people be more engaged , or will we see a difference in play rate , when we feature a video that’s displayed Òfull screenÓ in the width of a page section? OR with a video that’s displayed within a ÒcontainerÓ, similar to the container you see when watching a video on YouTube?

Essentially, we wanted to see weather classic YouTube style video sizing would make people more committed to watching a video (thereby watching it to the end), or whether one size or the other would make folks more likely to follow through with a CTA.

• When I say Òfull screen-widthÓ, I mean a video displayed like this, in a page section (with that prominent play button).

• And when I say ÒcontainerÓ, i mean something like when the video is featured in a rectangle, centered on the page, but doesn’t take up the full width of a page section.

Now, this experiment is still in play (and I don’t want to pollute the data by sending everyone to the variants in question), but I can say we’re collecting data and some very early results show that:

• As of a month in, our hypothesis seemed to be true regarding engagement, but not play rate. I.e. people were slightly more engaged with the video (watch it closer to its entirety) when it was shown to them in an embedded box container vs full-width across a page section. But both videos had (and continue to have) near equal play rates.

There may have been slightly lower engagement with full-width a month in, because (as I’ve rationalized it) visitors may have been tempted to view the rest of the content on the page instead of continue watching once they get the general idea.

• HOWEVER, a month in (and to date) people were more likely to follow through with the CTA on the page and convert when presented the full screen/full-width video.

The initial results we saw a month in seem to suggest that the way you feature your video on your landing page really should depend what you want as the goal of the campaign. I.e. do you want people to view the video in full? Or to follow through with your page’s CTA? This will vary per campaign (everyone’s goal will be different).

I did a quick check-in today and it looks like the engagement rate on the container video is still beating the full screen-style video (88% to 74% respectively).

The test is still inconclusive, but I can followup with more results as we have this campaign run longer if you guys are interested!

We’ll also be tying video plays to achieving the goal of the campaign so we can see how valuable a video play is in achieving our ultimate goal.

How do you feature video on your landing page when it’s the cornerstone of your campaign or message?

How about when you prioritize conversions vs. completion of the video? Does video competition matter in all cases if leads follow through with your intended goal?

Would love to hear your thoughts 🙂

Hey Margot (and errbody),

Thought I’d chime in with a video-on-landing-page test we’ve got in the works here at Unbounce.

In a recent campaign Ð where video acts as the heart-n’-soul of the content Ð we were wireframing the landing page design and asked the age ol’ important question…Òwhy are we designing it like this?Ó.

We’d planned on designing the page to feature a large full-screen type video that took up the first page section entirely (like we normally do when video’s involved), but then we thought, Òis full page-width really the best way to prompt a higher play rate? How about higher engagement?Ó (ie. the amount of people who watch the video all the way through).

So we set up an A/B test with the following hypothesis/question:

• Will people be more  engaged , or will we see a difference in  play rate , when we feature a video that’s displayed Òfull screenÓ in the width of a page section? OR with a video that’s displayed within a ÒcontainerÓ, similar to the container you see when watching a video on YouTube?

Essentially, we wanted to see weather classic YouTube style video sizing would make people more committed to watching a video (thereby watching it to the end), or whether one size or the other would make folks more likely to follow through with a CTA.

• When I say Òfull screen-widthÓ, I mean a video displayed like this, in a page section (with that prominent play button).

• And when I say ÒcontainerÓ, i mean something like when the video is featured in a rectangle, centered on the page, but doesn’t take up the full width of a page section.

Now, this experiment is still in play (and I don’t want to pollute the data by sending everyone to the variants in question), but I can say we’re collecting data and some very early results show that:

• As of a month in,  our hypothesis seemed to be true regarding engagement, but not play rate. I.e.people were slightly more engaged with the video (watch it closer to its entirety) when it was shown to them in an embedded box container vs full-width across a page section. But both videos had (and continue to have) near equal play rates.

There may have been slightly lower engagement with full-width a month in, because (as I’ve rationalized it) visitors may have been tempted to view the rest of the content on the page instead of continue watching once they get the general idea.

• HOWEVER, a month in (and to date)  people were more likely to follow through with the CTA on the page and convert when presented the full screen/full-width video.

The initial results we saw a month in seem to suggest that the way you feature your video on your landing page really should depend what you want as the goal of the campaign. I.e. do you want people to view the video in full? Or to follow through with your page’s CTA? This will vary per campaign (everyone’s goal will be different).

I did a quick check-in today and it looks like the engagement rate on the container video is still beating the full screen-style video (88% to 74% respectively).

The test is still inconclusive, but I can followup with more results as we have this campaign run longer if you guys are interested!

We’ll also be tying video plays to achieving the goal of the campaign so we can see how valuable a video play is in achieving our ultimate goal.

How do you feature video on your landing page when it’s the cornerstone of your campaign or message? 

How about when you prioritize conversions vs. completion of the video? Does video competition matter in all cases if leads follow through with your intended goal?

Would love to hear your thoughts 🙂

That’s cool, Margot. In the past, I’d only ever done the container style video, similar to you guys. I’m partial to it above the fold on the right on landing page, and it seems to be what audiences are familiar with, so I wonder if there are isolated cases (maybe event pages?) for which the full screen is especially ‘epic’ or contributes to conversion?
I liked the full-width one you guys did for this year’s Wistia fest, for example. Seems to fit the epic conference vibe having the video be so prominent for an announcement.