Did you ever wonder how can you easily test multiple variations of the same single page?
Something that will work when testing small page variations just to get information about how each small change performs.
Of course, you can create separate landing pages with slight variations, then split-test traffic between them in order to see which ones perform better.
But let me tell you this: there is an easier way to do multivariate testing with a single physical landing page, without generating more pages.
All you need is a multivariate testing tool which can rotate variations directly in the page, then report results back to the tracker.
Case Study
Let’s say that you have a landing page for which you want to test 3 headlines, 2 images and 3 buttons (CTA).
This means that you will have to generate 3x2x3 = 18 different pages to test all these variations.
But what if there was a way to test all these 18 page variations within a single landing page?
You would probably think that it must be complicated or that it must require coding skills…
Well, this is super-easy and it can be done with no coding at all in MV Lab.
We did a small case study on: how to use MV Lab and see how each small variation of a page improves the CTR.
You will see here all the steps we did, with a real campaign in order to show how MV Lab can be used and what the multivariate testing results were after 2 days tests.
What we were looking for specifically was how to track multivariate results and the CTR of different page versions.
This campaign was not about making $$$, it was just for learning multivariate testing in an easy way. 😀
So our focus was on the landing page and how to adjust it based on the metrics tracked.
Information about our campaign:
🔧 Tracking Tool: CPV Lab Pro + MV Lab
✅ Affiliate Network: Traffic Company
🎯 Traffic Source: PropellerAds
👍 Type of Offer: not great, not terrible
Landing Page Setup
The initial landing page was pretty simple and it looked like this:
We wanted to see how small changes in this page will improve CTR and we planed on performing multivariate tests for the following items:
- Subtitle text – test 3 variations: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “YOUR LOCATION”
- Background image – test 2 variations: with background image and without background image
- Button color – test 3 colors: “LawnGreen”, “Yellow, “Red”. These 3 will be the snippets from my page that I will want to rotate multiple variations for.
The difficult approach involved generating multiple pages, one for each combination, but as I mentioned before this will lead to 3x3x2 = 18 pages to be tested.
So let’s see how the MV Lab approach is and how much time this saved us.
There are a few requirements for using MV Lab for multivariate testing:
- You need to have access to the page HTML code, so you need a page that is hosted on your server and not a page from a landing page builder tool (but , I know this from the inside 😼, a version compatible with WordPress is being prepared)
- You need a PHP page – if you have a page with *.htm or *html extension, then you can simply rename it to have a *.php extension
- The mvlab.php file generated in your MV Lab account should be uploaded on the server, in the same location as your landing page
- CPV Lab Pro as the tracker in order to see the results of your tests directly in the tracker reports
The campaign was created in CPV Lab Pro just as any regular landing page + offer campaign.
There was one extra thing to do in the campaign: to setup one extra token for each of my snippets in the multivariate test.
These extra tokens will receive information about the variations displayed for each visitor.
Using the visual editor for MV Lab included in CPV Lab Pro, we have defined each snippet and the variations that we want to be rotated for it.
It also allows to set a Share (number of visitors) for each variation, but for now the Shares were left blank in order to have traffic distributed evenly.
The code that needs to be included in the Landing page was generated from the MV Lab Editor from CPV Lab Pro.
We’ve copied the code and pasted it at the beginning of the HTML code in the landing page, before the tag and any other tags. This code will quickly execute before the page is loaded and set the variations for each visitor.
Also we defined the places where we wanted the variations to be rotated inside the landing page.
I will skip that technical part here, as my focus is on the results. 🙂
After configuring MV Lab in the landing page, all we had to do is to upload the landing page on the server and it was ready for traffic.
You can see the page at the URL below. Refresh the page several times to see how different variations will appear in the 3 snippets with each page refresh… this is how the page variations will be rotated between the visitors (and yes, the url has a token to help me count how many times you clicked it 😀)
https://win-money-online.net/lucky-tiger-promo.php?src=blog
First Results
After running a few hours of traffic in this campaign, we had the first results.
As mentioned before, our focus was to increase the page CTR.
The initial overall results of the page from CPV Lab Pro are as follows:
So, the page got a 8.33% CTR across all variations. Not too good… but, hey, this is testing here!
I will dig deeper and see how the multivariate test went.
- Let’s look first at the first snippet which rotated 3 texts for the subtitle:
- Black Friday
- Next 10 minutes
- Your Location
This brings some useful information into what happened with the landing page.
As you can see, 2 of the Subtitle variations were getting higher than average CTR, while the third variation wasn’t performing at all.
Subtitle 3 is a certain candidate for removal from the landing page.
2. Let’s look now at the second snippet: the background of the page.
It was tested in 2 variations: with and without a background image.
Similar with snippet 1, we have a clear underperforming variation here: the one without a background image (var2-black).
Not as drastically, but still pretty big difference between the 2 variations from here.
3. Let’s check now the third snippet: the button background color.
This was tested in 3 variations: “LawnGreen”, “Yellow, “Red”.
The CTR for each of the 3 variations for the button background is pretty similar, there is a very small variation here.
This means that the button background color didn’t really have an influence of the clicks received in the page, so we need to focus on the other 2 variables (Subtitle and Image).
Results of a combined report for the 3 variables (Subtitle, Image and Button) sorted by CTR descending, will look like this:
First Conclusions
Based on this initial traffic, we’ve identified:
- that the third variation for the Subtitle snippet (“YOUR LOCATION”) isn’t working at all, no matter what other variations it was paired with. This is a good moment to remove this variation from the landing page as it isn’t bringing results.
- Also there is a clear difference between the background image variations. The variation containing a background image performs better and traffic should be increased to it. Or maybe the variation with no background image should be removed.
- The 3 background colors for buttons didn’t make a difference for Clicks and CTR, so they can be kept as they are.
What we do next
Change 1 – subtitle variation
Normally we would have removed the third variation for the Subtitle snippet, but instead we changed it to something else in order to test it further.
We changed the Subtitle variation from “YOUR LOCATION” to “<visitor’s city>”, where this value will be dynamic based on the actual visitor location.
Let’s see if the CTR improved when we showed the exact visitors’ location in the landing page instead of the generic “YOUR LOCATION” text.
It’s pretty easy to write the actual location for visitors in the landing page with MV Lab.
First CPV Lab Pro was configured to pass visitors’ city in the landing page URL in the “city” parameter by ending the Landing Page URL with “?city={!mm_city!}”.
The {!mm_city!} placeholder will get replaced by the actual city of each visitor and now MV Lab will write the actual city for each visitor when this Variation 3 will be displayed.
Change 2 – Background share
Another change we made was to send more visitors to the variation with background image.
We’ve set 75% of traffic to go to variation 1 (with background image) and 25% traffic to go to variation 2 (no background):
Snippet 3 (background color for button) remain unchanged.
Second Results
After another day of traffic, this is the overall results report:
First thing that we’ve noticed here was that the overall CTR really increased from 8.33% to 15.63%!
That’s a nice improvement, almost double, but let’s dig deeper into the data to see where it comes from.
- First we check snippet 1 from the page: the Subtitle text.
This rotated 3 variations for the subtitle: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “<actual-city-of-visitor>”.
What we see here is that Variation 3 for Subtitle got up to 20% CTR!!
So the change made for Subtitle to display the actual city of visitors in the page really worked! Wow!
That variation 3 for Subtitle was something that we initially considered removing all together, but then decided to just modify it and this performed well.
It was the top performing subtitle on the second day and had a 13 times increase in CTR from previous day(up to 20% from 1.54%).
2. Second snippet, the background of the page, which we’ve tested in 2 variations: with and without a background image.
We see that both variations show an increase in CTR, although the second variation (no background) image continues to perform worse, just like it did from the beginning.
And now let’s add both Subtitle and Background image in our report and see their combinations performing:
As expected based on the individual reports, the best performing option is the actual city as Subtitle and the background image variation.
Combined, these 2 got up to 22% CTR. Closed second comes the Black Friday Subtitle and the background image with 19% CTR.
3. The third variable of this test: CTA Button color
It has very close results again. So, these variations don’t make much difference in the context of this landing page performance.
The combined report for all 3 variables looks like this:
Final Conclusions & Actions:
- Variable 1 – Subtitle
The change to display the actual city name for each visitor proved successful as this made the corresponding variation the top performer for the Subtitle snippet.
This change should remain in the campaign.
But the “Next 10 minutes” variation for Subtitle remained with a pretty low CTR, so it looks like a good moment to drop it from my page.
2. Variable 2 – Background or not background image
For the second variable that we’ve tested, the background image, it was confirmed once again that the variation with no page background has poor results.
So, we should drop this variation from the page and keep only the option with a background image.
3. Variable 3 – Button color
As for the third variable, the background color of the CTA button, all 3 variations performed pretty similar and this means we can keep all 3 variations in the landing page.
The overall CTR increase from the page can be attributed to the better performing variation in Subtitle and also to the fact that we’ve set the background image variation to be displayed to more visitors than initially (75% of visitors).
Both these changes proved successfully and had a positive impact on the CTR.
Final Conclusion: keeping only the variations that performed well should allow you to get an even better CTR in the page. 🙂
As you can see… it is pretty easy to do all these tests and adjustments when using a tool like MV Lab and the reporting from CPV Lab Pro.
We did this test for the affLIFT community and you can check it here to see other conclusions as well.