A few months ago, I began walking a new path. It was a strange path at first, one that entailed changes to the way I understand the most holy of all entities, the internet. I slowly started seeing things. Things that were there all along, laying there, unnoticeable to my then innocent eyes.
After spending my first couple of weeks going through every resource I could find on the subject, I started reviewing hundreds of landing pages, providing critique, suggesting tweaks learned from conversion optimization lessons and applying fixes. Using Talia’s unique psychological methodology for emotional triggers implementation, or what we at Conversioner call “Emotional Targeting”, I was starting to get the hang of what conversion optimization is all about.
3 Lessons from my first 3 a/b tests
Actually it wasn’t until I set up my first few a/b tests that it hit me: how freakin cool this job really is! There can be a real rush to testing. You somehow, at the same time, feel like both a scientist and a magician.
My first two tests were honestly not all that great, at least in terms of the end game result. After running for almost two weeks, no substantial increases to the client’s revenue were anywhere to be seen. Nonetheless, they were crucial for my understanding of what you should and shouldn’t do while a/b testing. I’m talking about the best kind of lessons, ones learned from your own mistakes, but I’m also talking about insights regarding specific user experience and behavior of the client’s website and product.
1. Track don’t Slack
Without a doubt, my biggest mistake on the first two tests was not fully comprehending the importance of implementing efficient and accurate tracking methods. At first I found myself struggling with third party tools like Optimizley and VWO, setting up custom variables and goals. My initial thought was I can overcome the fact that the Google Analytics tracking wasn’t implemented properly. I thought this could be an easy fix, one I can do by myself without having to depend on developers.
Integrating Optimizley with Google Analytics using custom variables
It actually went quite well at first, that is until we started seeing some major discrepancies between the numbers we see on Optimizley, the ones in GA and most importantly the sales data from the client’s BI system. After a short evaluation, we stopped the test and insisted on implementing proper Ecommerce tracking. Using Google’s Measurement Protocol we made sure that the data was consistent between platforms. A few days ago we re-launched the test and are already seeing some promising results.
2. Understand what you’re measuring
On my third test we started seeing some amazing results. We set up a test for an ecommerce site using good old Google Analytics content experiments. While the test is still in progress, after 10 days of data, a variation we designed is producing 3 times more revenue than the original page!
Revenue X 3
At first, I was so excited to see that one of the variations we designed for the homepage had improved the conversion rate, I didn’t even look at the other metrics. This was my first distinct ecommerce test and also my first encounter with the “Per Session Value” metric calculated as revenue divided by visits. It turned out that not only did we improve the conversion rate by 25%, we also almost tripled the per session value by simultaneously doubling the Average order value.
Once we conclude this test we’ll have a case study out sharing a lot more details including our insights and findings from this mind-blowing test.
3. Think outside the box
As I told you before I’ve had some experience using third party a/b testing tools. One major advantage these tools have over Google’s content experiment is the native ability to segment experiments to different audiences. For example, the first test I ran with Optimizley was intended for desktop users only and I could very easily set up a condition to exclude mobile users from the experiment.
However, while setting up the new test using Google’s content experiment I’ve found that alas! There’s no built in feature to segment audiences. But, don’t worry, we do not despair that easily. We commissioned a hack for this specific function.
Segment Google content experiments by devices
It’s easy and it works great. Our developer came up with a simple solution, which we would be happy to share with you. Just drop me a line and let me know you’re interested at arad@conversioner.com and I’ll send you everything you need including simple instructions on how to execute the segment.
One tip to rule them all
In general, I would say that there’s only one important tip you need to know when getting into conversion optimization, and that’s to always be open to learn new things.
Having said that, in the interest of giving some more tangible advice, I will be sharing my insights on this blog. So, just to reiterate, here are my first top 3 noob tips:
1. Invest time and effort setting up proper tracking and selecting the right platform
2. Understand what you’re measuring – Know your metrics
3. Don’t be satisfied with the tools you find out there – Apply ingenuity
Share the wealth!
What were your first mistakes and lessons on your A/B tests? Drop a comment below I’d love to hear from you.
3 Conversion Optimization Lessons Learned the Hard Way 5.00/5 (100.00%) 9 votes
Related Posts
3 Conversion Optimization Lessons Learned the Hard Way
A few months ago, I began walking a new path. It was a strange path at first, one that entailed changes to the way I understand the most holy of all entities, the internet. I slowly started seeing things. Things that were there all along, laying there, unnoticeable to my then innocent eyes.
After spending my first couple of weeks going through every resource I could find on the subject, I started reviewing hundreds of landing pages, providing critique, suggesting tweaks learned from conversion optimization lessons and applying fixes. Using Talia’s unique psychological methodology for emotional triggers implementation, or what we at Conversioner call “Emotional Targeting”, I was starting to get the hang of what conversion optimization is all about.
3 Lessons from my first 3 a/b tests
Actually it wasn’t until I set up my first few a/b tests that it hit me: how freakin cool this job really is! There can be a real rush to testing. You somehow, at the same time, feel like both a scientist and a magician.
My first two tests were honestly not all that great, at least in terms of the end game result. After running for almost two weeks, no substantial increases to the client’s revenue were anywhere to be seen. Nonetheless, they were crucial for my understanding of what you should and shouldn’t do while a/b testing. I’m talking about the best kind of lessons, ones learned from your own mistakes, but I’m also talking about insights regarding specific user experience and behavior of the client’s website and product.
1. Track don’t Slack
Without a doubt, my biggest mistake on the first two tests was not fully comprehending the importance of implementing efficient and accurate tracking methods. At first I found myself struggling with third party tools like Optimizley and VWO, setting up custom variables and goals. My initial thought was I can overcome the fact that the Google Analytics tracking wasn’t implemented properly. I thought this could be an easy fix, one I can do by myself without having to depend on developers.
Integrating Optimizley with Google Analytics using custom variables
It actually went quite well at first, that is until we started seeing some major discrepancies between the numbers we see on Optimizley, the ones in GA and most importantly the sales data from the client’s BI system. After a short evaluation, we stopped the test and insisted on implementing proper Ecommerce tracking. Using Google’s Measurement Protocol we made sure that the data was consistent between platforms. A few days ago we re-launched the test and are already seeing some promising results.
2. Understand what you’re measuring
On my third test we started seeing some amazing results. We set up a test for an ecommerce site using good old Google Analytics content experiments. While the test is still in progress, after 10 days of data, a variation we designed is producing 3 times more revenue than the original page!
Revenue X 3
At first, I was so excited to see that one of the variations we designed for the homepage had improved the conversion rate, I didn’t even look at the other metrics. This was my first distinct ecommerce test and also my first encounter with the “Per Session Value” metric calculated as revenue divided by visits. It turned out that not only did we improve the conversion rate by 25%, we also almost tripled the per session value by simultaneously doubling the Average order value.
Once we conclude this test we’ll have a case study out sharing a lot more details including our insights and findings from this mind-blowing test.
3. Think outside the box
As I told you before I’ve had some experience using third party a/b testing tools. One major advantage these tools have over Google’s content experiment is the native ability to segment experiments to different audiences. For example, the first test I ran with Optimizley was intended for desktop users only and I could very easily set up a condition to exclude mobile users from the experiment.
However, while setting up the new test using Google’s content experiment I’ve found that alas! There’s no built in feature to segment audiences. But, don’t worry, we do not despair that easily. We commissioned a hack for this specific function.
Segment Google content experiments by devices
It’s easy and it works great. Our developer came up with a simple solution, which we would be happy to share with you. Just drop me a line and let me know you’re interested at arad@conversioner.com and I’ll send you everything you need including simple instructions on how to execute the segment.
One tip to rule them all
Having said that, in the interest of giving some more tangible advice, I will be sharing my insights on this blog. So, just to reiterate, here are my first top 3 noob tips:
1. Invest time and effort setting up proper tracking and selecting the right platform
2. Understand what you’re measuring – Know your metrics
3. Don’t be satisfied with the tools you find out there – Apply ingenuity
Share the wealth!
What were your first mistakes and lessons on your A/B tests? Drop a comment below I’d love to hear from you.
Related Posts
Tags: