For years, A/B tests have been the basic tool for optimizing conversions on a website. Their effectiveness is best demonstrated by the fact that in the age of technology, tech giants such as Apple, Netflix and Google make use of these.
At first glance, A/B tests seem to be a very simple procedure. We create two varieties of sites, where the only difference is the color of a button. After some time, we get data and we learn that a red button converts about 0.20% better than a green one. On the one hand, these results provide us with new knowledge. On the other, I think all of us would prefer to discover something much more spectacular.
What’s the secret to conduct valuable A/B testing for our business? In fact, everything starts much sooner, in the conceptual stage, where we analyze data, create hypotheses and based on that we model scenarios. All the work pays off at a later stage by obtaining more valuable data.
In this post, I’m going to take a closer look at how we conduct A/B testing at Divante.
Determining Business Objectives
Our work starts from defining the objectives of business websites and pointing out performance indicators. At this stage, we often encounter the problem with getting a clear answer to the question: Why does your website even exist?
This question often arises for the first time in many years, and it turns out that the objectives set in the past, are now outdated. The company has grown, it has changed its priorities and, in reality, you are facing new challenges.
Therefore, it’s worth to reevaluate what you are really striving for. Whether it is an increase in sales or leads, by increasing the retention rate. Maybe you want a larger involvement of your users or any other factor, that will further develop your business.
Analysis of Quantitative and Qualitative Data
Once we know what your purpose is, we analyze the status quo. We are looking for answers to such questions like: Where are you now? Where do you want to be? Which places on the website are currently making this impossible? What places are worth optimizing? In what way?
At this point, we should use all the data available to us, applying the Mixed Methods Research approach. For more about the approach itself, have a look at this article: Mixed Methods Research in website optimization. After analyzing all the information, we should be ready to construct the first hypothesis, which you’d want to verify, in the subsequent steps.
Development of Methodology and Test Scenarios
We already know what we are striving for and how we can probably accomplish that. Let us move to the technical and methodological part. For this purpose, it is worth considering such elements as: Which areas and sides should be tested? What tools are worth installing? What volume of traffic do we need?
Before making a choice, let’s see what we want to achieve at the end of the experiment. A common mistake committed in this step, is to use tools that do not have the ability to measure performance indicators set by us, or to plan an experiment for a shorter period of time, that it’s possible to obtain statistically valuable results.
At this point, we are leaving the conceptual phase and entering the stage of realization of our ideas. Finally, we can begin to test graphic design variations, the process of adapting them to the standards of SEO and the installing all measurement tools.
After the completion of the technical work, it is very important to check and verify the installed solutions before the official launch. In situations like these, it is worth to keep in mind, that an error of an active experiment may cause unwanted changes on the website, or even problems with its loading. The consequences can be much more severe than taking an extra hour before to verify everything is working correctly.
We move on to the most pleasant part. We start! We run our experiment, we keep monitoring its performance and its achieved results.
Result Analysis and Conclusions
After testing, we analyze the results. Have our actions in some way contributed to the specified business purpose? Which modification had the greatest significance? Is our hypothesis confirmed? It’s fascinating how many new things we learn about the user this way! The variant, that achieved the highest conversion rate won, and the verified hypothesis can now be applied safely throughout our site.
The process of A/B testing:
Done! We have successfully conducted an A/B test! Implementation of these measures has provided us with very valuable conclusions, from a business point of view. Though it may seem the project was time-consuming, it actually doesn’t take that long. Leading brands perform up to 600 tests per month. Of course, not everyone can afford such dedication. But it is enough to verify one of the existing hypothesis once a month. Within one year, you will definitely get to know more of you users, their habits, behavior, and thus maintain a high conversion rate
This article was written by Natalia Malec-Lachowska, a Web Analytics Specialist at Divante.