There are a lot of variables to consider when running A/B experiments for SEO as opposed to for users. Some aspects of test design, controlling for external factors, and some of the tools that can assist you in running experiments are a source of confusion. Everything you need to know is in this guide, which also includes case studies and sample tests.
Where can I learn more about SEO A/B testing?
A website’s organic search traffic can be difficult to measure after a website change.
You can’t get by with just tweaking your website and monitoring metrics like rankings and click-through rates (CTR).
Otherwise, there is no way to tell whether changes in rankings or CTR are the result of your own efforts versus external factors.
Before and after testing refers to the process of making a change and evaluating metrics. Tests that measure the impact before and after a change are better than doing nothing, but they aren’t controlled tests, so they are useless for drawing conclusions. It’s easy to be misled and think that your change was responsible for the observed impact, when it could just as easily have been a competitor’s change, or many other external factors.
Out of consideration for the lack of a control group of pages, the length of time it takes search engines to crawl your site and take the changes into account also complicates analysis. It could happen in an instant or it could take a couple of weeks to complete the process. That makes it nearly impossible to look at data, such as traffic or rankings, and try to match them up with a change you made.
SEO split-testing with a control group and variant groups is when you split a group of statistically similar pages into two groups, and then make changes to the variant pages in each group.
These two groups can be compared to each other as well as a forecasted level of visitors had no changes been made.
Exactly what kinds of websites are eligible for SEO experiments?
Split-testing SEO is not suitable for all websites or parts of websites. Prior to running tests, you must meet the following requirements:
Your site must have many pages on the same layout.
You’ll need a lot of people to visit your site.
How much traffic do you expect to see?
Good question and the answer depends on your site’s design. According to a general rule of thumb – the more stable your traffic patterns are, the easier it will be to run experiments with less traffic. A robust traffic model requires more traffic the more irregular your website’s traffic is.
If there is less traffic, is it possible to see if you have it? Some of our customers run tests on sections of their site that only get a few thousand visits per month, but the traffic changes must be much higher to be statistically significant.
You’ll be able to reach statistical significance faster and with a smaller effect if you have more traffic and pages.
Create an SEO A/B test. SEO tests are composed of five components:
- A selection of test pages
- A hypothesis is created of the selected pages
- Controlling and modifying pages
- Creating change
- Analyzing results
· A selection of test pages
A group of pages on the same template is the first step. You can choose from a group of category pages or product pages on an ecommerce website, or you can use destination pages or flights pages on a travel website.
A six-page test would be extremely rare in reality, but the principles are the same whether there are six pages, 600 pages, or 6 million pages. When we get to the test analysis section, it will be easier to explain everything if we keep it simple.
(Picture 1)
· Developing a hypothesis for your study
In spite of the fact that there are many differences between SEO testing and user testing (we’ll get to that later), one thing they both have in common is that it’s crucial to start with a solid hypothesis before you begin.
That Google gives different levels of importance depending on the position on the page, is well-documented.
When moving text higher on a page, we believe that it will increase its relevance for our target keywords by making it more important to readers.
Pages that have content lower on the page compared to those that have it more prominently displayed on the page, we’ll see if there is a difference in organic traffic.
(Picture 2)
· Making a choice between control and variant pages (bucketing)
The next step is to decide which pages will be the control pages and which pages will be the variants once you have decided what you want to test.
One of the most important, yet least understood, aspects of SEO testing is deciding which pages should be included in each bucket of results. It is important to keep in mind the following two factors:
It is expected that both buckets will have a similar amount of visitors. If, for example, control pages receive 10X as much traffic as variant pages, you’re unlikely to get good results.
Statistically speaking, the two buckets should be similar. They should have a similar up-and-down pattern.
( Picture 3)
· Making the switch to alternative pages
After the buckets have been selected, only the variant bucket of pages needs to be changed. Upon the change, two different templates, but only one version of each page will be available at any given time.
Important to note is that each page only has one version. If you’re not familiar with SEO testing, or if you’re used to running user-based A/B tests, it’s easy to get confused.
A user or a search engine will see the same thing, regardless of who requests a page. Details on this can be found under SEO A/B testing vs. user A/B testing in the section on the differences.
Measuring the experiment’s impact
For this explanation, I’m intentionally simplifying the math(s) involved in SEO A/B tests. In order to analyse SEO experiment results, our engineering team has spent more than five years building a neural network.
A/B testing for users and search engine A/B testing: What’s the difference?
We are frequently asked a variation of the following question:
User testing, isn’t that what this is?
However, there are a few key differences between product teams and marketers when it comes to user A/B testing.
Design a test
Users’ testing is relatively straightforward. Assigning users randomly to the A or B version of a page that you want to test is done by creating two versions of that page.
Several factors prevent us from performing SEO A/B testing in this manner:
- Splitting Pages – Page splitting vs. people splitting Googlebot is the “user” we are testing for, not humans. Determining a random split of ten thousand “Googlebots” into control and variant is therefore impossible. Googlebot is the only one. Two versions of a single page are also out of the question due to issues such as duplicate content. We split a group of pages into control and variant pages instead of splitting users for SEO testing because it’s more efficient to do so.
- Bucket Selection – Users can be randomly assigned to control or variant pages using a bucket selection system. When it comes to SEO testing, this isn’t ideal. All of the most popular sites could end up in one group if you bucketed them at random, which could skew results. External factors would also be more difficult to account for. They must have similar traffic levels and be statistically similar to each other in order for them to be compared. You can find out more about SEO bucketing in the section on SEO bucketing.
Comparing testing on the server vs. client side
In addition, the way changes are made to a page differs between user testing and SEO testing.
In order to change the page during the test, most user testing tools use client-side methods that change the page. This means that the user requests the old version of the page from the server, the file arrives in the user’s browser unchanged, and then JavaScript makes the changes.
Client-side A/B testing tools are notorious for their “flickering”. Sometimes the old version of a page will be displayed to the user before it changes to the new version quickly.
The different types of SEO A/B Testing Examples:
- Beginner SEO A/B Testing
- Intermediate SEO A/B Testing
- Advanced SEO A/B Testing
It’s strange that we don’t use ranking or click-through data.
In our analysis, we don’t use either because of three primary reasons: 1. However, the short version is:
What percentage of organic traffic you receive comes from your clickthrough rate (CTR), and many SEO tests change how your site appears in the search results, which affects your CTR. This cannot be measured by a position tracking tool.
Keyword rankings can never cover the entire tail of keyphrases, and will miss or misrepresent cases where the effect is different in the head and long tail of a keyphrase
CTR data and the long tail are theoretically available in Search Console, but the data is sparse, inaccurate, or averaged in practice.
This is a quite a detailed information on SEO A/B Testing. If you need more information or want to take affordableseo services help for the same, you can reach out to us at GlobalHunt Technologies.
