Recipe: Running an SEO A/B Test

SEO A/B testing lets you compare two versions of a page’s SEO elements (the title or the description) to see which one performs better in Google search results. This recipe explains how to use Rank Forge PRO’s A/B testing feature.

Step 1: Choose What to Test

Not everything is worth A/B testing. Focus on changes that can directly impact click-through rate from search results.

1. Choose a page with consistent search traffic: the comparison names a winner only when each variant has at least 2 full weeks and 1,000 impressions in Google Search Console, and the first 3 days of every week are not counted
2. Decide on exactly one element to change — do not change the title and description at the same time

Step 2: Set Up the Test

1. Make sure Google Search Console is connected in Rank Forge Pro with a property chosen: the A/B Testing box in the post editor counts the page views of each variant on your own site and compares the variants by the clicks, impressions, CTR and average position that Search Console reports for the page. You can still open the page in Search Console yourself and read the click-through rate of each period there
2. Open the post or the page in the editor, go to the Advanced tab of the Rank Forge box and scroll to the A/B Testing section at the bottom
3. The first time, the section shows an early-access notice: tick the box that says you understand this is an advanced feature and click Unlock A/B Testing (asked once for the whole site)
4. Tick Enable A/B Testing for this post
5. Variant A is the title and the description the page has now, so there is nothing to enter for it
6. Enter Variant B, your new hypothesis, in the two fields under Variant B (Alt title… and Alt description…). A field you leave empty keeps the normal text of the page
7. Update the post. There is no duration to set and no start button: the test runs, changing the variant every week, until you untick the box

Step 3: How the Test Works

1. Rank Forge alternates between Variant A and Variant B on a weekly rotation
2. During Week 1, Google sees Variant A; during Week 2, Google sees Variant B; and so on
3. The plugin counts the visits to the page on your site for each variant. It also asks Google Search Console for the clicks, impressions, CTR and average position of the page, and shows the figures of each variant, added up over its full weeks, once at least one full week has counted (a week counts about 3 days after it ends). The first 3 days of each week are not counted, and neither is the week still running
4. Do not make any other SEO changes to the page during the test — this would introduce variables that invalidate the results

Step 4: Monitor the Test

1. Open the post and look at the A/B Testing section on the Advanced tab of the Rank Forge box to check progress at any time
2. You will see:

• The week and the variant that is live now (for example, Week 3 showing Variant A)
• A note on how far the test has come: Early, Collecting data, or Enough weeks have passed
• Page views so far for variant A and for variant B (visits to the page on your site)
• A note on whether both variants have cleared the floor of 2,000
• The Search Console comparison: impressions, clicks, CTR and average position for each variant, and either which variant is ahead or why there is no winner yet

1. Do not end the test early based on early numbers: small sample sizes can be misleading
2. There is no set end date. The test runs until you untick the box, so wait until the box says Enough weeks have passed, and read the Search Console comparison below it, before you choose

Step 5: Analyze Results

1. When the box says Enough weeks have passed (4 weeks and 3 days after the start at the earliest), read the Search Console comparison in the A/B Testing section. To check it, open the Performance report of Google Search Console and filter it by this page
2. The comparison shows, for each variant, the impressions, clicks, CTR and average position of its full weeks. In Search Console you can compare the same figures for the weeks when Variant A was live with those of the weeks when Variant B was live
3. Look at CTR as the primary metric: it is most directly affected by title and description changes
4. The box says that a variant is ahead only when each variant has at least 2 full weeks and 1,000 impressions in Search Console, the better CTR is at least 10 percent higher than the other and the gap is not down to chance, the same variant is ahead in every pair of weeks (week 1 against week 2, week 3 against week 4, and so on, at least two pairs), and the average positions of the two variants are within 1.5 of each other. It shows no confidence figure. Judge the difference yourself, and be careful with a small one, because a small sample can be misleading

Step 6: Apply and Iterate

1. If a clear winner emerges, untick Enable A/B Testing for this post and update the post. If Variant B won, first copy its title and description into the SEO Title and Meta Description fields of the Rank Forge box. Rank Forge has no button that applies a winner
2. If results are inconclusive, either extend the test or try a different variation
3. Document your findings for future reference — over time, you will learn what types of titles and descriptions perform best for your audience
4. Move on to test another page using the same methodology

—