Core Web Vitals for A/B Testing: Is Your A/B Testing Software Slowing Down Your Site?

Will My A/B Testing Tool Slow My Site And Affect My Core Web Vitals Score?

Let’s get this out of the way right up top. The Convert app is incredibly fast and shouldn’t negatively affect your Page Experience or Core Web Vitals score, as long as you follow best practices for both testing and CWV setup.

Not every site follows best practices, however, and in those situations, your A/B testing could affect your Page Load speed, First Input Delay, Cumulative Layout Shift, or Largest Contentful Paint, depending on how you have your test and site set up.

The good news?

Each of these elements is easily fixable. 

What Are Google’s Page Experience Metrics, the Three Core Web Vitals Metrics, And How Can We Improve Them?

There are 4 basic Page Experience metrics and 3 more in a specific subset called Core Web Vitals (the focus of the latest Google update).

Learn what Google qualifies as a great user experience & read more about the 4 basic Page Experience metrics.

Each of these basic metrics is quite easy to achieve. All you need is a responsive site, no dodgy code, not to cover the site in popups, and to be running through HTTPS.

These are just the basic elements though. There are 6 more Page Experience metrics used by Lighthouse when it measures your Page Experience performance using Lab data.

Now although there are 6 Lab metrics to focus on, they are all interconnected. This means an improvement in one usually sees an improvement in others.

To help simplify all this, Google has broken these down into 3 Core Web Vitals:

• Largest Contentful Paint
• First Input Delay, and
• Cumulative Layout Shift

These are where we need to improve and it’s also where our tests may affect our rankings.

Let’s work through each of these Core Web Vitals below and what you need to do to improve them, before looking at how your tests might affect these scores.

1. Largest Contentful Paint (LCP)

Largest Contentful Paint is based on the loading speed of the largest visible element on your screen. This could be a hero shot, a background image, or even the headline text.

This score is designed to replicate how long it takes for your audience to start to see the main content on your page and get an idea of what the page is about.

Currently, LCP is weighed at 25% of your CWV score.

Your readers being able to understand your page is important to fix, but it’s more than that. You see, most issues that cause a slow LCP are usually the root causes of what slows down pages and causes other CWV issues. This means that if you fix these LCP elements then you have done the bulk of the work.

Your goal should be to get your LCP to load in under 2.5 seconds.

The main issues that lower your LCP score/speed are:

• Slow server response times
• Render-blocking JavaScript and CSS, causing element delays
• Slow resource load times
• Slow client-side rendering
• Poor/Incorrectly set image optimization.

How To Improve Your LCP Score

There are several things that you can implement to improve your LCP score.

Here you can see my example sales page LCP score before I make any of the recommended tweaks.

As you can see, this page needs some improvement.

It currently takes 3.4 seconds for my LCP element to load on the page, even though it’s just a headline of text, and my page takes 14.7 seconds to load before it becomes interactive.

If we scroll down through the PageSpeed Insights tool and look at the opportunities, there are a bunch of things I can do to improve the overall page speed and stop some of the things that are slowing the LCP down.

Let’s go through all of them.

a. Preload The LCP Element

The first thing you need to do is check what the actual LCP element is for your current page, as it can vary between pages.

While on the mobile view of the PageSpeed tool, scroll down the page to the diagnostic section and click on the ‘Largest Contentful Paint’ element and see what comes up.

On this particular page, my LCP element is my headline.

I can improve the load speed of my text by sorting out other things on my page such as compression and caching, but what if my LCP element was an image?

In that case, I would want to preload the image on the page so that it starts to load before the page even starts to render.

Source

This way, the image starts to load immediately and is not slowed down by other code or requests.

This is a huge thing.

The standard practice is to lazy load all images on a page, to help with page speed. But when you do that to your LCP image, it actually makes it load slower, decreasing your LCP score!

(This is a big issue if you’re A/B testing your LCP element as well!)

So how do we fix it?

We want to write some code to specify that this particular LCP element should be preloaded on one particular page.

The script you want to add is the rel=”preload” script and it will look something like this:

The rel=preload script

In this example, I’m telling this particular page to preload the TRAFFIC-SMALL-HEADER image (which is the LCP image element for that page). I’m also specifying multiple dimension options so that it can load a responsive image for either mobile and desktop.

This change alone will help out any slow-loading images that are affecting your LCP score.

Some WordPress or Shopify themes will allow you to add custom code into the header of that particular page, while some plugins will allow this. Failing that, you can also edit the header.php file for your page and add the code directly.

I just use a plugin.

I’ve covered some of the basics here but most of the fixes we’re going to cover can vary depending on your site and what you use. (If you’re not on WordPress or have a developer, then get them to check out Google’s Dev advice for optimizing LCP here).

As my site is built on WordPress, I’m going to be using the WPRocket plugin as it will help me fix the majority of the issues with LCP all in one place.

b. Use High Performance/Dedicated Hosting

A super simple fix to implement. When your user loads up your webpage, it sends a request to your web host for the page information and stored files, etc.

Some web hosting services work on a shared hosting platform. This means that they share the infrastructure between multiple sites. Because of this, it means that other sites’ traffic on your shared hosting can slow down your own site’s performance.

Moving to a dedicated hosting service that is 100% for your site only is not only faster, but also more secure, and can help with page load issues and server response times.

Here you can see that my site’s initial server response time was 2.67 seconds.

After upgrading to a dedicated host, it removed that server response delay completely, saving me 2.67s in load time, and also improved the Speed Index and Time to Interactive.

c. Enable Caching and Increase Cache Length (If Needed)

Caching allows you to save on server requests by storing a saved copy of your site content for users so that it loads fast on repeat visits.

This way if they come back and want to view the content again, it will load incredibly quickly.

d. Defer Non-Critical JS + Remove Unused JS

When a page loads, it takes turns either loading things one at a time (async), or slows down and tries to load multiple things at once (sync). This isn’t so bad if you have a fast server, or if the 3rd party application you are pulling from loads fast, such as the Convert Experiences app.

But, we can help improve our score and page speed by either deferring some elements (telling them to load after more important things) or removing elements that don’t need to be loaded on every page.

This is usually a large cause of LCP loading issues, as those elements try to load before the LCP element. (They can also affect First Input Delay).

You can specify which JS, apps, or plugins to defer or remove inside WPRocket. (Just make sure to set the Convert script to load as a priority and not be deferred).

This way any non-essential JS is removed, other JS is deferred until used, and the Convert script can run asap.

Sidenote:

You can also use this section to prioritise loading of any above the fold elements such as sliders or carousels. Simply add the code to the exclusion section and it will load as normal.

2. First Input Delay (FID)

First Input Delay is a measure of how long it takes for the page to respond when the user tries to take an action, such as a button press or link click.

The most common causes of a poor FID are:

• First-party script execution causing a delay in interaction readiness.
• Data-fetching impacting interaction readiness.
• Third-party script execution delaying interaction latency.

First Input Delay is weighted as 30% of your CWV score and your goal is to get that response down to 100 milliseconds or less.

How To Improve Your First Input Delay Score

We can’t measure FID without a live user, so instead, we try to improve Total Blocking

Time (TBT) as they are both connected.

So let’s look back at our page results…

Back when I first measured my page, my TBT was 1.5 seconds (or 1,560 milliseconds).

Since I improved the LCP elements, it’s dropped to 0.2 seconds (210 milliseconds), and 3.5 seconds until fully interactive.

This is because we’ve solved a few of the issues that slow Total Blocking Time already, simply by fixing some LCP issues such as code minification and deferring or removing JS.

It’s already close to the desired speed range but let’s show you a few other things that you can do, just in case your score isn’t quite there yet.

a. Pre-load Content And Links

Here’s a cool feature inside of WProcket. With the pre-loading of LCP images, we’re telling the page to start loading the LCP image asap.

With pre-loading links and sitemaps, we’re telling the site to start preloading content in the background, when the user mouses over a button or link.

This means that those assets start to load before the user even requests them, speeding up that FID and lowering Total Blocking Time for those other pages that they click through to.

The benefit here is a faster FID on other pages, so let’s look at some more ways to improve the first page they load on.

The main thing we can do to improve FID is to remove code bloat from our site.

Go ahead and load up your page in GTMetrix.

Source

Wow, that score looks amazing, right!?

Well, that’s because this is your Desktop score, not your mobile. 

This is fine as what we want to look at is the Waterfall section, to see areas where your page load is being delayed.

The bars in beige are areas that we need to improve, as these are times where another code is being blocked from loading.

I can see in the waterfall that some old plugins and fonts are slowing down the page load, so I can go back and preload these custom fonts.

Open them up in the waterfall, then copy and paste the font URLs into WProcket. (I should have added them earlier but forgot whoops!)

3. Cumulative Layout Shift (CLS)

Have you ever tried to click on something on a site and then it moves and an advert or banner shows instead?

Frustrating, right?

It’s especially important for us as optimizers, as a poor user experience like this can cause issues with broken UI or simply audiences not understanding what to do.

(Some unscrupulous people will actually build this in as a dark design to get specific clicks, etc. Especially sites selling ad space…)

CLS measures how much elements move around on a page (aka its visual stability) and then scores your site on this. Its goal is to stop these sites from creating this poor user experience, and its current weighting is 15% of your CMV score.

Your goal is to get a CLS score of 0.1 or lower.

The good news is that we’ve already solved a lot of this by fixing issues with LCP and FID.

• We’ve set image, ad, and iframe video dimensions.
• We’ve preloaded any custom fonts so they shouldn’t cause layout shifts.
• And if you removed your page builders, you should have lowered the overall DOM elements of your page, making it both faster and making fewer requests!

How to not Negatively Impact the Largest Contentful Paint Score when A/B Testing?

Remember that the LCP elements are what’s seen on the viewfinder or screen on the initial page load, so you might not even be testing any LCP elements, but just in case:

What if you’re testing an LCP element such as a new background, hero image, or headline?

The issue you might have here is not with the Convert Experiences tool, but rather with bad practices for LCP improvements.

Like we mentioned earlier, most people make the mistake of lazy loading all images, including the LCP image, which affects the LCP score.

Convert Experiences uses both ‘synchronous load’ and ‘body hide’ methods to run tests. Simply put, the tool recognizes the element to be tested and then hides it as soon as it starts to load so the user never sees it, before changing it in under 50 milliseconds.

If you’ve lazy-loaded that LCP element though, the Convert Experiences app is going to have to wait for that element to load before it can do its job.

You can fix this by preloading the specific LCP elements on your control and test pages like we suggested above. This will get them to start loading immediately on page load, allow them to be recognized and hidden by the Convert app, and be replaced asap before the user notices.

Not only will this reduce flicker, but it will also give a good LCP score as the main LCP element (even when tested) will load fast.

And if the LCP element is not what you’re testing? You should be pre-loading it anyway to improve that LCP score.

How to Improve First Input Delay when A/B Testing

The Convert code is incredibly fast, but your FID score still relies on all the other elements of your page.

Make sure to hit all the main requirements for speed that we talked about before so that there is no delay in the Convert code running. (Otherwise, the element you are testing will be hidden until it loads and then changes.)

• You can remove code bloat from other JS and defer them from running.
• You can also ‘exclude’ the Convert script from being deferred so that it loads up before any other JS so that it can make those test changes asap, while not hurting FID.

If you don’t prioritize the Convert code and instead defer it by accident, it may not affect your FID score that much, but it might cause your test element to delay loading, while the script waits to run.

How To Decrease Cumulative Layout Shift Issues When A/B Testing

The potential issues with Cumulative Layout Shift and testing come down to how you run a layout test.

Ideally, you don’t want to test massive layout changes with just the Convert Experiences editor. Instead, you want to test similar elements for variations i.e. swapping out a hero image for another image or swapping text for variations, etc. This means that the page layout stays the same, but a key element is swapped out for a variation.

But what if you want to test a minor layout shift, such as swapping the hero image and text location?

I recommend using Convert Experiences for small layout shifts to set up the test and then see how much it affects your CLS score before you push it live.

For the full round-up of suggestions to improve your Core Web Vital scores and protect your rankings from any (potential) negative impact of A/B testing, read the original article by Daniel Daines-Hutt, here: https://www.convert.com/blog/a-b-testing/improve-web-vitals-with-convert-script/