SiteCare’s Email Marketing A/B Testing Guide

A/B testing is a great method for figuring out what does and doesn't work in driving clicks and conversions. Find out how to A/B test your emails to eliminate the guess-work.

by Asta Geldenhuys

Email Marketing

| 10 min

“You have to test, otherwise you are just making an educated guess.”

Stuart Clark, Red C (Litmus Live, London, 2017) 

It can be tempting to use our intuition alone to predict which marketing strategies will drive people to convert, but relying solely on guesswork can be detrimental to results.

A/B testing is a great method for figuring out the best marketing strategies for your business. In the context of lead generation, it’s the most effective way to figure out what does and doesn’t work with your target audience to ultimately drive clicks and conversions. This blog takes a look at the Whys, Whats and Hows of A/B testing.

Table of Contents

What is A/B testing?

Email marketing A/B testing (or split testing) tests different variations of the same email campaign to confirm which one is the most effective in encouraging engagement. An example of an A/B test would be setting up two versions of an email campaign – intended for two separate recipient groups – and including one differing test variable. This variable could be a different subject line, call to action, offer, or style of messaging.

The ‘winning’ version is determined by the metric you are measuring. For instance, if you want to know which variation encourages recipients to respond to your emails, use the reply rate as your success metric. Similarly, use your open rate to measure which email campaign gets the most recipients to open your emails.

What should you test?

When it comes to A/B testing, small changes can have a significant impact on your results. While you can test almost any aspect of your lead generation campaign, you should focus on key campaign elements that are most likely to increase results and improve performance, such as subject lines, calls to action (CTAs), and email content. 

Let’s delve a little deeper into each of these elements.

A/B testing subject lines

Testing your subject line is a great way to begin optimizing your lead generation campaigns. While your subject lines are one of the easiest elements to test, they are the most important. An effective subject line can make all the difference in a successful email outreach campaign; as the first point of contact with a recipient, your subject line needs to encourage as many opens as possible to facilitate conversions.

Some elements to test in your subject line to encourage recipients to open your email include:

1. Test different subject line lengths

According to data from Marketo, the optimal length for subject lines is seven words or 41 characters. With over 40% of emails being opened on mobile devices, which typically show fewer characters, shorter subject lines will help ensure your email campaigns are optimized for mobile.

Here is an example of two different length subject lines: 

  • Short subject line: “{company} intro.”
  • Long subject line: “The stage is set for {company} to crush Q4 goals.”
2. Personalize your subject lines

A recipient’s name is the single most impactful word you can add to your email subject line.

In the context of lead generation, personalization is a very effective tactic. Incorporating and testing personal details, such as a recipient’s name – “Can you spare 5 minutes, {name}?”– or location, creates a sense of familiarity and builds connection. 

3. Ask a question

Inspire recipients to open your email by testing various questions – it’s a great way to catch a reader’s attention. The questions asked should always relate to the content within your email.

  • For example:
    • “{name}, what would you change about your company culture?”
4. Use symbols or emojis

Another easy way to capture a reader’s attention is using non-standard character sets in a subject line. Test one version of your subject line with non-standard characters and one without to see whether this resonates with your target audience. Again, keep it relevant by always relating the non-standard characters to the content within your email. 

Here are examples of ways to use non-standard character sets in a subject line: The version without a non-standard character could be, “Jane // {name},” while the one using an emoji could be: “Coffee meeting, {name}? ☕️”

A/B testing email content

Email content can be a tricky element to test. Simply changing a few sentences here and there can either overcomplicate your messaging or deliver little to no results. Instead, focus your efforts on key aspects that are sure to move the needle of your campaign. Let’s look at some different ways of A/B testing your email content:

1. Test different lengths and formatting

The length and formatting of your content can have a huge impact on your engagement rate. Test different formatting options to see what works best.

Here are some examples of formatting options you could test:

  • Message length: Test a long and detailed version against a short and punchy version.
  • Bulleted lists: Test writing out lists in paragraph form against bulleted lists.
2. Personalize your content

As with subject lines, personalizing your content creates a feeling of connection. In addition to using the recipient’s name, location, or company name, make a good first impression by including a customized introduction sentence. A personalized introduction indicates to the reader that you have thoroughly researched their business before reaching out to them.

To see if this method resonates with your target audience, try testing one version of your messaging with a custom introduction and one without.

Examples of personalizing introductions include:

  • Referencing a mutual connection: “I received your contact information from {mutual connection}.”
  • Adding interesting information about a recipient learned from their LinkedIn profile or About pages on their website: “Fascinating to see your case study on increasing event attendance using SMS to drive signups is super smart!”
3. Leverage your call-to-action (CTA)

Include an engaging CTA link to drive recipients to the exact place you want them to be.

A/B test this element by considering the various link locations in the body of the email, the anchor text options on the link, or the copy options leading up to the link.

Have a look at this example of an engaging CTA link:

A/B testing different content tones

The tone of an email can be professional or conversational, edgy, positive or neutral; there are various tones you can use to resonate best with your specific target audience.

The bottom line is that the tone you use in your email campaigns can greatly impact your engagement rate. Carefully consider your communication style with readers and test different versions to establish a winning formula.

Below are examples of different communication styles for A/B testing.

1. Professional

These emails are very professional in nature and less conversational. They use longer sentences and fuller paragraphs. They’re intended for an audience that appreciates a level of respect and detail in their emails, such as legal professionals, people in government, and doctors or healthcare professionals. 


Hi {title} {surname}, 

My name is Jane Smith from SiteCare.

I’m aware that {company} uses WordPress for {website} so I thought I would inquire about how you maintain it all and keep it secure? 

The monthly management of a website can pose some extensive threats and problems that can create major burdens and setbacks for many companies. 

SiteCare can help {company}’s website to stay secure, updated and free of bugs at all times while needing zero resources from your end. 

If you’d like to understand more about how we can help keep {website} running smoothly for you every month, perhaps we can have a discussion. 

We’re also happy to run a complimentary audit of your site to see where it may not be functioning optimally or where there may be some security vulnerabilities. 

Are you available for a discussion next Tuesday? 

Jane

2. Conversational 

This tone is natural and engaging. It maintains a personal touch but without losing a level of professionalism expected in an email outreach sequence. 

Hi {name}, 

Jane here from SiteCare. 

I know that {company} uses WordPress for {website} so I wanted to ask how you maintain it all and keep it secure? 

I know the monthly trudge of updates (and the problems they can trigger sometimes) can be a major time-suck and hassle.

That’s why I’m in your inbox 🙂 

Think of SiteCare as your WordPress security and maintenance team that gives you the peace of mind that your site is always secure, up-to-date and running smoothly.

I’d be happy to do a quick security and speed check on your site if you’re ok with that? 

Jane

3. Edgy 

The tone of this email is edgy and almost jarring. It’s cheeky and pokes fun at itself. The purpose of it is to be read. It ensures your voice is heard among the many emails that your reader might be getting in a day and often forces a reply because of its bravery. 


Hi {name}, 

Here’s the not-so-fun side of managing your website… 

Keeping WordPress up-to-date, bug-free and secure. 

These tasks can suck the life out of even the most energetic business owner. 

But here’s a secret. Bring your ear closer; I’m going to whisper it…

You don’t have to do any of it.

SiteCare does it for you. 

If that sounds like a plan, let’s have a call next Tuesday? 

And that’s it; shortest sales email ever. 

Jane

Tips for running more effective A/B tests

Now that we’ve covered the different elements of A/B testing, let’s look at the most effective ways to implement and execute your A/B tests.

  • Limit your tests to one variable at a time. Testing more than one variable at a time can complicate your results. Isolate and measure individual variables to evaluate how effective a change is. 
  • Determine your sample size. To get statistically significant results, you should send an A/B test to a percentage of your total recipients. Use an A/B sample size calculator to find the right sample size – the larger your test sample, the more accurate your results will be.
  • Listen to your audience. Instead of looking exclusively at reply rates, you should also look at the actual replies themselves. Why? Your audience could very well be telling you what they want or don’t want to hear. When you start seeing a common theme in replies, create a test variation that addresses that particular question, concern, or skepticism. Once you start identifying patterns, use these insights to revise your subject line, content, or CTA and improve the effectiveness of your email marketing strategy.
  • Be patient. Allow enough time for the A/B tests to produce significant and useful data. Depending on the number of sequences, or the frequency of your sending schedule, this could mean recording and analyzing data every two to four weeks. 
  • Build on your learnings and keep testing. A/B testing is one of the most effective ways to gain insight from lead generation campaigns to improve future ones. It can be tempting to stick to a ‘winning’ formula for too long. The reality is your audience is constantly evolving and changing – what works today won’t necessarily work tomorrow. If you’re always testing, you’re always improving. 

Key takeaways

Email marketing is not an exact science, but one thing is certain; it is much harder without A/B testing. There is no end to what can be tested and what knowledge can be gleaned from this form of testing. The key takeaways are to:

  • Focus your efforts on key elements that are most likely to increase results.
  • Limit your A/B testing to one variable at a time.
  • Use large test samples for more accurate results.
  • Allow enough time for the test to produce significant outcomes.
  • Build on your learnings.
  • Test, test, and A/B test again. 

Take the guesswork out of email marketing and get started A/B testing your email outreach campaigns today. At the very least, you will learn something about your audience that can help you create better campaigns in the future.

Get in touch!

Most brands don’t have a one-size-fits-all marketing solution, and email is just one of many lead generation and marketing channels.

If you want to improve your email marketing campaigns or learn how best to incorporate other channels into a comprehensive marketing strategy, consider contacting our expert team of digital marketers to help you meet your business goals.

Contact SiteCare

Asta Geldenhuys | Lead Generation Project Manager

Asta is the Lead Generation Project Manager at SiteCare. For three+ years, she has streamlined operations, managed 145 email marketing campaigns, and helped generate 1,400+ leads for global B2B businesses. She has also enjoyed approximately 2,350 cups of coffee while raising three cat kids and documenting life in Cape Town as an amateur photographer.

Leave a Reply

Your email address will not be published. Required fields are marked *