How to use A/B testing as a Marketer


A recent Gartner Survey found that of 206 marketers, only 33% were regularly using A/B or multivariate testing technology and 28% were in the process of deploying a solution.   We sat down with HI’s Client Strategy Director & Sitecore Digital Strategist MVP, Angela Gustafson, to find out why more marketers weren’t taking an advantage of this important tool.

1. Tell me what you know about A/B testing and how to best use it for clients?

We always preach to our clients that you’re never done with your development and marketing strategy. Even after launch you should be looking for opportunities to optimize. A/B and multivariant testing is an important tool in your optimization strategy from both a user and business stand point. It provides business units a way to test customers interactions with different messaging or visual calls-to-action that contribute to a rise in conversions.

2. What should you know before you begin A/B testing?

While Sitecore makes it very easy to setup A/B and multivariant tests you also need content strategy. Meaning, you will need to create at least 2 versions of something to test - because of this, the maintenance and man power needs to be factored into your planning. Also, for a successful measurement, you will need an engagement value strategy in place which will require some initial strategy, planning and setup.

According to Gartner research, marketers should focus on blending user experience, customer experience & marketing analytics to determine where to focus on testing.

3. What technology do we use at HI to carry out those tests?

At HI we use our CMS platform, Sitecore. Sitecore is really focused on their analytics and their optimization tools. It makes it easy for marketers to set up tests within their existing content authoring process.

4. What is the best way to get started and implement A/B testing?

Start by working backwards –

  • Ask yourself: What path do you want your users taking?
  • Look at all the possible ways they could get there
  • Look at your content strategy to navigate them there
  • Place engagement values on all steps in the journey
  • Choose your entry point
  • Test an area on that page


Knowing that only 33% percent of companies have deployed A/B and multivariant testing successfully, at Horizontal Integration we would suggest our clients follow our Crawl, Walk, Run philosophy.

Start by putting an engagement value on 2-3 pages on the site and choose a simple test. For example, you could start by testing Calls to Actions on your homepage like “Learn more about our special offer,” or “Earn a special offer today.” Once you have a handle on the tool and processes, you can start brainstorming more applications.

And we’re always here to help, get in touch today to learn more.