A/B test Catalog Ads in Facebook

A guide on how to create an A/B test in Facebook.

Table of Contents

Step-by-step guide1. Save variants in Confect2. Make a "Manual sales campaign"3. Duplicate ad sets and choose designs4. Open the A/B test tool5. Finish the setupApply learnings

One of the most groundbreaking benefits of being able to use multiple designs in a single catalog is that you can A/B test with ease.

With different designs for each product, you can select which creative to use on the ad level.

Summary:
 Add multiple design variants to your catalog and create one new ad per tested variant. Run these ads through Meta's A/B test tool.

Steps:
- Add 3+ design variants to your Confect catalog.
- Download our Chrome plugin and log in.
- Create a new Manual sales campaign (just for this test).
- Create one ad set and ad per variant you're testing.
- Switch each ad to a unique variant.
- Use Meta's A/B test tool on this campaign.

We recommend testing multiple designs. Using at least 3 different designs will give you a higher chance of success and learn what type of design choices work for your audience.

You can find inspiration for what to test here.

Step-by-step guide

1. Save variants in Confect

The first step is to create the individual designs you'll be using for A/B testing, and add them to the catalog as design variants.

⚠️ IMPORTANT Test all variants (incl. original) in the same catalog to avoid inaccurate results.
Never use multiple catalogs for A/B testing, as an older catalog with more pixel data will always outperform a newer one.

If you haven't yet done so, click here and follow STEP 1 of this guide.

If this is your first time testing Catalog Ads, and you want to test the difference between the original pictures and Confect designs, we recommend these designs:

  • "Original" design (includes only the product - the same look as your original)
  • Confect design #1
  • Confect design #2
  • Confect design #3

For example:

2. Make a "Manual sales campaign"

The next step is to create a NEW campaign for this A/B test. Click "Create" in the Ad Manager, select "Sales" and then "Manual sales campaign".

Using a manual campaign allows us to create multiple ad sets and test between them.

IMPORTANT Even if you prefer to use Advantage+ campaigns for your catalog ads, we highly suggest using a Manual campaign for testing. Later, you can use the winner in an A+ campaign.

If you have other reasons to use an Advantage+ campaign for testing, click here (more complicated method).

3. Duplicate ad sets and choose designs

A new campaign will have one ad set and ad. The ad set should be duplicated (using "Quick duplicate"/ CTRL+D) so that you have one per variant you're testing.

For our example (1 original, 3 designs), we'll need to duplicate 3 times so that we have 4 individual ad sets.

Once you've duplicated the ad sets, you'll need to change the design variants in each ad. To do that, follow STEPS 2 and 3 of this guide. Go to each ad, and make sure the right design variants are used.

⚠️ IMPORTANT All ads and ad sets in the test, including the original variant, must be entirely new to avoid an unfair advantage for old ads that skip the Learning Phase.

4. Open the A/B test tool

All that's left to do is select the ad sets you'll be using in this test (all the ad sets in this new campaign).

With ad sets selected, click the "A/B Test" button (you might have to click "Edit" to see it).

5. Finish the setup

Here it's important to pick "Existing ad sets" for comparison (this should be selected by default, but make sure it is correct).

For the key metric, we recommend using "Cost per result"; however, this can change based on your advertising objectives (i.e., clicks, leads, etc...).

We recommend running the test for a period of 2 weeks MINIMUM and not concluding on tests with less than 30 results in each variant.

Once the A/B test has been set up, just click "Publish Test" to save your changes and begin testing.

Apply learnings

After the testing period ends, compare the key metric(s) between the ad sets to find your winner.

Since all the creatives come from the same catalog, the only variable creating the difference will be the design itself.

After finding your winning design, reflect on which design choices led to higher performance.

You might for example find that the main difference between your best and worst performing design is that the winning variant uses your logo, while the losing doesn't.

In that case, it's a good sign that your target audience responds well to seeing your logo in the ad, and you should be basing your future designs on this learning.