A/B test catalog ads in Meta

A guide on how to create an A/B test in Facebook.

Table of Contents

Best A/B testing practicesUse 3 (or more) distinct designsGather enough dataDon't use old campaigns or different catalogsHow to A/B test1. Save variants in Confect2. Make a Manual sales campaign3. Create one ad set per design4. Change the designs5. Start the A/B test6. Apply the learnings

One of the most groundbreaking benefits of being able to use multiple designs in a single catalog is that you can A/B test with ease.

With different designs for each product, you can select which creative to use on the ad level.

You can find inspiration for what to test here.

Summary:
 Add multiple design variants to your catalog and create one new ad per tested variant. Run these ads through Meta's A/B test tool.

Steps:
- Add 3+ design variants to your Confect catalog.
- Download our Chrome plugin and log in.
- Create a new Manual sales campaign (just for this test).
- Create one ad set and ad per variant you're testing.
- Switch each ad to a unique variant.
- Use Meta's A/B test tool on this campaign.

Best A/B testing practices

Use 3 (or more) distinct designs

Each design should have different dynamic elements to figure out what works best for your brand. Also, use different layouts and colors (if possible, within brand guidelines).

The goal is to create a broad variation to optimize reach and identify the best-performing design. You will not see a significant change if the designs are too similar.

If this is your first time testing Catalog Ads, and you want to test the difference between the original pictures and Confect designs, make sure to include a "Blank" template (a design that only includes the product).

Gather enough data

Allocate sufficient budget to reach at least 20-50 conversions per design for reliable results.

If your average CPA is 10$ and you're testing 4 designs - you should aim to spend between 800$ and 2.000$ in ad spend for this test (CPA*20-50 conversions*4 designs).

Have equal ad spend across all test designs for fair comparison.

Run A/B tests for 20-25 days to gather the minimum required insights.

Don't use old campaigns or different catalogs

Why not use existing campaigns?

As a campaign runs, it gathers optimization data. If you add new ads to this, the new ads will compete against already optimized ads - giving you highly unreliable results.

Why not use different catalogs?

In the same way as ads, catalogs also gather optimization data. If two separate catalogs are tested against each other, one of them will have more of this data - giving you highly unreliable results.

How to A/B test

1. Save variants in Confect

The first step is to create the individual designs you'll be using for A/B testing, and add them to the catalog as design variants.

If you haven't yet done so, click here and follow STEP 1 of this guide.

Useful tip 💡
If you don't want to affect your existing catalog ads, set the design of "Primary" to a blank design (see example below; including only the product layer).

2. Make a Manual sales campaign

The next step is to create a NEW campaign for this A/B test. Click "Create" in the Ad Manager, select "Sales" and then "Manual sales campaign".

Using a manual campaign allows us to create multiple ad sets and test between them.

Important ⚠️
We highly recommend performing the A/B test using a manual sales campaign.

This article is for "Manual sales campaings". If you want to use an Advantage+ campaign for testing, click here.

3. Create one ad set per design

A new campaign will have one ad set and ad. The ad set should be duplicated (using "Quick duplicate"/ CTRL+D).

Duplicate your ad sets so that you have one per design you're testing.

For our example (1 original, 4 designs), we'll need to duplicate until we have 5 individual ad sets.

Important ⚠️
All ads and ad sets in the test, including the original variant, must be entirely new to avoid an unfair advantage for old ads that skip the Learning Phase.

4. Change the designs

Once you've duplicated the ad sets, you'll need to change the design variants in each ad. Make sure to publish the ads before switching.

It is useful to rename the ads, or ad sets, after the design you're testing.

Go to each ad, and make sure the right design variants are used.

5. Start the A/B test

All that's left to do is select the ad sets you'll be using in this test (all the ad sets in this new campaign).

With ad sets selected, click the "A/B Test" button (you might have to click "Edit" to see it).

It's important to pick "Existing ad sets" for comparison (this should be selected by default, but make sure it is correct).

For the key metric, we recommend using "Cost per result"; however, this can change based on your advertising objectives (i.e., clicks, leads, etc...).

Once the A/B test has been set up, just click "Publish Test" to save your changes and begin testing.

We recommend running the test for a period of 2 weeks MINIMUM and not concluding on tests with less than 30 results in each variant.

6. Apply the learnings

After the testing period ends, compare the key metric(s) between the ad sets to find your winner.

Since all the creatives come from the same catalog, the only variable creating the difference will be the design itself.

After finding your winning design, reflect on which design choices led to higher performance.

You might for example find that the main difference between your best and worst performing design is that the winning variant uses your logo, while the losing doesn't.

In that case, it's a good sign that your target audience responds well to seeing your logo in the ad, and you should be basing your future designs on this learning.