A/B Test Your Program

Overview

Extole allows you to test and target content, rewards, and rules to motivate varying roles and segments or to compare configuration options. Our goal is to increase overall activity by tuning programs to your segments, new countries, or even new audiences like influencers, advocates, affiliates, and partners.

For more general information, please reference our product overview of A/B Testing.

How to Set Up an A/B Test

Please complete the following steps when configuring your A/B test in My Extole.

1. Log in to your My Extole account and land on the Programs page.

2. Duplicate the campaign you want to test. Find the program for which you would like to run your test. Hover over the campaign you'd like to test and hit the duplicate icon.

   

3. Rename your newly duplicated campaign. Make it clear this is the "variant" campaign in your test. You can preview and make changes in the Campaign Editor and, at this point, your newly duplicated campaign will not be launched.

   

4. Edit your campaign variant and apply the changes you'd like to test. Hit the the Apply Changes button on the top left corner of the editor to save your changes as you work and hit Publish Changes before you want to go live at (this will not launch your campaign).

   
   1. Note: If you are testing a reward offer, please refer to the checklist in our A/B Testing Your Offer article.

5. Launch your test. Once you've published all your changes to the variant in your Campaign Editor, find the menu in the top right corner of the page. From the dropdown, select the option to Go Live.

   
   1. Note: A message will pop up to ensure that you want to Go Live as a Test. Clicking this button means that all other live campaigns within that program will continue to stay live and the campaigns associated with your A/B test will begin splitting traffic. Because you are running a test, you will want this to be the case.

Monitoring Your Test

Once your test is up and running, you will see two (or more) live campaigns running against each other on your programs page. To compare performance, navigate to your program stats dashboard to view your test campaigns side-by-side.

Concluding Your Test

Once your test has run its course, you can choose to either pause your test campaigns or end them.

• Pausing your campaigns means that new customers will not see them, but customers who are already participating (i.e., friends) will still be able to finish their journeys and earn their rewards.
• Ending your campaigns means that no customers will see them, but rewards that have been earned will still be issued. If you are unsure of which route you would like to take, contact your CSM.

You can then work with your CSM to determine the outcome of the test and next steps. Using your program dashboard and Extole reports, you will be able to draw conclusions about your test and which experience performed better according your metrics. For example, if you tested two different Share Email subject lines, you will be able to determine which copy correlated with a higher open and/or click-through rate.

FAQs

How is traffic split between my A test campaign and B test campaign?

For A/B tests, the Extole campaign targeting engine uses a pure random number generator when each user begins their session. No other inputs are considered. This is true for all users who interact with your test, regardless of whether they do so on browser or mobile.

Will customers who have already interacted with my program continue to see the campaign they have already experienced?

If you would like current customers to continue to experience the campaign they have already seen, the steps detailed above will be perfect.

If you would like to ensure that any traffic, new or old, gets split for a test regardless of prior history, you will want to complete the above again to test two new campaigns against each other. You will also want to end your evergreen campaign so that it stops receiving traffic. To do so, navigate to your Programs page and open the menu on the right of the campaign you want to end, then select the End option.

What if I host the promotions I want to test?

If you control the promotions or CTAs you would like to test, we have two options for you.

1. Make your promotions and links generic and set up an A/B test.
2. Recommended: Change your promotion for a limited time and run a burst instead of a test. A burst will pause traffic to your live campaign in favor of a new campaign that features the promotion you would like to test for a limited time. Once your burst is launched, you will want to end your original campaign so that it stops receiving traffic. To do so, navigate to your Programs page and open the menu on the right of the campaign you want to end, then select the End option.

Do I need to re-tag my site in order to run a test?

No. If you have already added Extole tags to site you do not need to do any additional set up in order to run an A/B test.

What if I want to target a specific audience for my test?

This is possible using our Promote to Audience feature if you would like to initiate your test via email. You can select the audience for which you would like to run the test, then send group A to test A and group B to test B. For more information on how to set up this scenario, please reach out to your CSM.