Mobile Commons: How Do I A/B Test?

The Mobile Commons platform includes an A/B testing tool to allow you to test content engagement more easily. There is a lot of nuance to effective A/B testing within text messaging so be sure to work with your mobile strategist before starting your A/B Test.   

To A/B test in the mobile commons platform, you should first think of what metric of engagement you’ll use. If you are testing web clicks, be sure to create two different shortened URLs to accurately track clicks unique to each message. Additional points of engagement could be a response to a question, like a multiple-choice, or response to an information request, like asking for an email.

After you’ve determined the content or sets of content you wish to test you’ll need to create your groups. First, select the entire universe of your send; this might be all of your active subscribers, just those in a specific campaign, or any other filtered group. Then click “A/B Test” under Group Actions. We recommend that your test groups contain the size of subscribers that is statistically relevant to your test and database size.

After you’ve created your groups you’ll set up different broadcasts to the test groups and target each message to the group you want to test. Then based on the performance you see in the broadcast report you can send the winning message by cloning it and sending it to the final group.

Create your group, then select "A/B Test" 

Then, create your test groups

Determine how many test groups you want in the field "Number of test groups", then how big each group should be by determining the number of users in each test group in the field "Size of each test group".

Note that the profiles in each test group will be randomly selected from the main group you started with. Make sure you are using statistically significant sizes (in comparison with your total subscriber size).

There are a lot of great & free a/b testing calculators available online to help you determine the right size.

You'll then be on a page with all the groups you created. You can target your testing messages to the test groups and then, based on message performance, replicate the winning message and target the final group.

When you are ready to get started with A/B Testing email your mobile strategist or drop a line to mc_support@uplandsoftware.com and we’ll help you get set up!