Create an A/B test broadcast

An A/B test broadcast sends two or three versions of a message, called variations, to your audience so you can see which one performs better. This article covers how to choose the A/B Test broadcast type, set up your variations, and pick message types and channels for each one.

Before you start

• A/B Test must be turned on for your account. If you don't see the Broadcast type option on the Details step, it isn't enabled.
• A/B Test isn't available on non-passthrough accounts.
• To turn on A/B Test, contact Mobile Commons Support.

Choose the broadcast type

You choose the broadcast type on the Details step when you create a new broadcast.

1. Go to Broadcasts and create a new broadcast.
2. Enter the Broadcast Name and select a Campaign.
3. Under Broadcast type, select A/B Test. Standard is selected by default.
4. Click CREATE & CONTINUE.

Nothing else on the Details step changes when you select A/B Test. The broadcast type is saved when you click CREATE & CONTINUE. After that, the type is locked and the Broadcast type options are disabled.

Tip: To reuse an A/B test, clone the broadcast. The clone keeps the broadcast type, variations, messages and A/B test settings. The clone starts as a Draft, and it doesn't copy the original's results or winner.

Create and manage variations

On the Conversation step, each variation appears as a tab above the message editor. Two variations are created for you: Variation A and Variation B. An A/B test needs at least 2 variations and allows up to 3.

Each variation holds exactly one message. The editor shows "The conversation ends here" below it, and you can't add a second message. Changes you make in one variation don't affect the others.

To manage a variation, open the three-dot menu (⋮) on its tab. The menu has Edit Name, Duplicate and Delete.

You can edit the Conversation step only while the broadcast is Draft or Scheduled.

Add a variation

1. On the Conversation step, click + VARIATION to the right of the variation tabs.

   

2. Write the message for the new variation.

3. Click SAVE & CONTINUE.

The new tab opens automatically. It takes the first unused letter. For example, if you delete Variation A and then add a variation, the new one is named Variation A.

When you have 3 variations, + VARIATION is disabled. Delete a variation to turn it back on.

If a variation has a problem, such as a blank message, a banner at the top of the step shows how many errors each variation has. Fix them before you continue. In this example, the third variation also shows + VARIATION disabled.

Duplicate a variation

Duplicating is the quickest way to test a small change, such as different wording, an emoji, or a different link or call to action.

1. Make sure the variation you want to copy is saved.
2. On that variation's tab, open the three-dot menu (⋮).
3. Select Duplicate.
4. Edit the copy.
5. Click SAVE & CONTINUE. If you don't, the duplicated content isn't kept.

The copy includes the original's message content. It's named after the original with "copy" added, for example Variation A copy. A second copy of the same variation is named Variation A copy 2. The copy is added as the last tab, and a confirmation message appears.

Note: A variation must be saved before you can duplicate it. When you have 3 variations, Duplicate is disabled and shows "Variation limit reached."

Rename a variation

Give each variation a name that describes what's different about it, such as "Friendly with emoji" or "Short version." Clear names make your results easier to read.

1. On the variation's tab, open the three-dot menu (⋮).
2. Select Edit Name.
3. In the Name your variation dialog, enter a name in the Variation Name field.
4. Click SAVE.

Variation names follow these rules:

• Up to 100 characters.
• Each name must be unique in the broadcast. Capitalization doesn't count, so "Short version" and "short version" are the same name. If the name is taken, you see "Name already exists."
• Emojis and question marks are removed from the name.
• The name can't be blank.

Click CANCEL to close the dialog without changing the name. After you save, a confirmation message appears.

Delete a variation

You can delete a variation only when the broadcast has 3 variations, because an A/B test needs at least 2.

1. On the variation's tab, open the three-dot menu (⋮).
2. Select Delete.
3. In the Are you sure? dialog, click YES, DELETE IT. Click CANCEL to keep the variation.

Deleting removes the variation and its message permanently. The other variations and the broadcast's settings aren't affected, and the remaining tabs keep their names. For example, if you delete Variation B, you're left with Variation A and Variation C.

Preview a variation

Each variation has its own mobile preview to the right of the editor. The preview updates as you type. The highlighted tab shows which variation you're viewing.

1. Select a variation tab.
2. Check the preview for text, spacing, emojis, personalization fields, links and media.
3. Repeat for each variation.

When every variation looks right, click SAVE & CONTINUE to go to the Recipients step.

Choose message types and channels

Each variation's message can use any type from the standard broadcast menu, and each variation can use a different type:

• Send a message
• Send as RCS
• Multiple Choice Question
• Tell a Friend
• Trigger an mData
• Ask to Call In
• Advertise a Keyword
• Trigger an Email (if enabled for your account)
• Request a Donation (Mobile Giving accounts only)

You can also mix channels across variations: SMS, MMS, and RCS in Basic, Enhanced, Rich Card or Carousel format. RCS needs an active RCS agent on your short code.

Important: The winner is decided by replies and clicks on the variation's message. Actions that happen afterward, such as a friend opting in, a phone call, or a donation, don't count toward the winner.

Your channel choices affect which winner metrics you can use. Click-through Rate needs an mCommons shortened link in every variation, and it's unavailable if any variation is RCS.

How fallback works for RCS and MMS variations

Some phones can't receive RCS or MMS. When that happens, mCommons sends a fallback message instead:

• RCS variations: the recipient gets an SMS or MMS version of the message.
• MMS variations: the recipient gets an SMS with a link to the image. To send only to MMS-capable phones, select Suppress SMS Failover under the attachment. Recipients who can't receive MMS then get no message.

Standard RCS broadcasts have a Fallback step where you write and preview the fallback message. A/B test broadcasts don't have this step, so you can't write or preview a custom fallback for an RCS variation.

Fallback messages count toward that variation's results. The RCS variation report shows how many SMS and MMS fallback messages were sent.

Tip: Keep your RCS message text meaningful on its own. Recipients whose phones can't receive RCS get an SMS or MMS version instead.

Tip: Expect RCS and MMS results to mix channels. Check the variation report's fallback counts to see how many recipients got the fallback.

Related topics

• A/B testing overview
• Configure A/B test settings
• Understand A/B test winner metrics
• Schedule and test an A/B test broadcast
• Monitor results and send the winner