Testing the effectiveness of your messages

Use A/B testing to track message variations in order to determine which messaging approach is most effective.

Set up two separate messages (A and B) to send to two randomly selected groups of recipients from the list(s) or smart segment(s) you select. Track the results with Waterfall reports to determine which message was more effective in achieving your goal.

For example, if you want to generate more revenue from sales, set up A/B testing for two separate coupon codes. Send the first code (20% off storewide) to group A and the second code (40% off a single item) to group B. When customers redeem their coupons, you can measure to which coupon variation achieved better results.

For information on sending a broadcast message without using A/B test groups, see Building a new broadcast.

Setting up A/B split testing

To set up broadcast messages using A/B testing:

1. From the Broadcasts menu, select Send.

   

2. To define the subscribers who will receive your message, select List or Smart Segment from the dropdown list, and then select a list or smart segment from the adjoining drop-down list.

3. Select Add.

   The broadcast form expands, adding the subscriber group below the List or Smart Segment field. The right panel lists previously sent messages and upcoming scheduled messages.

   If you selected a smart segment, the segmentation slider is set to On and the segmentation options are added to the form.

   Depending on your audience selection, you can continue to add lists or smart segments, which will stack below the List or Smart Segment field.

   Note: You cannot mix lists and smart segments in your audience selection. If you change your selection, you will see a confirmation message. Select Yes to clear the previous selection.

4. Select the delivery date:

   • To send the message immediately, leave the Send field blank.

   • To schedule a message, specify a Send date and time:

     1. Select Calendar and select a date.

     2. Select Time at the bottom of the calendar and use the up and down arrows to select a time.

     3. Select Calendar.

     4. (Recommended) Optionally, select the Send in subscribers' local time checkbox.

        Note: The subscriber's local time is determined by their profile data, including the area code for their mobile phone number.

   • To reset the send date and time, select Clear.

5. Review the list(s) of subscribers who will receive the message.

   • To remove a list or smart segment, select Delete at the end of the row.

   • To refine a group of recipients in a list, drag the Segmentation slider to select ON and then specify the segmentation options for the subscribers in the selected list. For more information, see Building a new smart segment.

   

6. To test your message using A/B test groups, select Add A/B Split. The broadcast form is divided into two separate test groups, representing a 50/50 split for your test groups.

   

7. For each message, select the percentage of recipients in the selected list(s) or smart segment(s). If you need to adjust the messages to ensure a 100% total:

   • To add a message, select Add A/B Split again.

   • To remove a message, in the lower-right corner, select Delete .

   Note: The percentages must add up to 100 percent.

8. To base one or more messages on a standard broadcast template, under the text box, select a template from the Select Template dropdown list. The message is added to the text box. You can edit it as needed. For more information, see Working with templates.

9. For each message, select SMS (plain text message) or MMS (multimedia message, including images, videos and audio) for the message, and then enter your message in the text box.

   Tip: Be sure to add the opt-out disclosure (for example, "Reply STOP 2 quit") to your broadcast message.

   Note: MMS messaging is available in the United States and Canada only.

10. For each message, select a messaging option from the dropdown list:

    • Mobile Message sends the SMS or MMS message to the specified recipient list according to the schedule you select.

    • Forward to campaign sends the SMS or MMS message as part of the campaign you select from the dropdown list. Use this option when you require a response or multiple messages.

    • No message grays out the message box, so you cannot add a message. Use this option to do the following without sending a message:

      • Tag smart segments with metadata values.

      • Generate counts of users in smart segments.

      • Define audience criteria and A/B groups.

11. When you are done, select Review Message.

    

    You see a preview of your message, plus the schedule for sending, number of recipients, and guidance on improvements you may want to make.

    Note: If you have added a header, the details you selected (name, mobile, and/or company) appear at the top of the message.

12. (Recommended) Select Send test message, select a segment from the drop-down list, and then enter one or more mobile numbers, separated by , (comma). Select Send Test.

    

13. When you see a notification that the test message has been sent, select Done. Review your test message on your mobile device(s).

14. To continue working on your message, select Go Back, make final revisions to your messages, schedule, and recipient list, and then select Review Message to double-check your changes.

15. When you are satisfied with your message, select Send Now or Schedule.

    

    Your test messages are added to the Scheduled Messages table until they are sent.