Within the platform, you can use the Split Randomly action to create equally distributed paths that your contacts will randomly pass through, so you can test how well your flows work. We also provide a handy simulator in the flow editor so you can test flows as you build them.

Outside the platform, we encourage you to send real messages to measure flow performance and conduct A/B tests.

A/B testing (also called split or multivariate testing) is a method for testing the effectiveness of two or more versions of a single message, flow or campaign against a desired outcome. It's a type of controlled experiment conducted on similar audiences, with the initial version as the control and any additional versions as treatments.

Getting Started

• Define objectives: What are you trying to achieve with your flow: distributing information, collecting data or something else?

• State your hypotheses: Which version of your flow do you think will perform best, and why?

• Create a list of the things you want to test: For example, flow length, message length, vocabulary and method of engagement (will the contact start the flow with a keyword or phone call, or will the flow start on a schedule?).

Conducting an A/B test

Test one item at a time

Let's say version A addresses your contacts by name while version B doesn't. When testing two versions of a flow, make sure you're only measuring the impact of one change at a time. If version A is too different from version B, you'll have trouble isolating the variable responsible for the result.

Send the flow to similar representative samples

Show version A and version B to similar groups, and make sure each is representative of the entire target population. If you show version A only to experienced contacts and version B only to new contacts, your results may be skewed. Ideally, you want to split your target population evenly.

Be patient

One of the best things about the platform is how quickly you can deploy. You can sign up, connect a channel and send messages in as little as 5 minutes, but that doesn't mean everyone should take that approach. Since SMS is as much an extension of an organization's brand as email, larger deployments should run tests to make sure their flows are well received and driving results.

Creating flow versions

To create a new version of a flow, click the ☰ menu icon in the flow editor and select Copy. This makes an exact copy of the flow for you to edit and rename.

You can link your versions using a Split Randomly action. In the Reminder Test flow below, the split sends each contact down branch A or B to receive one of two reminder styles, saves the branch as Reminder Style, and adds the contact to the Reminder A or Reminder B group.

Evaluating the results of an A/B test

Along with the main measure of success you chose when you set your objectives, the flow list shows Runs, Ongoing and Completion for each flow. Completion is the percentage of contacts who entered the flow and finished it. Comparing the completion rates of two versions of a flow is the easiest way to measure their effectiveness.

The Activity column lets you visually compare how your contacts have interacted with each flow recently (over the last 14 days).