Experimentation is the key to marketing. Marketers use A/B or Multivariate tests to test how small changes in campaigns can impact results in a big way. One can then plough back the learnings from one experiment into the next campaign and build on it.
A/B experiments are conducted to test one variable. Multivariate experiments can be done across multiple ‘variables’ and one can compare how one variation performs better for a given conversion goal. As customers will be served either of the variations, their engagement with each experience is measured and collected to be shown as analytics. If possible, you should use Control Group to create a baseline. You can go as low as 2-5% for your control group allocation.
You can start experimenting with your messages:
1. Emojis
Sample Hypothesis: Having an Emoji in a message gives better CTRs
Experiment: Use Emoji in Variation 1 and send a plain message in Variation 2
Measure: Compare CTRs of Variation 1 & Variation 2 . If CTRs of Variation 1 are better then there are high chances you have a valid hypothesis.
2. Personalization
Sample Hypothesis: Using the customer’s First Name or Full Name results in better CTRs
Experiment: Use First Name attribute-based personalization in Variation 1 and create a plain message in Variation 2
Measure: Compare CTRs of Variation 1 & Variation 2 . If CTRs of Variation 1 are better then there are high chances you have a valid hypothesis.
3. Action Buttons
Sample Hypothesis: Using action buttons allows results in higher click-throughs
Experiment: Use notification action in Variation 1 while using only the default notification click in Variation 2. Set the conversion goal as per what conversion event you want to track.
Measure: Compare CTR for Variation 1 & Variation 2 . If the CTR of Variation 1 is better than Variation 2, there are high chances you have a valid hypothesis.
4. Images
Sample Hypothesis: Using images in Android push notifications results in higher click-throughs
Experiment: Use an image in Variation 1 while do not use any image in Variation 2
Measure: Compare CTRs of Variation 1 & Variation 2 . If CTRs of Variation 1 are better than that of Variation 2 then there are high chances you have a valid hypothesis.
5. Coupon Code
Sample Hypothesis: Sending Coupon Codes with push notifications result in higher conversions
Experiment: Create an experiment where Variation 1 is using the Coupon Code while Variation 2 is not using it.Set the conversion goal as per what conversion event you want to track.
Measure: Compare conversions of Variation 1 & Variation 2. If conversions of Variation 1 are better than 2, there are high chances you have a valid hypothesis.
6. Landing pages
Sample Hypothesis: Taking new customers to any deal page results in a higher conversion rate
Experiment: Create an experiment where Variation 1 is using the discount code of 10-15% OFF and taking the user to the Home page while Variation 2 does not serve any discount code but takes users to your deals page.
Measure: Compare conversions of Variation 1 & Variation 2. If conversions of Variation 2 are better than 1, there are high chances you have a valid hypothesis.
You can similarly experiment:
- if using a custom sound gets you more user attention resulting in higher CTRs
- if using a carousel of images gets more clicks in comparison to a single image
Do let us know what else are you experimenting with.