Analyze Multivariate Experiments

Analyze the performance of the A/B or Multivariate experiment in the campaign analytics page by navigating to the MoEngage Dashboard > All Campaigns Page > Campaign > Analytics.

Push

View the analytics where you can compare variation performance, platform-wise. The column with the crown represents the Winner variation - the variation with maximum Click Through Rate.

Screen_Shot_2016-09-15_at_7.27.02_PM.png

You can also view a graphical comparison:

Screen_Shot_2016-09-15_at_6.32.04_PM.png

Email

You should be able to see analytics as below where you can compare variation performance and deduce the best performing variate.

VariationPerformance.png

You can also view a graphical comparison:

VariationPerformance_Graph.png

In-App Message

The following describes the analysis results terms:

Best Variant 

This denotes the best-performing variant of your campaign. If the user distribution for the experiment is static then MoEngage will leverage the click-through rate of each variant to declare a winning variation. If the distribution is dynamic then MoEngage will leverage the selected optimization metric to declare a winning variation.

Chances to beat all other variations

This denotes to chances of each variation beating all other variations. This is again calculated basis Click Through Rate for Static user distribution and the basis of the selected optimization metric for dynamic distribution.

Uplift

This denotes the uplift in primary conversion goals of the campaign as compared to your control group. This metric will inform you if the campaign is successful or not. Please read this article for more information on how we measure uplift.

Onsite Message

You should be able to see the analytics below, where you can compare variation performance and deduce the best-performing variation.

SherpaTablewithData.png

You can also view a graphical comparison:

SherpaGraphwithdata.png

SMS

You can see the breakdown of performance metrics (Clicked, Delivered, Conversions, Conversion Goals) across variations, allowing you to assess and compare the effectiveness of the variations. You can choose the metrics for which you wish to see the comparison in the metrics dropdown and filter using them by locale or variation. When you choose to group by variation, the performance of each metric in all locales is grouped by the variation. Hover over the bar chart to understand how each locale has performed for the specific metric in a variation. You can switch to the tabular view by clicking on the table option beside the graph.

Variation Performance_SMS (1).png

Previous

Next

Was this article helpful?
0 out of 0 found this helpful

How can we improve this article?