Intelligent Path Optimizer allows you to A/B test up to 5 branches, where Sherpa automatically keeps adjusting the user distribution toward the best-performing path. You can focus on experimenting with which channel to use or what should be the messaging frequency in each branch and let Sherpa optimize the user distribution.
While deputizing Sherpa to optimize the user distribution, you must also configure the metric you are expecting to maximize in your flow. This can be one out of Engagement, Conversion, or both.
| info |
Worthy Alternative If you wish to handle the user distribution manually, you can opt for A/B Split. Read more about it here. |
Setup
You can Add an Intelligent Path optimizer at any part of your flow where you want to experiment with some worthy alternative options. You can find the option to add this in the Control section of the Flow stages.
Make sure you name each branch distinctly to identify them on the canvas. At every stage, you can experiment with up to 5 flow paths, and in each flow, you add up 5 such stages.
Maximized metric selection
MoEngage Flows are used to solve a variety of use cases that can be generalized into two main categories.
- For increasing user engagement
- For increasing user conversions
So while optimizing a flow, we depend on your selection of metrics to be maximized in order to align the optimization to the Flow Goals.
How does this work?
Initially, the users get uniformly distributed, and as we start getting more and more performance results, sherpa keeps adjusting the user distribution towards the best-performing branch. Based on which metric is configured to be maximized, we track different performance stats of each branch involved in the experiment.
- Maximizing Conversions
For each branch, we aim to maximize the ratio of converted branch trips/total trips of the branch. This can be loosely termed as the Branch CVR as well.
warning Things to notice
We refrain from using total branch conversions as the maximized metric as that would give preference to users doing multiple conversions on a branch and would also favor a branch having more action campaigns. Both necessarily would not mean that the branch that eventually wins is the most optimized one.
We believe that our approach relies on finding the optimized branch based on the most common user behavior leading to conversion. - Maximizing Engagement
Similar to the conversion calculation, for each branch, we aim to optimize the total engaged trips instead of the total engagement. To do this, we give an engagement score to each branch and try to maximize that score instead of the total click or email opens of the branch.
The engagement score for the branch is 100 times the number of branch trips that have fetched a click from at least one of the action campaigns(BTC) + one-tenth the number of branch trips that have only fetched email opens(BTO) divided by the Total Branch Trips.
Engagement Score = ((BTC+ 1/10(BTO))/Total Branch Trips)*100
info Help us
The engagement score allows us to optimize for the best-performing branch, which might not necessarily be the branch with the most clicks or most email open.
If this approach does not suit your use case or expectation, please reach out to us at product@moengage.com - Maximizing the combination of Engagement and Conversion
For each branch, we combine Branch CVR and Engagement score with emphasis on branch CVR as we believe fetching conversions is more important than generating engagement. So the Maximized metric, in this case, becomes Branch CVR + 1/10( Engagement Score)
Sherpa will distribute maximum users toward the best-performing branch at the time of user reaching the path optimizer stage.
| check_circle |
Keep Experimenting Sherpa will not stop distributing users towards low-performing branches even when a branch is identified to be performing better, as the best-performing branch may keep changing from time to time as more and more users go through the flow. |
Analyzing performance
Each flow branch's performance keeps updating on the flow canvas as more and more users go through it.
- A branch can be identified by its branch name. Email is the branch name in the above example.
- 4 done is the number of branch trips in the above example, and these 4 trips constitute 66.67% of the total trips reaching the intelligent path optimizer stage.
- The converted trip count is the number of branch trips that lead to at least one conversion from one of the action campaigns in the branch.
- The engagement score of the branch is also shown for each branch, irrespective of which metric is maximized for the branch.
| check_circle |
What is in a branch?
|
To analyze and compare the performances of all the branches, you can check out the report for your Split node. You can either click on the report option that appears on hovering over the stage on canvas or go to the split node stats section in the flow's detailed stats page and filter on a particular split node from the dropdown.