Control Group In Flows


Control Group in MoEngage Flows is the percentage of total user entries that you configure that would not receive any of the configured action campaigns of the flow. 

Initially, restricting some of the users entering the flow from receiving any of the messages may seem counter-intuitive at first but by not delivering any message to Control Group (CG) users we establish the baseline of the organic behavior of the Targetable audience by tracking the behavior of the CG users after they enter the flow. The flow's performance when compared to the baseline established by the control group helps the marketers validate the below assumptions with which they publish a flow:

  1. The configured flow can only lead to higher conversions.

  2. There is a need for engaging the users with the current flow. 

If the conversion ratio(CVR) of the control group is higher than that of the flow that would mean that users are organically more likely to convert than after receiving the campaigns from the configured hence the assumptions enlisted as not valid that flow and hence the flow stop not be continued. 

However, in case the CVR of control is less than that of the flow, then the efficacy of the flow can be understood by calculating the conversion uplift that flow has fetched concerning the control group. 

Conversion Uplift

The uplift analysis is the most common way to analyze the extent to which a MoEnagege Flow has nudged your users toward a conversion. Uplift is measured against the organic conversions from the control group users and conversions of the users who did receive at least one of the action campaigns of the flow. 

Uplift = [(Flow CVR - Control group CVR )/Control Group CVR] x 100




    A 50% conversion uplift signifies that the configured flow has fetched 50% more conversions than what would be possible had the Flow never been configured.

Understanding Uplift Percentage

The uplift calculation is dependent on the stage and the timeframe in consideration. Thus, the upliftment for the flow shown at the top of the Detailed Stats section might vary from the upliftment shown in the Engagement Trends in the Detailed stats as the latter is calculated only for the chosen Channel while the former is calculated for the entire flow.


Let's take a Flow that has a 36-hour attribution window and has been configured for a Push and an Email campaign.
Default scenario of 36 hr attribution window:

  •  Hour 0 - User Enters flow (90 TG, 10 CG)
  • All of them are sent the push campaign Push-1, assuming all of them have received (100% impression/view-through)
  • 0 to 24 hours - 5 users (4 TG, 1 CG) convert. These users are attributed only to the push campaign Push-1.
  • Hour 24 - Email campaign Email-1 is sent out, assuming all of them have opened the email (100% impression/view-through)
  • 24 to 36 hours - 8 users (8 TG, 0 CG) convert. These users are attributed to both the push campaign Push-1 and the Email campaign Email-1.
  • Hours 36 to 60 - 5 users (4 TG, 1 CG) convert. These users are attributed only to the email campaign Email-1. 
The uplift percentage is calculated as shown below for this example.

Uplift Percentages and CVR at the Child Level




TG CVR = 12/90 = 13.3%

CG CVR = 1/10 = 10%

Uplift = 33%


TG CVR = 12/90 = 13.3%

CG CVR = 1/10 = 10%

Uplift = 33%

The Uplift Percentage at the Flow level

  • Total TG converted users = 16 / 90 = 17.7%
  • Total CG converted users = 2 / 10 = 20%
  • Uplift = -11.5%

How to configure Control Group?

The option to add a Control Group is available in the Entry Condition node as well as on the flow setting page. The control group is enabled by default and can be changed anytime during the flow lifecycle. The users will become part of the control group only after the flow is published with the control group setting enabled. 


Control group users are randomly selected out of the total users entering the trip out of all the users in the Flow's target audience. 

In order to maintain the sanctity of the experiment, we ensure that once a user enters the flow as part of the control group, they would continue to be part of the control in all their subsequent entries into the flow until the control group is removed. You can choose to disable this if required.

How Control Group users are allocated?

Due to the scale at which MoEngage operates, we split users randomly into subgroups and apply the control group split percentage to each of the groups on a daily basis. The above process is made with the assumption that an equal number of users can be allotted to the different subgroups on any given day. So there is a need to have a minimum number of users to enter the flow for the split to be maintained. Also, with time the system scales down/up the number of users allotted as CG depending on the condition so that the % of users in CG is always close to the defined value.

When marketers check the “Users part of Control Group should continue to be part of it for their subsequent entries”(as shown in the above image), the system makes sure that a user marked as CG will always be a CG user. So, if CG users repeatedly enter the Flow more than the other type of users, then the % between CG trips and overall trips is going to be greater than the defined value. However, the % split on the unique user level will be as defined.

For example, let's take the case where a marketer defines a Flow with a 20% Control Group split and wants “Users part of the Control Group should continue to be part of it for their subsequent entries”. Let users U1, U2, U3, U4, and U5 enter 1,2,3,4,5 times, respectively. Let U5 be marked as the Control Group. In this situation:

Total Trips = 15
CG trips = 5
However, the CG split has been maintained at 20% as only 1 in 5 users was marked as the Control Group.

How big should the control group be?

You can make the control group as big as 95% or as small as 1%, depending on your target audience. For example, if you expect 100k users to enter the flow then 1% CG would be enough. The smaller the target group the larger the control group should be configured to effectively measure the efficacy of the flow.

How do Control Groups work?

Users are randomly allocated to the control group at the time of entering into the flow. The control group users after entering the flow traverse through the flow as per their behavior during the trip and eventually exit the flow. These users are not sent any campaign during their flow trip.

Conversion tracking for the Control group

Similar to the flow conversion tracking, control group users' conversion tracking starts as soon as they move past any one of the action campaigns configured in the flow. We assume that these users have received the campaign and we start tracking their conversion for the configured attribution window. As with the non-control group users also, conversions performed by CG users are attributed to the action campaigns as well as the flow.



Since the control group users will skip all the action campaigns, we get an interesting problem "What should the conversion attribution window be for the flow in such a case?" We can try one of the following solutions:

  • Approach 1: Calculate the longest path of the flow and keep waiting for that much period for control group users to convert after entering the flow. But this assumes that all the control group users who had not been part of the control group would have gone traversed the flow through the longest path, which seems very unlikely.
  • Approach 2: Wait for the attribution window period for control group users to convert after entering the flow. This approach gives an unfair advantage to the control group users if the configured flow is long. For example, if the largest path of the flow takes 3 days to traverse and exit, and if the attribution window of the flow is 24 hours. Then, a control group can enter and convert for the flow approximately thrice before the flow user completes one trip through the longest path.
  • Approach 3: We let control group users traverse through any of the possible paths based on their behavior after they enter the flow and continue to track the conversions. After they exit the flow from one of the nodes we track the conversion for an additional period that is equal to the attribution window for the flow. This way we let the control group user behavior govern the period till which the conversion should be tracked.

We identified the third approach to be the least assumptive and most intuitive. If you have any concerns or suggestions please reach out to

Editing the control group

While Marketers can choose to change the Control group variation at any time during the lifetime of the flow, we recommend against it as changing the traffic distribution may invalidate the experiment results. 

For Example, let's say you publish your flow with a 50% control group on Day 1 and 2000 user entries happen on that day. 1000 Flow user entries result in 250 converted trips and 1000 control group(CG) user entries fetch 200 conversions. Hence, the CG CVR is 20% whereas flow CVR is 25% which is a 25% uplift from the control group.

Now the next day if you change the CG variation to 10% and 1000 user entries happen on the next day. Now out of 900 regular flow entries, 45 lead to at least one conversion( i.e. become converted trips), and out of 100 CG user entries only 4 converts. Now for Day 2, CG CVR is 4% and Flow CVR is 5% which is again a 25% uplift from the CG. 

Now if combine the Day 1 and Day 2 results you will get CG CVR as 18.55% [(204/1100)x100) whereas the flow CVR during the same period drops down to 15.52% [(295/1900)x100] which gives an overall negative uplift of -16.33%. 

Even though Flow CVR is better on both days individually, the Control group wins the experiment. Changing the CG distribution has eventually altered the results of the experiment, this in statistical theory is known as Simpson's Paradox. Because of this, we strongly recommend fixing the control group variation before publishing the flow and not changing it unless you absolutely have to or you have validated the efficacy of that flow over time.

Removing the control group

If you are observing conversion uplift for your flows over a significant period of time, we recommend you disable the Control group setting for your flow. Once this is done, all the users entering the flow can potentially receive the action campaigns configured in the flow. 

If the Control group users re-enter the flow, they will also be eligible to receive the action campaigns. The list of previous control group users is also purged, so if the control group is added again, the CG user distribution will start from scratch without any influence from the previous control group.  


What happens if I remove the Control group by accident?

In such a scenario, there is no way to restore the earlier configuration, but you can choose to add the CG again and restart, analyzing the flow performance.


Finding the users marked as the control group

Once you have removed the control group, you might want to find them to target the users that were part of the control group and did not receive any of the flow action campaign events after going through the flow. 

Users Marked as the Control Group for the Flow

You can create a segment with all the users who have performed the "User entered flow" event with the Flow Id Attribute value being the flow-id of your flow, with the "with control group" attribute as "true" and "Control Group Type" as Flow.


Users Marked as the Global Control Group 

You can create a segment with all the users who have performed the "User entered flow" event with the Flow Id Attribute value being the flow-id of your flow, with the "with control group" attribute as "true" and "Control Group Type" as Global.



Was this article helpful?
4 out of 4 found this helpful

How can we improve this article?