Overview

A control group is a subset of the users you target with a specific campaign, who do not receive the campaign communication. Its primary purpose is to act as a baseline for comparison. By measuring the performance of this group alongside your "target" groups (which do receive the change), you can accurately determine the true impact and effectiveness of your marketing efforts.

What Constitutes the Control Group?

• The control group consists of users who do not receive a notification, serving as a baseline for comparison.
• The Active Target count in the control group includes users who are eligible for notifications but are intentionally excluded from receiving them.
• Impressions for the control group are always zero, as no notifications are sent.
• Conversions represent the count of users in the control group who completed a conversion event.
• The conversion rate for the control group is calculated as Conversions (CVR) divided by Active Targets.

Why Do You Need the Control Group?

As a marketer, you often face the dilemma of what should be communicated to customers and what is futile. You would specifically want to know, "Is the communication I am sending to my customers adding value to them or my business? Will I be able to fulfill my objectives without sending that communication?" Here is where marketing research experts suggest using the control group and the target group.

The primary purpose of a control group is to provide a baseline for comparison. By comparing the behavior and conversion rates of the target group (which received the campaign communication) with the control group (which did not), you can more accurately measure the incremental impact or uplift of their campaigns. Uplift measures the difference in performance between the target and control groups and is calculated to determine the communication's impact (positive or negative). This helps determine if the marketing activity itself caused a change in behavior or if the change would have happened organically.

You can check the campaign's impact on uplift, whether customer engagement and conversions would be better or worse if the specific campaign had not been sent to some customers. While no communication is being sent to the control group, MoEngage tracks the conversions for the users in this group.

Types of Control Groups

You can create two types of control groups:

Global Control Group

The global control group is a strategically selected subset of the entire user base that is intentionally excluded from receiving any marketing campaigns across all available channels. An administrator can configure it at the workspace level. For more information, refer to Global Control Group.

Campaign Control Group

A campaign control group is defined at the individual campaign level. It comprises a subset of users who specifically qualify for a particular campaign but are intentionally withheld from receiving its communications.

Campaign control groups are independent of the global control group. You can enable or disable campaign control groups regardless of whether your admin has enabled or disabled the global control group for your workspace.

Use Control Groups in a Campaign

You can create a campaign using the global control group, the campaign control group, or both. During campaign creation, the Global control group and/or the Campaign control group toggles are available in the control group section of step 1, Content.

Add the Global Control Group to a Campaign

If your admin has enabled the global control group for your workspace, the Global control group toggle is turned on by default in the control group section. That means the users in the control group will not receive the communication from this campaign.

If your admin has disabled the global control group for your workspace, the Global control group toggle is turned off by default. As a marketer, you cannot enable it at the campaign level. Contact your admin to enable it for the workspace (account) first.

If your admin has selected the Allow marketers to send campaigns to these users check box while enabling the global control group, you can turn the Global control group toggle off. When you turn this toggle off, MoEngage will send campaigns to the users in the global control group.

Add a Campaign Control Group

At the campaign level, you can add a dedicated control group to strategically restrict communication to a defined subset of users.

To add a campaign control group, perform the following steps:

1. Turn the Campaign control group toggle on to exclude the users in the control group from the campaign.
2. In the box below, specify the percentage of users from the target audience to be excluded from the campaign.

Channels That Support Campaign Control Group

Uplift

Uplift in campaign analytics refers to the increase in the desired metric (such as conversion rate, revenue, or customer engagement) that can be attributed to a specific campaign or action.

It is a way to measure a campaign's effectiveness or impact by comparing the conversion goal performance for users who received your campaigns (Target Group) to that of users who were not sent the message (Control Group).

Uplift can help you in the following ways:

• Analyze campaign performance (from impressions to conversions) with multiple attribution models like View-through attribution (x hours from impressions), Click attribution (x hours from clicks), and In-session attribution.
• Create and validate experiments using multivariate testing.
• Measure true uplift on the business metric against a control group.

Control Group Uplift

Control group uplift refers to the increase or decrease in the desired metric observed within the control group when comparing it with the target group. In campaign analytics, a control group is a group of individuals who do not receive the campaign or action being analyzed, while the target group refers to those who are exposed to the campaign. By comparing the results of both groups, you can determine the incremental impact of the campaign on the desired metric. The control group uplift helps evaluate the effectiveness of the campaign by considering the difference between the target and control groups.

Global Control Group Uplift

Global control group uplift refers to the uplift observed when analyzing the results of the entire customer base or a large sample of customers who were not exposed to the campaign. In this case, the entire customer base or a significant portion of it acts as the control group, while the target group consists of customers who were targeted with the campaign. The global control group uplift provides a broader measure of the campaign's impact across a larger audience, helping to assess the overall effectiveness of the campaign.

Positive and Negative Uplift

An uplift (control group uplift or global control group uplift) can be positive or negative. A positive uplift indicates that the campaign or action had a positive impact and led to better results compared to not running the campaign or taking that action. On the other hand, a negative uplift indicates that the campaign or action had a negative impact and led to worse results compared to not running the campaign or taking that action.

Both positive and negative uplift provide valuable insights into the effectiveness of campaigns and help you optimize your strategies.

Calculate Uplift Percentage

View the Uplift in Campaign Analytics

On the campaign analytics page, you can view uplift under the following analytics for each campaign:

• Campaign performance stats: The uplift percentage in performance stats indicates the primary conversion goal and the uplift of that primary conversion goal against the control group.
  
• Locale/Variation performance: The uplift percentage in Locale/Variation performance indicates the uplift of each conversion goal for each variation or locale against the control group.
  
• Conversion goal performance: The uplift percentage in performance stats indicates the uplift of each conversion goal against the control group.
  

FAQs 

• How are users distributed between the target and control groups in MoEngage?
  In MoEngage, users are distributed between target and control groups based on the percentage specified during the campaign setup. Before the distribution takes place, MoEngage also excludes users who are ineligible to receive notifications due to factors such as frequency limits, communication preferences, or reachability constraints. This ensures that the right users are included in each group for effective testing and measurement of campaign performance.
• Why does MoEngage use different baselines for calculating conversion rates (impressions for the target group and active targets for the control group)?
  MoEngage utilizes different baselines for calculating conversion rates to provide precise reporting on the impact of notifications on desired actions, such as purchases or successful transactions. By using impressions for the target group, MoEngage assesses the impact of notifications that were successfully delivered to users. Conversely, by using active targets for the control group, MoEngage evaluates the typical behavior of users who did not receive notifications.

  This distinct approach in baselines is crucial for accurately determining the uplift generated by the push notification campaign. It allows for a clear comparison between the behavior of users who received notifications and those who did not, enabling more accurate insights into the campaign's performance and effectiveness.