How do we measure Uplift?

Overall campaign Uplift is measured to gauge the impact of campaign i.e. how your message variations (one or more) helped you achieve the primary conversion goal over your control group to which campaign was not served. It compares the conversion goal performance for users who received your message (Test group) to that of users who were not sent the message (Control Group).

On your campaign analytics page, you can see Campaign Uplift as below:

When 1 variation is present with control group

When 2 or more variations present with control group:

A natural question is why, in our calculations, we have used different denominators - Impression (for Variation) and Active Targets (for Control). It is because of the fact that any experiment intends to compare the audiences in test that were exposed to the treatment vs to those who weren't. In this case, the treatment is "Receiving Push Notification." We are effectively comparing the behavior of users to convert when they received the notification vs. when they didn't.

We also measure improvement for other individual goals to identify how particular variation is performing with respect to your control group.

In above example, improvement for Variation 1 for Goal: sry is 50%. This is how we calculate it.

In absence of control group, we also measure how each variation is performing with respect to your variation 1. The winner (represented by crown icon) is declared on basis of best conversion rates for primary goal among variations.



Have more questions? Submit a request

Comments