Outbound Segment Sync

Introduction

Outbound Segment Sync allows you to sync and send with ease valuable customer segments created on MoEngage to any other platform in your tech stack.

You can sync these segments across various platforms, including Ad networks, Loyalty platforms, Recommendation engines, Experimentation tools, and more, to drive various business use cases such as driving targeted engagement, making more data-driven decisions, etc.

Outbound

 

Use cases

Brands can leverage Outbound Segment Sync to sync impactful segments across tools, enabling targeted engagement, data-driven decisions, and other business use cases. Here are some examples:

Sync segments to a Data Warehouse

Gain deeper insights and enhance data-driven decision-making by seamlessly syncing segments to your Data Warehouse for in-depth analysis.

Sync segments to a Loyalty platform

Enhance customer loyalty programs by syncing segments, enabling personalized rewards and promotions tailored to individual preferences.

Sync segments to an In-App Personalization platform

Create highly personalized user experiences by syncing segments, ensuring that each user gets content and features that match their preferences.

Sync segments to an Ad network

For precision-targeted advertising campaigns, sync your segments, ensuring your ads reach the most relevant audiences.

Sync customer segments to an Experimentation platform

Conduct A/B/n tests with precision by syncing segments to compare different customer groups' responses and behaviors.

Sync segments to a CRM platform

Improve customer interaction tracking by syncing segments, providing better insights into customer behaviors and preferences.

Sync segments to a Customer Service platform

Provide top-notch personalized support by syncing segments, so your customer service teams can better understand and assist individual customers.

Sync segments from one MoEngage App to another

Using Outbound Segment Sync with our Cohort Sync API makes it easy to sync your segments across multiple MoEngage Apps. Read more about how to set this up.

 

Syncing segments using this feature is quick and straightforward. Just choose your destination platform, select desired user segments, set the sync frequency, and start syncing.

To begin, you first need to set up and connect with the destination platform. There are two ways to do that. Learn more about these in the next section.

Connect Destinations

There are two ways of setting up, and connecting with the destination platform:
1. Using partner integrations
2. Configure custom destination using an API endpoint

Before we deep dive into these two methods, you must request access to enable Custom Destinations. To do so you can navigate to:
App Marketplace -> Click on Explore -> Select Integrate tab -> Request Access

Once enabled, you can get started with any of the two methods below.

Using partner integrations

MoEngage provides preexisting partner integrations that simplify the configuration process and API endpoint testing. With just a few clicks, you can start syncing segments to partner platforms. Check out the partner help docs for more information on these integrations.

Storyly 


Configure custom destination using an API endpoint

To set up a destination using API endpoints, go to the App Marketplace and navigate to the Custom Destinations section. Add a custom destination and configure it.

Screenshot_2023-02-15_at_3.28.06_PM.png

Setting up a Custom Destination to sync segments to your endpoint

  1. Provide an integration name to identify your endpoint.
  2. Provide a URL Endpoint to sync the data. Example:  https://api.example.com/segments
  3. Provide a list of key-value pairs that can be added to the headers and parameters as per your requirement. This is optional.
  4. Authentication - Basic auth and No auth are supported.
  5. Choose the user identifier you want to sync from the MoEngage user segment to your endpoint. This should be a key that uniquely identifies a user at your destination. You can rename this key as per the user identifier at your destination. 
  6. Request Body:
      • add_users - This list contains the users to be added to a particular segment.
      • remove_users - This is the list that contains the users to be removed from a particular segment.
      • Both add_users & remove_users fields are not mandatory in one request: Only one of these can come in the request payload, or both of them can come in the request payload or they can be an empty array as well.
      • These keys cannot be removed. However, they can be renamed as per your destination.
      • [Optional] List of Key-Value pairs can be added to the list of add and remove users as per your requirement.
  7. Choose the throttle:
    • MoEngage will send 25000 requests per minute.
    • Each request can have between 100 to 10000 users simultaneously that can be set in the throttle.
  8. Allow resync of all users in the segment to the destination: 
    • In case of any issue with a particular sync, your destination might hold incorrect users in the segment. To maintain the accurate state of the users in a segment, you have the option of resyncing all users in the segment.
    • Choosing to allow resync all users will enable the action on each synced segment. However, do ensure that the destination should be able to drop all users in the segment and allow the resync of the entire segment once again.
  9. Request Preview - When you edit the destination configuration, you can view the updated request body that will be sent by MoEngage.
  10. Test your connection by sending a sample user value to the configured destination. You will also be able to see the response - either successful or failed along with the message received from the destination.

ezgif.com-video-to-gif__8_.gif

API Request Format

When MoEngage sends a segment to your API endpoint, the request format will be as below: 

Request Body:

{
  "type": "segment_sync",
  "pagination": {
    "total_pages": 14,
    "page_count": 12
  },
  "data": {
    "moengage_sync_id": "{{sync_id}}",
    "moengage_sync_name": "{{sync_name}}",
    "moengage_execution_id": "{{execution_id}}",
    "moengage_integration_id": "{{integration_id}}",
    "add_users": [
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid99}}"
      },
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid00}}"
      }
    ],
    "remove_users": [
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid98}}"
      },
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid01}}"
      }
    ]
  }
}

 

info

Note

  • While testing the connection, the endpoint should return a 2xx code for success.
  • The response header from the destination should have content-type application/json.
  • For complete API reference of the SDK, refer to the docs in this link.
  • Total pages - It defines the number of pages in the “Synced Segment” page which consists of all the previously synced segments.
  • Page count - Page count denotes the page from which the data is currently shown in the payload

.

Definition of IDs in the above payload

  1. “moengage_sync_id": It is a unique ID given to the sync started for a particular segment, which will differentiate it from other synced segments
  2. “moengage_sync_name”: It is the name of the segment that is being synced
  3. “moengage_execution_id": A unique ID will be assigned to the sync execution to differentiate that sync from multiple syncs which is possible for a particular segment
  4. “moengage_integration_id": This ID is generated when a segment is mapped to a particular integration destination. Thus, the mapping of the segment with the integration destination denotes the moengage_integration_id
  5. “moengage_distinct_id”: It is a unique ID assigned to a user by the MoEngage system. This is different from the ID that the customer/partner has assigned to the user
  6. “partner_user_id": This is a unique ID that the customer/partner sends through their system to MoEngage so that they can identify the user as per their conventions

Structure for Key-Value pair

Payload should contain the structure when a key Value pair is added.

Screenshot 2023-11-12 at 1.38.52 PM.png

curl --location --request POST
''
-H 'Content-Type: application/json'
--data-raw '{
"type": "segment_sync",
"pagination": {
"total_pages": 1,
"page_count": 1
},
"data": {
"moengage_sync_id": "sync_id",
"moengage_sync_name": "sync_name",
"moengage_execution_id": "execution_id",
"moengage_integration_id": "integration_id",
"add_users": [
{
"moengage_distinct_id": "moengage_user_id",
"key_format_1": "value_format_1",
"key_format_2": "value_format_2",
"user_attribute": ""
}
],
"remove_users": [
{
"moengage_distinct_id": "moengage_user_id",
"key_format_1": "value_format_1",
"key_format_2": "value_format_2",
"user_attribute": "uid99"
}
]
}
}'

 

Error Descriptions

Sr. No Error Description
1 (attr_name} Type {datatype} attribute not found This error will be shown when the attribute is not present in the system and the user is trying to perform operations using a foreign attribute
2 Exhausted max retries for the segment 3 API call retries are possible where the frequency is 1 retry/3 seconds. If this limit is reached, the system will show the following error
3 An error occurred. Please try again or contact support.

This error might occur if the API call fails multiple times and if the retry is attempted within 30 minutes.

 

Setting up a segment sync

Now that you have already configured your destination platform, you can get started setting up a segment sync.

Trigger

To set up a new synced segment:

Step 1: Choose a custom segment to sync

In the Segment -> All segments page, select a Custom Segment and click on the action Sync Segment in the action dropdown. 

Step 2: Choose the destination

The destinations configured in Custom Destinations in App Marketplace will be available in the dropdown. Choose the destination from the available options or add a new destination for sync.

Step 3: Choose a schedule of sync and set up notification preferences

The sync can be either a one-time transfer of the users in a segment to the destination or a continuous periodic update of the latest status of the segment.

If periodic, segments can be synced daily, weekly, and monthly. Hourly syncs can be enabled on a case-by-case basis. Please connect with your account manager to enable hourly syncs for your account. 

You can also set up email ids to receive notifications of the following:

  • Any update in the sync configuration
  • Successful execution of sync
  • Failure of sync
info

Note

1. There is a cool-off period set in the outbound sync process. The cool-off period is defined to avoid any interruption to the already running segment sync. For eg: If the value of any attribute is changed in the middle of segment sync, it may affect the ongoing sync. Also, the changes made to the value of any attribute will not be reflected immediately while the sync is in process. It will reflect after 30 minutes from the time the changes were made.

2. The Outbound Segment Sync default limit is as follows:

  • The "active" segment sync limit is 10. The various status of outbound segment sync are active, paused, inactive, and failed.
  • You can create 10 segment syncs per day. If the active segment sync limit is reached, you cannot create more segment syncs
  • The segment sync-resync per day limit is 20
  • These limits are configurable, to change the limits you can reach out to support@moengage.com

Periodic Outbound Segment Sync

In periodic outbound segment sync, MoEngage will send the entire users in the segment only in the first sync. For the subsequent syncs, MoEngage follows the change data capture process, where we compare the previous sync to the current status of the segment and send only the difference in the users.

This difference is sent in two lists - add_users (which contains the list of users to be added to the segment) and remove_users (which contains the list of users to be removed from the segment). This difference is also maintained by MoEngage for the action Resync Last Difference, which syncs only the last difference to the destination. This option is provided to address any data discrepancy at the destination.

ezgif.com-video-to-gif__9_.gif

View All Outbound Synced Segments

You can view the list of synced segments in the All Segments → Synced Segments tab. Sync details such as the segment name, latest sync status, destination, schedule, last sync timestamp, and the actions that can be performed on the sync are available on this page.

Actions on Outbound Synced Segments

Action Description

View

Clicking on View will redirect to the selected synced segment’s detail page.

Edit

  1. The Edit action will open the sync's configuration page with options to edit the destination, schedule of sync, and notification preferences that were set earlier.
  2. If a sync is scheduled periodically, the sync frequency can be changed within the periodic options - daily, weekly, and monthly. A one-time sync cannot be changed to periodic sync and vice versa.

  3. The end date of a sync schedule can be edited for periodic syncs.

  4. If the start date of a sync has been crossed, it cannot be edited.

  5. The edit option is available for synced segments that are in active, paused, and failed statuses as detailed here.

Run Adhoc Sync

  1. Clicking on run ad-hoc sync will manually run the sync to the specified destination on demand.
  2. The run ad-hoc sync action will send the latest status of users in the selected segment to the destination.
  3. The run ad-hoc sync option is available for synced segments that are in active, paused, and failed statuses as detailed here.

Pause Sync

  1. Pausing a sync will pause all further syncs of the selected segment. The history of previous syncs is maintained in that synced segment's detail pages.

  2. Running a sync (Run ad-hoc sync action) will resume the sync of the selected segment.

  3. The pause sync option is available for synced segments that are in active and failed statuses as detailed here.

Resync All Users

  1. Choosing to allow resync all users while setting up an Integration will enable the action on each synced segment.

  2. In case of any issue with a particular sync, your destination might hold incorrect users in the segment. To maintain the accurate state of the users in a segment, you have the option of resyncing all users in the segment.
  3. Please ensure that the destination should be able to drop all users in the segment and allow the resync of the entire segment once again.

Resync Last Difference

  1. To maintain the accurate state of the users in a segment, you can also resync only the last difference of the selected segment to the specified destination.
  2. This option is available only to those synced segments that are in the failed status.

Archive Sync

Clicking on archive sync will stop all further syncs of the segment. A synced segment, once archived, cannot be unarchived.

Sync Status

Each synced segment can be in either one of the following 5 statuses:

Status Description

Active

The latest execution of the sync was successful, and the sync has not expired yet, ie. the end date of the sync has not been reached yet.

Paused

The sync has been paused manually by clicking on the Pause action. Clicking on Run Adhoc Sync will resume all upcoming syncs of that segment. 

Failed

There was a failure in the last run/execution of the sync. All future syncs will be paused if a sync is in a failed state. You can run the sync manually (Run Adhoc Sync) to restart the schedule. 

Inactive

  1. The end time for the sync, as mentioned, has been reached.

  2. Inactive syncs will be archived automatically after 30 days.

Archived

The sync has been archived from the dashboard.

 

info

Note

  • When the Segment Sync status is Inactive then you can’t create a new Segment Sync from the same custom segment having the same frequency.
  • When the Segment Sync status is Inactive then you can create a new Segment Sync from the same custom segment having a different frequency/destination.
  • You need to Archive Inactive Segment Sync to create a new segment sync from the same custom segment having the same frequency.

Syncing segments from one MoEngage App to another

Using Outbound Segment Sync and our Cohort Sync API, you can sync segments from one MoEngage App to another. This helps cross-regional teams to function synchronously and have better targeting.

Step 1: Login to your source App

To get started, log in to the app you want to send segments from (source). Ensure that Outbound Segment Sync is already enabled in this app.

Step 2: Create a new Custom Destination

To set up a new custom destination, go to the App Marketplace and navigate to the Custom Destinations section. Add a new custom destination called "MoEngage".

Step 3: Create a new endpoint

In the newly created Custom Destination, click on "+ Add integration" to configure a new endpoint. Give your endpoint a name like your destination app name. Follow the other fields as described - 

Field Value Description

Request Method

POST

 

URL endpoint

https://api-0X.moengage.com/v1/integrations/cohortsync

The 'X' in the API Endpoint URL refers to the MoEngage Data Center (DC). MoEngage hosts each customer in a different DC. You can find your DC number (value of X) and replace the value of 'X' in the URL by referring to the DC and API endpoint mapping here.

Parameters

Leave empty.

 

Authentication

Basic Auth

Username: This is your destination MoEngage App's APP ID.

Password: This is your destination MoEngage App's Data API Key.

Headers

  1. MOE-APPKEY - This is your destination MoEngage account's APP ID.
  2. MOE-DESTINATION-PARTNER - moengage

These two headers are mandatory.

User identifier

 

Select your user identifier. This should be the same for both the source and destination Apps.

Destination identifier

uid

Select the destination identifier as "uid" to send segments to another MoEngage app.

Once the above fields are configured, you can test the connection. If all your details are correct, then the test should be successful. Save the integration.

Step 4: Sync a segment to your destination App

Follow the details above to sync your segments to your destination App.

 

info

Note

  • When syncing segments from one MoEngage App to another, you have to ensure that the users exist in both the source and the destination Apps.
  • Ensure that the identifier of the users is the same across both apps.
  • Currently, it is not possible to sync anonymous users using this method.
  • It is also possible to sync segments across regions.

View Details of an Outbound Synced Segment

Clicking on the synced segment name or View action, you will be redirected to the details page of that synced segment.

Sync details such as destination, schedule, latest sync status, sync updated by, updated time, sync schedule, last synced time, next sync time, and a link to the parent custom segment will be available on the detail page.

Sync configurations such as the synced user attribute, sync schedule, and notification preferences are also available on the detail page. 

You can perform all actions associated with a sync from this page - Edit, Run Adhoc Sync, Pause sync, Resync All Users, Resync Last Difference, and Archive sync.

On the details page of a synced segment, you can also view the Sync history of all the sync executions performed for this segment.

Screenshot_2023-03-07_at_10.50.18_AM.png

Sync History

The sync history shows the details of each time the segment has been synced to the destination at the scheduled frequency.

  • Run time - The time at which the sync was executed
  • Status of execution - Each sync execution can be in one of the following statuses -
    •  Complete - The sync ran successfully, and all rows have been sent to the required destination
    •  Processing - The sync is currently running to the required destination
    •  Failed - There was a failure in the execution of the sync.
  • No. of users added - Count of users with <synced user attribute> added to the custom segment since the last time the sync was processed. This list of users is synced to the destination under 'add_users' in the Request Body.
  • No. of users removed - Count of users with <synced user attribute> removed from the custom segment since the last time the sync was processed. This list of users is synced to the destination under 'remove_users' in the Request Body.
  • Total no. of users - This is the number of users with <synced user attribute> present in the parent custom segment.
info

Note

  • You can sync one user attribute from a custom segment to a destination. You can also create up to five syncs daily and maintain up to 5 active syncs in your DB. You can also run up to five syncs daily (either by running the syncs ad-hoc or running resyncs). The above limits are customizable. Please contact your account manager to get them customized as per your requirements.
  • It will take approximately an hour for archived syncs to show up on the Archived Syncs page.
  • Please be careful while Editing, Renaming, and Archiving a Custom segment in case one or more syncs exist for the same. When the parent custom segment is renamed, the sync name does not reflect the new name of its parent custom segment. 
  • The user count in the custom segment may not always match the users synced from the custom segment to the destination. For example, let's say the count of users in a custom segment is 100, and the sync attribute chosen is the email id. Now assuming email id exists for 90 out of the 100 users in the custom segment, these 90 users will be synced to the destination.

FAQs

  1. Can we remove the 30-minute cool-off period?
     - No, the 30-minute cool-off period cannot be removed. To avoid any interruption to the ongoing sync and avoid chances of incorrect data sync, as a precautionary measure, the cool-off period is set by default.

  2. What does 'API already failed' mean?
     - This error will be shown if the API call is failed but still, a reattempt is made to make the API call. Some possible reasons for the “API already failed” error are:
    • Network connectivity issues: The API call could have failed because there was an issue with network connectivity, such as a weak signal or a server that is down.
    • Lack of authorization or authentication: The API call requires some form of authorization or authentication from the client side, and this could have been missing or invalidated, leading to the API failing.
    • Server Overloading: Too many requests are being handled by the server, causing the server to become overloaded and unable to handle further requests, which could result in an API failure.
  1. Can I change the Sync name to a name of my choice?
     - No, currently there is no provision to edit/change the Sync name
  1. What happens if the Segment name is edited midway, would the name get updated in the API call? and what if another segment of the same name is created? will both the API calls have the same sync name?
     - It is not recommended to perform any kind of action on the segment while sync is going on. It might lead to an interruption in the sync. Meanwhile, any changes made to the segment midway through the sync, the change will not reflect immediately. It will reflect after the cool-off period of 30 minutes. Please be careful while Editing, Renaming, and Archiving a segment in case one or more syncs exist for the same. When the parent custom segment is renamed, the sync name does not reflect the new name of its parent custom segment.
  1. Can I pass more than 1 attribute in Segment sync?
     - No, as per the current MoEngage system, it is not possible to pass more than one attribute in a Segment sync.
  1. Can I personalize the KV pair or have a dynamic timestamp?
     - Customers can personalize the KV (Key-Value) pair to their specific needs. However, please note that at this time dynamic Key Value is not supported, e.g. dynamic timestamp is not supported. Key value pairs need to be static.
  1. What is the retrial mechanism and what is the timeout period?
     - The retrial mechanism allows the customer/partner’s system to call MoEngage APIs in multiple attempts. However, there is a limit assigned to the retrial mechanism. In total 3 retry attempts can be made with the maximum frequency/timeout being 1 retry/3 seconds.

 

Previous

Next

Was this article helpful?
2 out of 2 found this helpful

How can we improve this article?