Outbound Segment Sync

Introduction

Outbound Segment Sync allows you to send user segments created and used in MoEngage to any destination at a schedule.

With this feature, you can configure your API endpoint, choose the segments you want to sync, define the sync schedule, and see the users in the segment sync to your destination. You can then utilize the segments to enrich your data warehouses, recommendation systems, and so on. 

Custom Destinations

User segments can be synced to multiple destinations. These destinations can be set up in the App Marketplace -> Custom Destinations section. 

Custom Destinations enables you to connect any endpoint to MoEngage. To enable it for your account, raise a request for Custom Destinations in App Marketplace, as shown in the screenshot below. Once enabled, provide your destination name and add an integration to get started. 

Screenshot_2023-02-15_at_3.28.06_PM.png

Setting up a Custom Destination to sync segments to your endpoint

  1. Provide an integration name to identify your endpoint.
  2. Provide a URL Endpoint to sync the data. Example:  https://api.example.com/segments
  3. Provide a list of key-value pairs that can be added to the headers and parameters as per your requirement. This is optional.
  4. Authentication - Basic auth and No auth are supported.
  5. Choose the user identifier you want to sync from the MoEngage user segment to your endpoint. This should be a key that uniquely identifies a user at your destination. You can rename this key as per the user identifier at your destination. 
  6. Request Body:
      • add_users - This list contains the users to be added to a particular segment.
      • remove_users - This is the list that contains the users to be removed from a particular segment.
      • These keys cannot be removed. However, they can be renamed as per your destination.
      • [Optional] List of Key-Value pairs can be added to the list of add and remove users as per your requirement.
  7. Choose the throttle:
      • MoEngage will send 25000 requests per minute.
      • Each request can have between 100 to 10000 users simultaneously that can be set in the throttle.
  8. Allow resync of all users in the segment to the destination:
      • In case of any issue with a particular sync, your destination might hold incorrect users in the segment. To maintain the accurate state of the users in a segment, you have the option of resyncing all users in the segment.
      • Choosing to allow resync all users will enable the action on each synced segment. However, do ensure that the destination should be able to drop all users in the segment and allow the resync of the entire segment once again.
  9. Request Preview - When you edit the destination configuration, you can view the updated request body that will be sent by MoEngage.
  10. Test your connection by sending a sample user value to the configured destination. You will also be able to see the response - either successful or failed along with the message received from the destination.

ezgif.com-video-to-gif__8_.gif

API Request Format

When MoEngage sends a segment to your API endpoint, the request format will be as below: 

Request Body:

{
  "type": "segment_sync",
  "pagination": {
    "total_pages": 14,
    "page_count": 12
  },
  "data": {
    "moengage_sync_id": "{{sync_id}}",
    "moengage_sync_name": "{{sync_name}}",
    "moengage_execution_id": "{{execution_id}}",
    "moengage_integration_id": "{{integration_id}}",
    "add_users": [
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid99}}"
      },
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid00}}"
      }
    ],
    "remove_users": [
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid98}}"
      },
      {
        "moengage_distinct_id": <moengage_user_id>,
        "partner_user_id": "{{uid01}}"
      }
    ]
  }
}
info

Note

    • While testing the connection, the endpoint should return a 2xx code for success.
    • The response header from the destination should have content-type application/json.
    • For complete API reference of the SDK, refer to the docs in this link.

Setting up a segment sync

To enable Outbound Segment Sync for your account, raise a request in the All Segments -> Synced Segments tab, as shown in the screenshot below. Once enabled, you will be able to add a custom destination as detailed above. 

Screenshot_2023-03-07_at_10.32.49_AM.png

To set up a new synced segment:

Step 1: Choose a custom segment to sync

In the Segment -> All segments page, select a Custom Segment and click on the action Sync Segment in the action dropdown. 

Step 2: Choose the destination

The destinations configured in Custom Destinations in App Marketplace will be available in the dropdown. Choose the destination from the available options or add a new destination for sync.

Step 3: Choose a schedule of sync and set up notification preferences

The sync can be either a one-time transfer of the users in a segment to the destination or a continuous periodic update of the latest status of the segment.

If periodic, segments can be synced daily, weekly, and monthly. Hourly syncs can be enabled on a case-by-case basis. Please connect with your account manager to enable hourly syncs for your account. 

You can also set up email ids to receive notifications of the following:

      • Any update in the sync configuration

      • Successful execution of sync

      • Failure of sync

Periodic Outbound Segment Sync

In periodic outbound segment sync, MoEngage will send the entire users in the segment only in the first sync. For the subsequent syncs, MoEngage follows the change data capture process, where we compare the previous sync to the current status of the segment and send only the difference in the users.

This difference is sent in two lists - add_users (which contains the list of users to be added to the segment) and remove_users (which contains the list of users to be removed from the segment). This difference is also maintained by MoEngage for the action Resync Last Difference, which syncs only the last difference to the destination. This option is provided to address any data discrepancy at the destination.

ezgif.com-video-to-gif__9_.gif

View All Outbound Synced Segments

You can view the list of synced segments in the All Segments → Synced Segments tab. Sync details such as the segment name, destination, schedule, last sync timestamp, latest sync status, and the actions that can be performed on the sync are available on this page.

Actions on Outbound Synced Segments

Action Description

View

Clicking on View will redirect to the selected synced segment’s detail page.

Edit

  1. The Edit action will open the sync's configuration page with options to edit the destination, schedule of sync, and notification preferences that were set earlier.
  2. If a sync is scheduled periodically, the sync frequency can be changed within the periodic options - daily, weekly, and monthly. A one-time sync cannot be changed to periodic sync and vice versa.

  3. The end date of a sync schedule can be edited for periodic syncs.

  4. If the start date of a sync has been crossed, it cannot be edited.

  5. The edit option is available for synced segments that are in active, paused, and failed statuses as detailed here.

Run Adhoc Sync

  1. Clicking on run adhoc sync will manually run the sync to the specified destination on demand.
  2. The run adhoc sync action will send the latest status of users in the selected segment to the destination.
  3. The run adhoc sync option is available for synced segments that are in active, paused, and failed statuses as detailed here.

Pause Sync

  1. Pausing a sync will pause all further syncs of the selected segment. The history of previous syncs is maintained in that synced segment's detail pages.

  2. Running a sync (Run adhoc sync action) will resume the sync of the selected segment.

  3. The pause sync option is available for synced segments that are in active and failed statuses as detailed here.

Resync All Users

  1. Choosing to allow resync all users while setting up an Integration will enable the action on each synced segment.

  2. In case of any issue with a particular sync, your destination might hold incorrect users in the segment. To maintain the accurate state of the users in a segment, you have the option of resyncing all users in the segment.
  3. Please ensure that the destination should be able to drop all users in the segment and allow the resync of the entire segment once again.

Resync Last Difference

  1. To maintain the accurate state of the users in a segment, you can also resync only the last difference of the selected segment to the specified destination.
  2. This option is available only to those synced segments that are in the failed status.

Archive Sync

Clicking on archive sync will stop all further syncs of the segment. A synced segment, once archived, cannot be unarchived.

Sync Status

Each synced segment can be in either one of the following 5 statuses:

Status Description

Active

The latest execution of the sync was successful, and the sync has not expired yet, ie. the end date of the sync has not been reached yet.

Paused

The sync has been paused manually by clicking on the Pause action. Clicking on Run Adhoc Sync will resume all upcoming syncs of that segment. 

Failed

There was a failure in the last run/execution of the sync. All future syncs will be paused if a sync is in a failed state. You can run the sync manually (Run Adhoc Sync) to restart the schedule. 

Inactive

  1. The end time for the sync, as mentioned, has been reached.

  2. Inactive syncs will be archived automatically after 30 days.

Archived

The sync has been archived from the dashboard.

View Details of an Outbound Synced Segment

Clicking on the synced segment name or View action, you will be redirected to the details page of that synced segment.

Sync details such as destination, schedule, latest sync status, sync updated by, updated time, sync schedule, last synced time, next sync time, and a link to the parent custom segment will be available on the detail page.

Sync configurations such as the synced user attribute, sync schedule, and notification preferences are also available on the detail page. 

You can perform all actions associated with a sync from this page - Edit, Run Adhoc Sync, Pause sync, Resync All Users, Resync Last Difference, and Archive sync.

On the details page of a synced segment, you can also view the Sync history of all the sync executions performed for this segment.

Screenshot_2023-03-07_at_10.50.18_AM.png

Sync History

The sync history shows the details of each time the segment has been synced to the destination at the scheduled frequency.

  • Run time - The time at which the sync was executed
  • Status of execution - Each sync execution can be in one of the following statuses -
    •  Complete - The sync ran successfully, and all rows have been sent to the required destination
    •  Processing - The sync is currently running to the required destination
    •  Failed - There was a failure in the execution of the sync.
  • No. of users added - Count of users with <synced user attribute> added to the custom segment since the last time the sync was processed. This list of users is synced to the destination under 'add_users' in the Request Body.

  • No. of users removed - Count of users with <synced user attribute> removed from the custom segment since the last time the sync was processed. This list of users is synced to the destination under 'remove_users' in the Request Body.

  • Total no. of users - This is the number of users with <synced user attribute> present in the parent custom segment.

info

Note

  • You can sync one user attribute from a custom segment to a destination. You can also create up to five syncs daily and maintain up to 5 active syncs in your DB. You can also run up to five syncs daily (either by running the syncs adhoc or running resyncs). The above limits are customizable. Please contact your account manager to get them customized as per your requirement.
  • It will take approximately an hour for archived syncs to show up on the Archived Syncs page.
  • Please be careful while Editing, Renaming, and Archiving a Custom segment in case one or more syncs exist for the same. When the parent custom segment is renamed, the sync name does not reflect the new name of its parent custom segment. 
  • The user count in the custom segment may not always match the users synced from the custom segment to the destination. For example, let's say the count of users in a custom segment is 100, and the sync attribute chosen is the email id. Now assuming email id exists for 90 out of the 100 users in the custom segment, these 90 users will be synced to the destination.
Was this article helpful?
1 out of 1 found this helpful