AWS Data Exports

MoEngage now allows you to generate a file with your events present in MoEngage on a daily basis and export this file to your own S3 location. The format of the exported csv file would be like - 

Screen_Shot_2020-02-24_at_7.51.42_PM.png

The filename that we place in the S3 location would be - S3_Location/MoEngageAppName/YYYY-MM-DD/file_id where file_id is an alphanumeric string.

A sample filename would be like s3://dataexports.bucket/MoEngageAccountName/2020-01-20/0b408e30-2d67-46c6-a9aa-3efcb8d46e74.csv

There are 3 steps to set this up for your MoEngage Account -

  1. As of now, to setup automated daily data exports, you would need to contact your Customer Success Manager or write a note to support@moengage.com

  2. You would need to share a list of events, user attributes, device attributes and campaign attributes that you would want to see in your daily exports.

  3. You would need to setup your own S3 bucket so that MoEngage can place the file there every day by following the instructions specified in the next section.

 

S3 Integration for Data Exports

Amazon S3 is a highly scalable storage system offered by Amazon Web Services. Follow the instructions on this page to get started with your AWS S3 integration which will allow MoEngage to share a daily export of required data in your S3 bucket.

 

Create S3 Bucket

If you already have an S3 bucket, we still recommend creating a new bucket specifically for MoEngage so you can limit permissions.

  1. To create a bucket for your app, open the Amazon S3 console and follow the instructions to Sign in or Create an Account with AWS.

  2. Once signed in, select “S3” from the “Storage & Content Delivery” category.

  3. Select “Create Bucket” on the next screen and you will be prompted to create your bucket and select a region.

 

Authentication

This authentication method generates a Secret Key and an Access Key ID that enables MoEngage to authenticate as a user on your AWS account for the purposes of writing data to your bucket.

 

Step 1 - Create User

To retrieve your Access Key ID and Secret Access Key, you’ll need to create an IAM User and Administrators Group in AWS .

 

Step 2 - Get Credentials

Once a user has been created, click “Show User Security Credentials” so your Access Key ID and Secret Access Key will be revealed. Next, either note these credentials somewhere, or click the blue “Download Credentials” button as you will need to share this with MoEngage later on.

 

Step 3 - Create Policy

Now, navigate to the Policies tab in the navigation bar and select “Get Started” then “Create Policy”. This will allow you to add permissions for your user. Select “Create Your Own Policy”. This will give limited permissions so we only have the ability to access the bucket that you specify.

Input the code below when creating your own policy. Specify a “Policy Name” of your choice, and input the code below into the “Policy Document” section. Be sure to replace INSERTBUCKETNAME with your own bucket name.

{

"Version": "2012-10-17",

"Statement": [

{ "Effect": "Allow",

"Action": ["s3:GetBucketLocation"],

"Resource": ["arn:aws:s3:::INSERTBUCKETNAME"]

},

{ "Effect": "Allow",

"Action": ["s3:PutObject", "s3:ListBucket"],

"Resource": ["arn:aws:s3:::INSERTBUCKETNAME*", "arn:aws:s3:::INSERTBUCKETNAME/", "arn:aws:s3:::INSERTBUCKETNAME"]

}

]

}

 

 

Step 4 - Attach Policy

Once your policy has been created, navigate to “Users” and then click into your specifc user so you can attach this new policy. On the “Permissions” tab, click “Attach Policy”.

Search for the new policy that you created and click to attach.

You are now ready to link your AWS credentials to your MoEngage account.

 

Share the following with MoEngage -

  1. Access Key ID - Your S3 Access Key ID

  2. Secret Access Key - Your S3 Secret Access Key

  3. Bucket Name - Your S3 Bucket Name

  4. Folder path - The path to your S3 bucket folder

 

 

Step 6 - Whitelisting MoEngage IPs

If you need to whitelist specific IPs from MoEngage to write data to your bucket, you could use the following IPs - 52.4.7.54, 52.4.221.49, 52.6.221.107, 52.6.248.125, 52.7.27.232, 54.84.198.141, 18.195.110.23

 

 

 

Frequently Asked Questions

1. How soon will I start getting the data in my S3 bucket?
A. It typically takes up to 7 days for the automated data exports to be set up for your account and for you to start seeing data there.

2. Can I export additional events at a later time?
A. Yes, if you need to export new events at a later time, you can always contact our team and get the new events added to your exports.

3. Can i export historical data using Automated Data Exports?
A. As of now, you cannot export data prior to configuring Automated data exports. Once configured, you will start seeing data for each data in your S3 bucket.

4. Is there a pricing involved for Automated Data Exports?
A. If your billing plan does not cover this already and you need to get this enabled at a later time, you would need to work with your Customer Success Manager to get this included in your billing plan. Based on the amount of data that MoEngage will be placing on your S3 bucket on a monthly basis, this could lead to additional charges.

Was this article helpful?
1 out of 1 found this helpful