In order to run queries on data from Snowflake, you will first need to connect MoEngage to your Snowflake data warehouse.

Steps to create a new Snowflake Connection

1.  On your MoEngage Dashboard, go to the App Marketplace.
2.  Search for Snowflake.
3. Go to the Integrate tab, and click Add Connection
4.   Provide your Snowflake data warehouse details.

Enter the following fields in Snowflake.

1. Connect Name: Give this connection an identifiable name, e.g., My Snowflake Prod
2. Snowflake Account Identifier
   • It is always found at the beginning of your Snowflake URL, for example; (https://ACCOUNT_IDENTIFIER.snowflakecomputing.com).

   • The format may differ based on Snowflake account age. For details, refer to Snowflake documentation.

3. If you're using Snowsight, you can find your account name at the bottom of the left navigation. Select the account you want to import from and then click on the “copy” icon to copy the Account ID. For up-to-date information and details, refer to Snowflake documentation.
4. Warehouse: The name of the warehouse MoEngage uses to execute the queries. For example, COMPUTE _WAREHOUSE
5. Database: The database that MoEngage should use to read the tables from. You can choose the schema while setting up warehouse segments/imports/exports. e.g., MY_DEMO_DB
6. Username: The database that MoEngage should use to read the tables from. You can choose the schema while setting up warehouse segments/imports/exports.
7. Authentication Method: MoEngage can connect to your Snowflake Instance using two methods:
   • Password: If you select a password, provide the password of the database user (username) you entered above.
   • Key: If you select "Key" as your authentication method. Perform the below-mentioned steps.
   • Click on "Generate Key," and MoEngage will display your public key. You will need to add this public key to your database user. Follow the instructions mentioned on Snowflake help docs to set this up: Key - MoEngage can connect to your Snowflake Instance using the Key pair authentication method. To use this method:
     
   • Once you have added this key to your database user, click on "Test connection," and MoEngage will attempt to verify the credentials.
8. MoEngage also supports the rotating of keys. If your IT Policy requires to rotate the keys, you can do it by following these steps:
   • Go to the App Marketplace > Snowflake > Integrate and edit the connection.
   • Click on the generate new key icon:
   • You will be asked for a confirmation to review. Please read the instructions carefully. Once you save the connection with the newly generated key, MoEngage will no longer use the previous key. You have to ensure that the new key is attached to your database user as the second RSA key (see Snowflake help doc on rotating keys) for the rotation to work properly. If the new key is not attached after saving the connection, your imports and exports might break:
   • Your new key will be generated and shown to you. At this point, it is highly recommended to copy this new key and attach it as an additional key to your Snowflake database user by following the instructions provided on the Snowflake help docs. Once you have attached the new key, you need to test the connection successfully to complete the rotation.
   • If you decide to close the edit form or do not click Connect at any point after generating the new key, your old (existing) key will continue to be used as before, and the newly generated key will be deleted from our system.
9. Role: This role will be used to execute queries from MoEngage.

Once you have set up a Snowflake Connection, you can use it to set up warehouse segments, imports and exports in MoEngage. If your Snowflake instances aren't public and are on AWS, we recommend you set up AWS Private Link with MoEngage AWS account. Please reach out to your account manager for more information on this.

In order to run queries on data from Databricks, you must connect MoEngage to your Databricks warehouse. Ensure, you have administrative privileges in the Databricks platform.

Steps to grant read access to MoEnagage

1. Add a service principal in Databricks
2. Assign a service principal to Databricks workspace
3. Provide data reader access to the service principal
4. Provide compute permissions on SQL warehouses to service principal
5. Create a personal access token (PAT)
6. Fetch the HTTP path and Hostname
7. Connect Databricks on the App Marketplace

Add a Service Principal in Databricks

Account admins can add service principals to the Databricks account using the account console. For more information, refer here.

Perform the following steps to add a service principal:

1. Log in to your account console in Databricks as an admin.
2. In the sidebar, click User Management.
3. On the Service principals tab, click Add Service principal.
4. Enter a name for the Service principal.
5. Click Add.

Assign a Service Principal to Databricks Workspace

As an admin, assign a service principal to a workspace using the account console. For more information, refer here.  

Perform the following steps to assign permissions:

1. Log in to your account console in Databricks as an admin.
2. In the sidebar, click Workspaces.
3. Click your workspace name.
4. On the Permissions tab, click Add permissions.
5. Search for and select the service principal, and assign the permission level (Workspace User). You can assign the workspace admin role to the workspace admin settings page.
6. Click Save.

Provide Data Reader Access to The Service Principal

Provide data reader access, as shown below to the entire workspace or individual schemas to the service principal, ensuring that the tables you want to query in MoEngage are available in the schema.

Provide Permissions for Service Principal to Compute on SQL Warehouses

Perform the following steps to provide compute permissions:

1. In the Databricks dashboard, Click Settings > Compute > SQL warehouses

2. Select the warehouse that needs to be accessed by the service principal.

3. Click Permissions. In the list, select Can use for the service principal.

Create a Personal Access Token (PAT)

You can create a PAT for a service principal using Databricks CLI. For more information, refer here.

Fetch the HTTP Path and Hostname

Perform the following steps to fetch the HTTP path and Server hostname, which MoEngage requires to access your Databricks workspace. 

1. In your Databricks workspace, select SQL warehouses.
2. Choose your warehouse.
3. Click the Connection details tab and copy the hostname and HTTP path provided, which is required for MoEngage.

Connect Databricks on the App Marketplace

To connect Databricks on the App marketplace, perform the following steps:

1. On the left navigation menu in the MoEngage dashboard, click App marketplace.
2. On the App Marketplace page, search for Databricks.
3. Click the Databricks tile.
4. On the Databricks page, go to the Integrate tab and click +Add Connection.
5. Enter the following details:
   

   Field	Required	Description
   Connection name	Yes	Type a name for the Databricks connection.
   Host name	Yes	This refers to the unique identifier assigned to a specific cluster. Type the hostname that you want to connect. To find your server hostname, visit the Databricks web console and locate your cluster. Then, click to reveal Advanced options and navigate to the JDBC/ODBC tab.
   Port	Optional	Type the port to which you want to connect your Databricks server. It defaults to 443.
   HTTP path	Yes	Type the HTTP path of your compute resource on Databricks. To find your HTTP path, go to your Databricks workspace, locate your warehouse, and then get your HTTP path from the connection details tab.
   Access token	Yes	This is an authentication token used to access Databricks APIs securely. Type the Access token, which helps you make authorized requests to the Databricks server with the necessary permissions. On your Databricks workspace, go to Settings > Developer > Access tokens > Manage tokens.
   Catalog	Yes	Type the Databricks Catalog name to which MoEngage will have access.

   
   
6. Click Connect. Your Databricks connection is now integrated.

After you have set up a Databricks connection, you can use it to set up various imports and exports in MoEngage.

To run queries on data from Redshift, you must connect MoEngage to your Redshift warehouse and ensure you have administrative privileges on the Redshift platform.

Steps to create an IAM role and Provide Access to MoEnagage

Step 1: Create an Identity and Access Management (IAM) Role in Your Account

1. Sign in to your AWS account in the AWS Management Console.
2. In the Find Services search box, search for IAM. You will be directed to the IAM dashboard.
   
3. In the IAM dashboard, select the Roles tab.
4. Click Create role.
5. Select AWS account.
   
6. Under the Select trusted entity section, select Custom trust policy.
7. Paste the Trust Policy JSON obtained from MoEngage, as shown below. MoEngage's AWS Account ID is 612427630422. Ensure this ID is present in the trusted relationship JSON. 
   

   {
   "Version": "2012-10-17",
   "Statement": [
   {
   "Effect": "Allow",
   "Principal": {
   "Service": [
   "redshift.amazonaws.com",
   "ec2.amazonaws.com"
   ]
   },
   "Action": "sts:AssumeRole"
   },
   {
   "Effect": "Allow",
   "Principal": {
   "AWS": "arn:aws:iam::612427630422:root"
   },
   "Action": "sts:AssumeRole"
   }
   ]
   }

8. Select Allow assuming roles from this account option. 
9. Click Next > Permissions.
10. ctvtvgd4t Add the permissions mentioned below:

    redshift_finegrained (customer-managed policy)

    
    {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Sid": "DataAPIPermissions",
    "Action": [
    "redshift-data:ExecuteStatement",
    "redshift-data:CancelStatement",
    "redshift-data:ListStatements",
    "redshift-data:GetStatementResult",
    "redshift-data:DescribeStatement",
    "redshift-data:ListDatabases",
    "redshift-data:ListSchemas",
    "redshift-data:ListTables",
    "redshift-data:DescribeTable"
    ],
    "Effect": "Allow",
    "Resource": "*"
    },
    {
    "Sid": "Redshiftminimal",
    "Action": [
    "redshift:Get*",
    "redshift:Describe*"
    ],
    "Effect": "Allow",
    "Resource": "*"
    },
    {
    "Effect": "Allow",
    "Action": "iam:CreateServiceLinkedRole",
    "Resource": "arn:aws:iam::*:role/aws-service-role/redshift.amazonaws.com/AWSServiceRoleForRedshift",
    "Condition": {
    "StringLike": {
    "iam:AWSServiceName": "redshift.amazonaws.com"
    }
    }
    }
    ]
    }

    stsAssumerRole (customer-managed policy)

    {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Sid": "123",
    "Effect": "Allow",
    "Action": [
    "sts:AssumeRole"
    ],
    "Resource": [
    "*"
    ]
    }
    ]
    }

11. You can optionally restrict access to specific db-groups and users by selecting those options in the policy editor. 
    
12. Click Next.
13. In the Add tags section, click the Add new tag button to add the tags required for this IAM role.
14. Click on Next > Review.
15. In Role name box, enter the name for your role (for example, moengage-redshift-data-access-role").
    
16. Click on Create IAM role.
    

Step 2: Share Details with MoEngage

Share the following details to warehousesegments@moengage.com to get warehouse segments on Redshift enabled on your workspace:

• ARN (Amazon Resource Name):
  • In your AWS Management Console, go to the IAM dashboard and select the IAM Role that you created in step 1.
  • Copy the ARN for the IAM role.

    MoEngage will use this ARN to assume the IAM role in your account to access the Redshift cluster.

    

• Database name: The default Redshift database name is used for the queries that will be running. It is recommended that separate connections be created for each database.
• Redshift Cluster Identifier name: It is recommended that separate connections be created for each provisioned Redshift cluster.

  #{
  "role_arn" : "arn:aws:iam::5822XXXX730:role/Segmentation_Redshift",
  "aws_regions" : "ap-south-1",
  "cluster_identifier" : "redshift-segmentation-cluster",
  "default_db" : "dev",
  }