Skip to content

Connectivity via AWS PrivateLink

Editions

This feature is for customers on our Enterprise edition only. Visit Matillion pricing to learn more about each edition.

AWS PrivateLink is an AWS service that allows you to connect to an AWS virtual private cloud (VPC) via a secure, private connection. Using AWS PrivateLink, no traffic is exposed to the public Internet when it travels between two different VPCs. For further details of the service, read What is AWS PrivateLink?

The Data Productivity Cloud can use AWS PrivateLink to:

  • Enable a secure integration with Snowflake.
  • Create a secure connection to your AWS-hosted data sources, for example Redshift or other data sources such as an RDS database.

The instructions in this document assume you are using the Data Productivity Cloud in a Full-SaaS configuration. If using Hybrid SaaS with a Data Productivity Cloud agent hosted in your own AWS account, read Connecting agents via AWS PrivateLink.

Prerequisites

Before configuring PrivateLink to Snowflake, ensure the following:

  • You are using a Business Critical Snowflake edition.
  • Your Snowflake account is hosted in one of the following Matillion supported AWS regions:
    • eu-west-1 (eu1).
    • us-east-1 (us1).

Configure the connection to Snowflake

  1. Snowflake must enable AWS PrivateLink connectivity on your account before you can use it with the Data Productivity Cloud. For this, you must contact Snowflake support and open a support request. In your message, include the following details:

    • Matillion’s AWS Account ID: arn:aws:iam::926494931119:root.
    • Your Snowflake account.

  2. Run the following SQL query within your Snowflake environment to retrieve your PrivateLink account configuration:

    SELECT SYSTEM$GET_PRIVATELINK_CONFIG();

  3. Share the full JSON output from the above query with your Matillion account representative. This will have a format similar to the following:

    {
    "privatelink-account-principal": "arn:aws:iam::xxxxxxxx:root",
    "regionless-snowsight-privatelink-url": "app-xxxxxxxx.privatelink.snowflakecomputing.com",
    "privatelink-account-name": "xxxxxxxx.eu-west-1.privatelink",
    "privatelink-vpce-id": "com.amazonaws.vpce.eu-west-1.vpce-svc-xxxxxxxxxx",
    "snowsight-privatelink-url": "app.eu-west-1.privatelink.snowflakecomputing.com",
    "regionless-privatelink-ocsp-url": "ocsp.xxxxxxxx.privatelink.snowflakecomputing.com",
    "privatelink-account-url": "xxxxxxxx.eu-west-1.privatelink.snowflakecomputing.com",
    "spcs-registry-privatelink-url": "xxxxxxxx.registry.privatelink.snowflakecomputing.com",
    "app-service-privatelink-url": "*.dubavn.privatelink.snowflake.app",
    "regionless-privatelink-account-url": "xxxxxxxx.privatelink.snowflakecomputing.com",
    "spcs-auth-privatelink-url": "sfc-endpoint-login.dubavn.privatelink.snowflakecomputing.com",
    "privatelink_ocsp-url": "ocsp.xxxxxxxx.eu-west-1.privatelink.snowflakecomputing.com"
}

  4. Matillion will use this information to configure a PrivateLink interface endpoint in the Data Productivity Cloud environment, and establish a secure connection to your Snowflake instance.

  5. Matillion will confirm when the PrivateLink connection is available for you to use, and provide you with a PrivateLink host address.
  6. Once the setup is complete, you will be able to connect to your Snowflake account from the Data Productivity Cloud by creating an environment configured to use the PrivateLink. When configuring the environment, you will need to use the PrivateLink host address provided to you by Matillion. This ensures that your Snowflake traffic is routed through the PrivateLink endpoint rather than the public Internet.

Connectivity to AWS-hosted data sources

To connect to AWS-hosted data sources such as Redshift or RDS, you can use AWS PrivateLink to create a secure connection between the Data Productivity Cloud and your AWS VPC. The data is queried over PrivateLink, and processed securely within the Data Productivity Cloud platform.

The Data Productivity Cloud supports AWS PrivateLink for the following AWS-hosted sources:

  • Amazon Redshift
  • RDS
  • Databases hosted on EC2
  • AWS MSK

PrivateLinks are service specific, so you will need to create a separate PrivateLink for each AWS-hosted service you wish to connect to. For example, if you want to connect to both Redshift and to another data source hosted in your AWS VPC, you will need to create two separate PrivateLinks following the instructions given below.

Prerequisites

Before configuring AWS PrivateLink:

  • An AWS VPC endpoint service must be configured and associated with the target destination.
  • If the destination is hosted on an AWS-managed service, you must provision a Network Load Balancer (NLB) in your VPC. The NLB receives requests from the Data Productivity Cloud and routes it to the destination. To create an NLB, follow the instructions in the AWS documentation.
  • Security groups and routing tables must be correctly configured to permit traffic from the PrivateLink endpoint to reach your backend target. Health checks on the NLB should reflect the readiness of your destination service to avoid dropped connections.

Configuring the connection to your AWS services

  1. In AWS, create a Network Load Balancer (NLB) in each Availability Zone where your service is deployed. Ensure it's configured to route traffic to your target service (for example, EC2 instances or ECS tasks).
  2. In AWS, create a VPC endpoint service and associate it with the NLBs you've provisioned. This endpoint service will expose your AWS application over PrivateLink.

  3. Grant access to the Data Productivity cloud by allowing the following AWS account to connect to your endpoint service:

    arn:aws:iam::926494931119:root

  4. Note the Service Name generated by AWS for your endpoint service. It will follow this format:

    com.amazonaws.vpce.<region>.vpce-svc-xxxxxxxxxxxxxxxxx

  5. Contact your Matillion account representative to request that PrivateLink be enabled on your Data Productivity Cloud account. Matillion will need to know the Service Name you noted above.

  6. When the Data Productivity Cloud initiates the connection, you will receive a confirmation request on the Endpoint Connections page of the AWS console, under the VPC service. When you receive the confirmation, you must accept the connection request. If you've enabled automatic acceptance in the endpoint service settings, this step is not required.

You may also wish to explore the Data Productivity Cloud's other secure connectivity features, including:

© Copyright 2024 Matillion Ltd | Legal | Privacy Policy | Disclaimer | Modern Slavery Statement