Connectivity via AWS PrivateLink using AWS Hybrid Maia agent🔗
Editions
This feature is available to customers on specific editions. Visit Matillion pricing to learn more about each edition.
AWS PrivateLink is an AWS service that allows you to connect services such as Maia to your own AWS virtual private cloud (VPC) via a secure, private connection. Using AWS PrivateLink, no traffic is exposed to the public Internet when it travels between two different VPCs. For further details of the service, read What is AWS PrivateLink?.
Prerequisites🔗
This article assumes you are using Maia in a Hybrid SaaS configuration with a Maia agent running in your own AWS account.
If you are using Maia in a Full-SaaS configuration, read Connectivity via AWS PrivateLink instead.
Note
Use of AWS PrivateLink will incur a cost with AWS. For details, read AWS PrivateLink pricing.
Enabling PrivateLink🔗
If you require PrivateLink to be enabled in Maia, raise a support ticket with Matillion, providing the following information:
- Whether you require Full SaaS or Hybrid SaaS PrivateLink.
- The service name (VPCe). For example,
com.amazonaws.vpce.<region_id>.vpce-svc-xxxxxxxxxxxxxxxxx. - Your Matillion account number. To find this, log in to Maia and click the Profile & Account icon in the bottom-left of the screen. Your account number is the 8-digit number listed next to ID.
Cross-region support🔗
AWS PrivateLink can enable connectivity to the Maia region from a different AWS region. To do this, you need to:
- Configure a VPC in the region in which the endpoint service resides.
- Create an inter-region VPC peering connection from the PrivateLink connected VPC to the remote VPC.
For further details, read What is VPC peering?
Maia will reside in one of the following regions:
- eu-west-1 (eu1)
- us-east-1 (us1)
Set up AWS PrivateLink🔗
Apply a security group🔗
You will need to apply a security group to control who can access the Elastic Network Interface (ENI) and the target application.
Create the VPC endpoint🔗
Note
Before creating the AWS PrivateLink endpoint, you must have created the VPC and subnets you wish to use.
- Log in to the AWS Console.
- Type
VPCin the search bar, and click VPC (it should be the top search result). - Under PrivateLink and Lattice in the left-hand menu, click Endpoints.
- Click Create endpoint.
- On the Create endpoint screen, select Endpoint services that use NLBs and GWLBs.
-
For Service name, enter the appropriate name for your Maia region, as follows:
Region Service name eu-west-1 com.amazonaws.vpce.eu-west-1.vpce-svc-05d76c667b72daf2dus-east-1 com.amazonaws.vpce.us-east-1.vpce-svc-0e24b7e2cd2b24e3f -
Click Verify service and ensure you see a "Service name verified" response.
- From the VPC drop-down, select the VPC in which your Maia agent is located.
- In the list of Subnets, select the VPC subnets that your Maia agent uses.
- Click Create endpoint.
- Copy the DNS names listed under the details of the new endpoint. These will be needed to configure Route 53, as described below.
Configure DNS requirements🔗
Create a hosted zone in Amazon Route 53 and create alias records that point at your VPC endpoints. Use the DNS names that you noted when creating the endpoint, above.
Read Routing traffic to an Amazon Virtual Private Cloud interface endpoint by using your domain name for more details.
The DNS entries used by Maia are:
- For region eu-west-1:
opentelemetry.eu1.privatelink.matillion.comapi.agent-gateway.eu1.privatelink.matillion.com
- For region us-east-1:
opentelemetry.us1.privatelink.matillion.comapi.agent-gateway.us1.privatelink.matillion.com
Authentication🔗
Authentication will be handled by Keycloak at https://keycloak.core.matillion.com, where a token will be generated. This will be the only connection over the public Internet prior to connecting to services over AWS PrivateLink.
Configure the Maia agent🔗
To enable the Maia agent to use AWS PrivateLink, you need to add the environment variable MATILLION_PRIVATELINK_ENABLED = TRUE. This requires you to create a new task revision and restart the Maia agent service. Ensure that there are no pipelines actively using the Maia agent before you begin this process.
- Log in to your AWS console.
- In the AWS console, type
Elastic Container Servicein the search bar, and select that service. - In the left-hand menu, click Task definitions.
- Select the task defintion for your Maia agent and click Create new revision.
-
On the Create new task definition revision screen, under Environment variables, add the following:
Key Value type Value MATILLION_PRIVATELINK_ENABLED Value TRUE -
Click Create.
- Return to Update service.
- Select the latest task definition and click Update.