Skip to content

Agent installation using a CloudFormation quick-create link

This article details how to install a Matillion agent on Amazon ECS Fargate using a CloudFormation template quick-create link. The quick-create CloudFormation link will pre-fill values into the template for you.

Alternatively, you can download the CloudFormation template and edit as needed.

Prerequisites

To get started, you'll need:

  • A Hub account. To register, read Registration.
  • Access to AWS, with the permissions to create a stack on a billable account. You may need an administrator from your organization to either give access or perform this process with you.
  • Permissions to provision cloud resources in the AWS environment that the source database is running in.
  • An IAM role that has similar permissions to a Matillion ETL role. If you're using the same IAM role, you might need to add cloudformation.amazonaws.com to the Service section in the Trust Relationship section of your IAM role to allow the role to work with the ECS service. An example trusted entity can be found below. Please read IAM roles and permissions for more information about creating IAM roles.

You'll also need to allow the IP addresses listed in Allowing IP addresses.

Example trusted entity:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

To learn more about the trust relationship IAM policies, read trust policies.

We recommend that you read the documentation and prerequisites before beginning this process. You may require input from your organization's cloud administrator for access and permissions.

For detailed AWS permission information about deploying an agent in the Data Productivity Cloud, read Agent deployment permissions.

Created resources

This template will create the following resources in your AWS account:

  • ECS task definition.
  • ECS Fargate cluster.
  • CloudWatch Log group.
  • An entry in AWS Secrets Manager.
  • IAM Roles with permissions for the following:
    • Task Roles to your ECS, EC2, and Secrets Manager.
    • Task Execution Roles to pull from ECR Public repositories.

Visualization

CloudFormation visualization

  1. Create an agent.
  2. On the Agent details page, note the Credentials: client_id and client secret.
  3. At the bottom of the page, click the Run in CloudFormation button. This will launch the CloudFormation stack in the AWS Console with pre-filled values for most parameters.

  4. Complete the remaining parameters using this table:

    Parameter Description
    Stack name An arbitrary name given to the created stack. Must be unique. If you are reinstalling an agent, you will need to change the stack name. Considering incrementally upping the number that is added to the stack name when using the CloudFormation quick-create link.
    OAuth Client ID Copy the retrieved client_id from the Agent Details page.
    OAuth Client Secret Copy the retrieved client secretfrom the Agent Details page.
    AWS VPC Select the ID of an existing VPC. For more information, read VPCs.
    AWS Subnets Select at least one VPC Subnet to use.

  5. Click Create stack.

Once your stack is ready, you can return to the agent dashboard and view the agents list. Your new agent will be on the list and have a Running status.