Agent installation using a CloudFormation quick-create link

This article details how to install a Matillion agent on Amazon ECS Fargate using a CloudFormation template quick-create link. The quick-create CloudFormation link will pre-fill values into the template for you.

Alternatively, you can download the CloudFormation template and edit as needed.

---

Prerequisites

To get started, you'll need:

• A Hub account. To register, read Registration.
• Access to AWS, with the permissions to create a stack on a billable account. You may need an administrator from your organization to either give access or perform this process with you.
• Permissions to provision cloud resources in the AWS environment that the source database is running in.
• An IAM role that has suitable permissions for Data Productivity Cloud. If you're using the same IAM role, you might need to add cloudformation.amazonaws.com to the Service section in the Trust Relationship section of your IAM role to allow the role to work with the ECS service. An example trusted entity can be found below. Read Roles and permissions for more information about creating IAM roles.

You'll also need to allow the IP addresses listed in Allowing IP addresses.

Example trusted entity:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

To learn more about the trust relationship IAM policies, read trust policies.

We recommend that you read the documentation and prerequisites before beginning this process. You may require input from your organization's cloud administrator for access and permissions.

For detailed AWS permission information about deploying an agent in the Data Productivity Cloud, read Agent deployment permissions.

---

Created resources

This template will create the following resources in your AWS account:

• ECS task definition.
• ECS Fargate cluster.
• CloudWatch Log group.
• An entry in AWS Secrets Manager.
• IAM Roles with permissions for the following:
  • Task Roles to your ECS, EC2, and Secrets Manager.
  • Task Execution Roles to pull from ECR Public repositories.

---

Visualization

---

Using the CloudFormation link

1. Create an agent in Data Productivity Cloud.
2. On the Agent details page, note the Credentials: client_id and client secret.
3. At the bottom of the page, click the Run in CloudFormation button. This will launch the CloudFormation stack in the AWS Console with pre-filled values for most parameters.

4. Complete the remaining parameters as follows:

   • Stack name: An arbitrary name given to the created stack. Must be unique. This will be pre-filled, but if you are reinstalling an agent you will need to change the stack name. Consider increasing the number that is added to the end of the stack name.
   • OAuth Client ID: Copy the retrieved client_id from the Agent Details page.
   • OAuth Client Secret: Copy the retrieved client_secretfrom the Agent Details page.
   • VPC Id: Select the ID of an existing VPC. For more information, read VPCs.
   • VPC Subnet Ids: Select at least one VPC Subnet to use.

5. Click Create stack.

Once your stack is ready, this means the ECS service has now successfully deployed with 0 (zero) ECS tasks. This zero-task deployment is deliberate, to prevent fixing the service to a specific container image version, which would prevent the automatic updating functionality from working. The next step is to increase the number of tasks to your requirement.

1. In the AWS Console, locate the service and click Update service.

2. On the Deployment configuration screen, set the Desired tasks count to your normal operational requirements.

   Note

   Ensure that Force new deployment is not selected on this screen.

3. Click Update.

The agent will then start. You can return to the agent dashboard and view the agents list. Your new agent will be on the list and will have a Running status.